Real-time computer vision for game automation
A bot that reads a live game screen with computer vision, finds the cursor, enemies and loot, plans a path, and carries out combat and looting without manual control.
What this project is
A client wanted a bot for the game The Lord of the Rings Online (LOTRO) that could play the repetitive parts on its own. I built it with OpenCV so that it reads the live game screen, finds the cursor, enemies and lootable objects, and decides what to do next.
It handles path navigation, combat and looting from start to finish, so a character can keep progressing through routine tasks with nobody at the keyboard.
What needed solving
Repetitive combat and looting can take hours that players would rather spend on the interesting parts of the game. Watching a fast-moving screen for the right cursor state or the next target is also tiring to keep up for long.
Reading the screen with computer vision and driving navigation and combat automatically removes that burden. The same real-time vision and input-automation approach applies to other repetitive, screen-based tasks that would otherwise need constant attention.
What I did
- Built the bot end to end in Python.
- Wrote the vision step that reads each captured frame and detects the cursor, enemies and lootable objects.
- Added path and target planning so the next move is chosen automatically.
- Connected the decisions to automated input so sequences run without manual control.
How it works
Real-time screen capture
The bot continuously captures the game screen and applies vision algorithms to detect the cursor, enemies and lootable objects.
Path and target analysis
NumPy and scikit-image process each frame to work out the best path and pick the next combat or loot target.
Automated input
PyAutoGUI turns the decision into movement, combat and looting inputs, and the sequence runs without manual control.
Tools and methods
What came out of it
| Measure | Result |
|---|---|
| Cursor and object detection accuracy | 95% |
| Combat and looting sequences automated | 500+ per week |
| Repetitive path navigation effort | Reduced by 80% |
| Gameplay without human input | Extended sessions, fully autonomous |
Interested in similar work?
Have repetitive, screen-based work that a computer could do for you? Send me a message and let's talk.
👋 Let's talk