Computer Vision Android Emulator Bot

A bot that uses live reading pixels to handle state, with a FSM controlling the phase of actions. CV with a locally trained model for loot collection on 3d space.

Technologies
pythonpillowcomputer vision

DISCLAIMER: I do not endorse or encourage cheating, but I do encourage the exploration of difficult topics in the least-destructive ways possible.

If you are going to study this discipline I recommend:

Minecraft + local Paper server + Mineflayer. This is probably the closest thing to a practical bot-development laboratory. Run a Minecraft Java server locally or in Docker, then connect programmable clients using Mineflayer. Mineflayer exposes movement, world/entity state, inventory, combat, digging, pathfinding, chat, etc. It is explicitly designed for programmable Minecraft clients and supports localhost/offline setups.

https://github.com/PrismarineJS/mineflayer

That lets you intentionally create things such as:

  • deterministic aim
  • reaction-time bots
  • pathfinding bots
  • farming bots
  • combat bots
  • inventory automation
  • perfect timing
  • input macros
  • multiple coordinated agents
  • abnormal movement patterns
  • humanized/randomized behaviors

For more advanced ML/CV approaches, I would look into https://docs.unity3d.com/Packages/com.unity.ml-agents@4.1/manual/index.html

This project was made years ago for the purpose of exploring bots, which I have always been curious about ever since I was a child playing MMOs. I learned about Computer Vision, python, windows libraries & automation, machine learning, and state machines.