search

JartanFTW / kot-skipper

Automatically find certain gems in King of Thieves
Apache License 2.0
9 stars 1 forks source link

[REQ] Chest Identification Improvements #31

Open JartanFTW opened 1 year ago

JartanFTW commented 1 year ago

Is your feature request related to a problem? Please describe. Chest identification commonly fails when there are traps or other objects in-front. This causes issues with gem identification, which is dependent upon chest identification to narrow down gem slot location.

Describe the solution you'd like

  1. Implement a custom chest identification model, replacing pyscreeze. This would be used both in gem identification and gold identification. Text identification would still use easyocr, at least for now.
  2. Optionally create a new gem identification model that accepts full screen input rather than narrowed images.

Additional context A gem identification model that accepts full screen input was attempted previously, but I didn't have any way of creating the sheer amount of data I needed. Now, I can modify a version of the existing solution to generate training data for this new solution. Along with this, at the time I hadn't learnt much/anything about convolution layers which should also help significantly.

JartanFTW commented 1 year ago

Gem identification and gold text reading currently has to go through multiple preparatory stages to crop the game screen into just the necessary parts of the totem. This introduces some delay and jank that could be optimized. The primary delay is from locating the totem, as this occasionally fails due to being partially blocked by a trap.

Solution: A R-CRN object detection model should be trained to identify the boundaries of the totem.

As we are only training to identify a single object (the totem), this model should work near instantaneously once trained. It should also be significantly more accurate than the existing pyscreeze implementation.