timotheeg / nestrischamps

A web-based OCR and restreamer system for NES Classic Tetris players
MIT License
48 stars 12 forks source link

Drop OpenCV and do first calibration with floodfill #117

Closed timotheeg closed 2 years ago

timotheeg commented 2 years ago

Context

The open CV calibration is error prone. Not working for many people, who then have to move all the fields manually, which is very tedious.

@alex-ong had suggested long ago to use a flood fill algorithm to detect fhe field, and then use it, and knowledge of the typical Tetris layout, to compute the capture coordinates of all the other fields.

Finding the Tetris field became the annoying part, especially when capturing from browser windows youtube replays, where the layout might not be standard, perhaps with a player cam on the side, breaking the relative position in the overall window.

Approach

  1. Drop open CV entirely (saves 8MB!)
  2. Implement the flood fill algorithm to detect edge of the tetris field
  3. Update the calibration wizard steps to show instruction on what to do
  4. Finally to find the starting point of the floodfill, use the most intelligent entity in the room: the user himself/herself!

After having selected the input device, and the rom type, the user will be asked to click somewhere black in the field. This will provide the starting point of the floodfill to find the edge of the field.

Bonus: For capture devices, brightness defaults to 1.65 on first calibration.

timotheeg commented 2 years ago

Tested with ClassiqNQ input!