alex-ong / NESTrisOCR

OCR for statistics in NESTris
24 stars 7 forks source link

Add color lookup option for field scanning #38

Closed timotheeg closed 4 years ago

timotheeg commented 4 years ago

Context

In NES Tetris, blocks in the field can have 4 possible values: black, white, color1, color2, where color1 and color2 change for every level.

In NESTrisOCR master, color1 and color2 are read from fixed screen locations: the piece stats on the left. Blocks areas are read from target pieces, and to prevent accidental mis-calibration, the read areas are downscaled with antialiasing to a single pixel, representing the average color of the area.

This works well, but had minor drawbacks:

  1. Since averaging is involved, and there are white bits in the color areas, the resultant computed colors are quite far from the actual colors used in the field (they're "washed out"). The algorithm for field scanning is forgiving however (finding the closest match of 4 colors), so it works well, but one could argue, it's "lucky" in a way.
  2. There are many training mods out-there which replace the stats area by something else (e.g. das trainer with das stats, TAUS with useful game stats). Field scanning can't be done for these roms because there are no fixed location to read color1 and color2 from the screen.

NES Tetris has a systematic setup however, and reference colors can be hardcoded and looked up from a table by level.

On a "perfect" rendering system like an emulator, that should be good enough. black, white, color1, and color2 will render per specs. On console with capture cards however, gamma correction or various artefacts might make the 4 colors "less perfect".

Because the algorithm which finds the nearest color is forgiving (closest match to 4 supplied colors), hardcoded black, white, color1, and color2 could still work well as-is. Still, one might want more flexibility:

This PR introduces a lookup system for color1 and color2 based on hardcoded values by level.

Approach

Tested to work well with Naive strategy with various setting combinations. On my box, everything works beautifully without the need to read any color or blak+white dynamically

{
  "calibration.dynamic_color": false,
  "calibration.dynamic_black_n_white": false,
  "calibration.color_interpolation": false
}

Fastest strategy seems to work well-ish too, but preview is not being scanned for some reason. It's not an issue with the black luma since it works with naive. Something else I need to look into.

Notes:

Enjoy! :D