alex-ong / NESTrisOCR

OCR for statistics in NESTris
24 stars 7 forks source link

Fix Palette compute script and update easiercap palette accordingly #47

Closed timotheeg closed 4 years ago

timotheeg commented 4 years ago

I messed up the palette computation script (took 4x4 square instead of 3x3), but still only divided by 9, so averages would all be too bright, and in some cases above 255! (which is obviousl wrong... duh 😑)

This PR fixes the script and upload the correct palette for easiercap, as recomputed from the assets:

python3 -m scripts.compute_color_palette easiercap nestris_ocr/assets/sample_inputs/easiercap/lvl%d/field.png
alex-ong commented 4 years ago

How are you generating the sample images? if there is a misalignment, the 3x3 square could be eating black in the corners; I suggest that you update the code to instead not blur, and then check the color with ref color1, ref color2, white, black, and discard colors that aren't close enough. As long as the original reference colors are kind of close this should lead to an identical result, with more leniency.

timotheeg commented 4 years ago

How are you generating the sample images?

The reference field images were taken by saving from inside scan_field(), so they are basically crops of the input frame, straight from device, exactly as NESTrisOCR sees them.

if there is a misalignment, the 3x3 square could be eating black in the corners;

Yeah, I do assume good calibration for this :/

I additionally make one more basic assumption: "the field is at least 2x pixel perfect (160x320 or greater)". With that, getting 3x3 where at the lower-right quadrant, even after blurring should be right smack in the middle of the "pure color" for color1 and color2. It's cutting a little close for white because of the internal border, but the blur has radius 1 so there's minimal bleeding.

Basically, what I'm trying to capture is this:

image

And here is the illustration on the input frame for level 0, with and without blur, with the exact 3x3 captured areas highlighted:

image

With some zooming:

original blurred
image image

I could try to filter all pixels but I think that's overkill 🤔 . with my device, the input is quite dirty so I worry about dropping arbitrary pixels. Instead I'm still thinking that being able to pick known good areas is better.

But with that said, since I already do color averaging, the bur is indeed likely not needed. Also, the script is currently doing the "correct" color averaging method on the 3x3 areas (averages of squares), but "incorrect" color averaging for the average of averages, so I should be more consistent there, and clean that up.

There's lots of room to play with. With these particular captures, the result are pretty good. One more illustration below, the 4 images are:

  1. original capture
  2. As seen from color averaging
  3. re-drawn from OCR with default palette
  4. re-drawn from OCR with easiercap palette (as computed by the script)

image

timotheeg commented 4 years ago

Oh, thanks for merging first BTW!

We can add more modifications and improvements later, while the easiercap palette was definitely broken (causing block color flips in OCR 😅)