timotheeg / nestrischamps

A web-based OCR and restreamer system for NES Classic Tetris players
MIT License
48 stars 12 forks source link

Level OCR upper limit of 29 #173

Closed Javantea closed 3 months ago

Javantea commented 6 months ago

ocr_main.js:22

level: { crop: [416, 320, 30, 14], pattern: 'TD' }, // TD, because we only care about start level, which is 29 or lower

This causes level 30 onward to not be correctly detected. I stepped through the debugger and it shows up as 2 even though the number on the left is a 3 because the value of PATTERN_MAX_INDEXES['T'] is 3. To correctly detect the level it should use pattern DD. This issue was found while trying to understand why AlexT's capture is breaking on high levels. This does appear to be unrelated considering level 30 onward are detected when the line count is correct. This does still seem to be a bug because players can start on just about any level including level 30+.

timotheeg commented 6 months ago

Hey @Javantea ! Thanks for the report. It's great to see folks report issues here 😄 . It's very rare, and even more rare that folks have dug into the code to find how things work, so well done on that! 💪 👏

Right, so per the comment in code, the pattern TD is deliberate 🤔 . Let me give you more context and background info and we'll see what we can do.

On the deliberate choice made, below are the considerations I made:

So with all that said, NTC indeed works based on these 3 assumptions:

This does still seem to be a bug because players can start on just about any level including level 30+.

Right! Well that is indeed true with Gym, but it's not on the original rom; and also in practice, I believe that while folks do sometimes (often?) start at 29 to practice killscreen speed, nobody in practice starts at level 30+; and in competitions specifically, no match is ever done starting at level 30+, correct?

So because of that, I don't imagine I'd change this to be honest, at least not unless people start having competitions that start at level 30+ 🤔

Now in term of having good calibration to read the start level and the number of lines, there are few things that can be mentioned:

  1. players should have good calibration
    • always calibrate at level 0, to have clean edges on the digits
    • no black border in the normalized views
    • for next box, calibrate on a O or T piece for the top and bottom edges, and an I piece for left and right edges
    • For the field, dump pieces on the left and right all the way to the top and verify the normalized views is "clean": white blocks should show their 1px border, and there should be no black gaps at all
  2. players can tune the brightness and contrast to have a sharper/clearer picture
  3. if capture quality is bad, there's a hidden settings that allow using more of the source video, but it needs a full reset to do the calibration all over again, the procedure is like this:
    • Add the query string ?disable_half_height=1 into the producer url and hit enter
    • In the Settings tab, click the button Clear config and Restart
    • Redo the calibration as per normal, with all the calibration rules discussed above
    • Remember to always use the query string ?disable_half_height=1 when loading the producer again, so the memorized capture settings are applied on the right content size

On 3) I've been wanting to put some toggle in the UI to make the process of using the setting easier (as opposed to having this hidden "dirty" query string params), but I haven't had time yet...

Javantea commented 6 months ago

Thank you for the thorough response and advice on fixing our player's setup. I consider this issue closed. I agree that no competition has needed level 30+ start. In my own play I have chosen 148-184 level starts for obvious reasons, but DD would not correctly work on those levels either. The only reason I ran into this issue is because we encountered the rather serious desync on level 35, I tried a level 35 start to debug. Now that we know that level OCR is not used after start, we won't spend time trying to fix that. PXL_20231221_052412026alo