search

dicekeys / beta-program

Information for the DiceKeys beta
5 stars 3 forks source link

Improve performance in shadows test case #38

Open UppaJung opened 3 years ago

UppaJung commented 3 years ago

This test case has four bit-read errors, possibly due to shadows. The W1 die and B2 die are not read with enough confidence to come to a conclusion. I'd like to think the algorithm can do better here.

Test case Y6bS2rG4bR4lF6lU5bH3tN4bX4lB2tP5tZ1lJ2bM1lE1lT1rL3lO1lI3tK6rW1tD3lV1lA6rC5b

Current performance image

Test case provided by @p0lt in https://github.com/dicekeys/beta-program/issues/14

yeliaBdE commented 3 years ago

It occurred to me that, if the size of the dice were larger, but the imprinting on them is left the same size as it is now, the shadows from the top would be much less likely to reach so far as to obscure the imprinting, thereby making successful scanning more likely.

Of course, changing the dice size at this point does violence to the work already done on the overall case, not to mention the increased cost due to the larger dice and case retooling, but if other forces are driving a larger dice size, doing so could make this part of the project a bit easier.

Anyway, just a thought I wanted to pass on.

UppaJung commented 3 years ago

When you hold a DiceKey up to a camera, the entire DiceKey needs to fit within the viewport. If the DiceKey is bigger, you'll hold it further away but from the camera's perspective, it will look roughly the same as a closer/smaller DiceKey.

So, if we grow the DiceKey but not the images on the dice, they will appear smaller to the camera and we'll need higher resolution images to scan them. The only thing that won't need to grow proportionally is the thickness requirements for the injection molded walls and, possibly, the width of the walls (if the tolerances for manufacturing of dice are of absolute variance and not proportional variance.)

yeliaBdE commented 3 years ago

Or, to put it another way, there's no such thing as a free lunch. Thanks for patiently explaining that to me!

UppaJung commented 3 years ago

True, though as cameras improve and on-device processing power increases, there will be lots of lunches that will be highly subsidized by others in the ecosystem.

dragon788 commented 3 years ago

Even a "low" quality camera can do a pretty good job if you've trained the algorithm well enough. A project I worked on specifically targeted a 640x480 image for recognition of patterns because the smaller image was much faster to process, and almost every device (even slow ones) had cameras available of that quality or higher. Setting a fixed resolution for the image/video also means that you have a predictable amount of data to process instead of it increasing as camera resolutions go up. I'd imagine trying to do the scan on a Nokia 808 with a 41MP sensor would be bonkers.

That said, I was impressed that I was able to scan my DiceKeys via the webcam of my Chromebook, as it is probably only a 720p camera (approximately 1 megapixel since 1024x1024 would be a million pixels, but most sensors these days are 4:3 or 16:9 widescreen like laptop screens).

The reason I'm commenting on this issue is the shadows definitely were the biggest challenge using this camera that didn't have its own LED. If I had thought of this earlier, I would have set my system to use a white background or opened a pure white webpage alongside the tab where I was attempting the scan as well as setting the screen brightness to 100% and I think it would have worked much more quickly. Since laptop webcams are almost all front facing, using the screen for extra illumination is feasible, and as long as you aren't using an external monitor to display the DiceKeys.app the interface, this could also work on mobile devices and it is what many "selfie" or "mirror" type applications do to give better lighting for their photos. This might be a way to better utilize the front facing camera and optimize the algorithm for lower resolutions instead of fighting for more and more CPU/GPU to handle 1080p and 2K and 4K (and 8K) video streams.

I tested scanning again with the DiceKeys.app full screen on the laptop display with the backlight at 100% and primarily white pages open on all the screens, and it definitely scanned a lot faster than my initial attempts where no matter how I moved it, there was just enough shadow from the external monitors positioned above the laptop that I couldn't get a good scan. I also tested this in the "shadow of the giants" with the screen below the webcam at 100% and the externals using a 90% black background and couldn't get a good scan, but as soon as I opened a new tab with a mostly white background on the external behind the laptop screen/webcam it scanned very quickly.