search

jomjol / AI-on-the-edge-device

Easy to use device for connecting "old" measuring units (water, power, gas, ...) to the digital world
https://jomjol.github.io/AI-on-the-edge-device-docs/
5.67k stars 612 forks source link

Deactivate illumination and triggering the reading #172

Closed Github197 closed 3 years ago

Github197 commented 3 years ago

Is there a feature to deactivate the illumination? Would be fully enough to have a switch in the config.ini for that.

I also would need to trigger the reading, because I need first to switch on the display and navigate to the correct screen. This I can do automatically, but after a while the screen switches off again, so I need to tell your device when to make the photo.

Background is that I would like to use it for an selfiluminated display to read out numbers (heating system)...

This is the screen I would like to read out: grafik

tvunen commented 3 years ago

I had the same problem with reflection from the LED, in my case i solved it with a small piece of thick paper stick directly on the LED with double sided tape. There is still some illumination left for a good result and the reflection is gone. you could also rotate the ESP32 90 degrees and correct the angle in the config. then the reflection will be outside field of view.

jomjol commented 3 years ago

Currently there is no option to switch of the illumination, but I put it on the feature request list.

Regarding the reflection, there are meanwhile some creative ideas.

  1. Rotate the camera to have it at a place, where it does not hurt anymore
  2. but some black peace of paper or something comparable to the surface to absorb the light instead of reflection
  3. redirect the light with a small mirror to the side an make some kind of indirect illumination through the walls.

Maybe, one of these could help in your case

Github197 commented 3 years ago

Thank you for putting the switch on the feature request list. Would be nice if you could put also the triggering of the reading. Would be enough to call a page like "/capture" to start the reading. Before I could setup the screen and then send a request.

Regarding the illumination: for the moment I guess I will just use some block paint I put on the LED to cover it :-)

Nevertheless I am still struggeling with the reading of the numbers (I used your neural network to train the model with the new pictures, but it is not working yet. Maybe because I have only a low number of numbers, maybe the picture taken is not as good. Because when I check (with the recognition feature in your webpage) most of the time the numbers are not clear: grafik like on my water meter: grafik

No idea why this is not clear. Maybe because of the green numbers on green background...

jomjol commented 3 years ago

Be a bit carefull with covering the LED. It might get pretty hot - quite some power consumption, but short time. If the pictures look like the upper one, something is going wrong with the aquisition (eigther camera or alignment). There you should see some nice pictures of the original image, just cut out.

With respect to training. Great, that you try it on your own. It took me some time of experience. I can offer to support you there and/or incorporate your pictures into my next training session.

Github197 commented 3 years ago

Thank you for your support! After I covered the LED with a drop of black paint, I am finally getting some pictures out which are as expected. grafik I found out that the web interface is not reacting anymore in case the device is not finding the expected pictures. This is a problem for me, because I need first to switch to the right screen. I hope I could stop this issue by putting the timer for pictures to a very high number (9999999). This proofs even more that I need a trigger to start the camera ;-) Regarding the training I will restart from scratch and first collect new numbers as I am finally getting nice pictures. When I have some I will try to retrain the model and see if it is working. One question I have. Do you think it would be clever to mix my numbers with the ones you are using? I mean they are completly different. I have the fear then it will not work at all. What do you think?

René

Github197 commented 3 years ago

One more comment from my side. The little pictures with the numbers at the bottom of the image above are much darker than what is shown in the photo of the screen. So I guess you are trying to enhance the pictures, which is working well for the water meter, but in my case it makes it worse. :-( Correct?

jomjol commented 3 years ago

First of all: I would give the mixing a try. The network is relatively flexible and can handle all different type of digits. Currently already different type are present, including LCD-digits. In your final setup you should consider some kind of frame around the images. This is needed for the training algorithm, as it does some augmentation with shifting as well.

Regarding the cutted out ROIs: I do not do any enhancement here (only for the alignment marks). So they should have exactly the same darkness as in the full image - the "felt" darker pictures, are an optical effect, due to the white background.

Github197 commented 3 years ago

You are right it was just an optical effect! :-)

I will collect the digits. Where shall I put the collected ones then?

Regarding your adivce to put a frame around the images, I am not sure what you mean. This is the LCD panel of my heater system which I cannot change. I have no possibility to adapt it, because I need this also to change the heating parameters. Or do you mean around the collected images? If so, could you please provide me with an example?

jomjol commented 3 years ago

Misunderstanding: I mean a free space around the collected image. With the newest version of the watermeter, you have a recommended spacing suggested by the thinner inner frame:

grafik

Github197 commented 3 years ago

Understood. Unfortunately when I do it this way, the next number will be also in the picture: grafik I guess this would also be a problem, because the numbers on the sides are also changing...

jomjol commented 3 years ago

Sorry for the very late reply. Overlapping is not a problem.

Do you still need a switch for the illumination?

numminorih-git commented 3 years ago

Sorry for the very late reply. Overlapping is not a problem.

Do you still need a switch for the illumination?

I think it will be an excellent to control brightness from web interface.

Github197 commented 3 years ago

Sorry for the very late reply. Overlapping is not a problem.

Do you still need a switch for the illumination?

Yes. I am still looking for it. Currently I have the led covered with black paint but this is not the best solution. Also it would make it more stable due to less peak power consumption when reading.

I am also still looking for an option to tell the device to read now, because as said in the beginning I need first to switch to the right screen. One more question here, because to do the switching I need to push 3 buttons on the heating interface in a certain order to get to the screen (1x ESC, 7x Up, 1x Enter) I want to read out. Would it be possible to use 3 of the output pins for this? If so I would need to define the order how they should be triggered. As a workaround I am thinking about another device (e.g. NodeMCU) to navigate through the menu and then calling the other device (e.g. call a web site to make a photo)

Github197 commented 3 years ago

Be a bit carefull with covering the LED. It might get pretty hot - quite some power consumption, but short time. If the pictures look like the upper one, something is going wrong with the aquisition (eigther camera or alignment). There you should see some nice pictures of the original image, just cut out.

With respect to training. Great, that you try it on your own. It took me some time of experience. I can offer to support you there and/or incorporate your pictures into my next training session.

Please find here pictures of the digits taken by your software from my heater interface: Digits.zip

Thank you for your support!

jomjol commented 3 years ago

Just wanted to start training your images, but the contrast in some cases is really bad. Example: -1_ROI15_20210419-081502-1_ROI16_20210415-155357-1_ROI17_20210412-133641

Firstly you should try to improve the contrast otherwise any training will not be satisfying. Do you see a change to improve this?

Regarding the triggering of you menue with 3 buttons: There are only maximim 2 GPIOs available. All others are in use by the SD card, camera, PSRAM or not accesible on this board.

Github197 commented 3 years ago

Just wanted to start training your images, but the contrast in some cases is really bad. Example: -1_ROI15_20210419-081502-1_ROI16_20210415-155357-1_ROI17_20210412-133641

Firstly you should try to improve the contrast otherwise any training will not be satisfying. Do you see a change to improve this?

Regarding the triggering of you menue with 3 buttons: There are only maximim 2 GPIOs available. All others are in use by the SD card, camera, PSRAM or not accesible on this board.

I am still thinking how to improve the contrast, but I am not sure how I could do it. The background light is always there and cannot be changed also the contrast at the device is as it is. No possibility to change it. Maybe I could cover the whole thing so there is no light coming from the outside. But this I would first need to make.

Regarding the buttons this would not be a problem as long as I could trigger when to take a picture, because then I would use a another device for the navigation and then triggering your device to make the photo.

jomjol commented 3 years ago

Actual, just remember, that you already can start a flow manually. Just forgot it, because it is a relict from the beginning. To do so follow this steps:

  1. disable autostart in the config.ini:

    [AutoTimer]
;AutoStart = true
;Intervall = 1

  2. Restart and wait until the inital setting is done (about 30s)

  3. Manually trigger a flow with the following html-code

    http://IP-ADRESS/doflow

Let me know, if this works for you!

jomjol commented 3 years ago

No further activities. Please reopen in case of questions.