Wrong Alignment after a few correct reads

[theslash]

So I'm trying to get this to run reliable and I'm having a few issues:

A few reads always are right, and then everything falls apart:

2021-04-10_09-12-54:    FlowControll.doFlow - ClassFlowDigit
2021-04-10_09-12-58:    FlowControll.doFlow - ClassFlowAnalog
2021-04-10_09-13-04:    FlowControll.doFlow - ClassFlowPostProcessing
2021-04-10_09-13-04:    FlowControll.doFlow - ClassFlowMQTT
2021-04-10_09-13-04: sent publish successful in MQTTPublish, msg_id=3124, home/wasserzaehler, 344.5982
2021-04-10_09-13-04: task_autodoFlow - round done
2021-04-10_09-13-04: CPU Temperature: 56.1
2021-04-10_09-13-20: download_get_handler
2021-04-10_09-13-37: download_get_handler
2021-04-10_09-17-43: task_autodoFlow - next round - Round #2
2021-04-10_09-17-43:    FlowControll.doFlow - ClassFlowMakeImage
2021-04-10_09-17-51:    FlowControll.doFlow - ClassFlowAlignment
2021-04-10_09-18-21:    FlowControll.doFlow - ClassFlowDigit
2021-04-10_09-18-25:    FlowControll.doFlow - ClassFlowAnalog
2021-04-10_09-18-32:    FlowControll.doFlow - ClassFlowPostProcessing
2021-04-10_09-18-32:    FlowControll.doFlow - ClassFlowMQTT
2021-04-10_09-18-32: sent publish successful in MQTTPublish, msg_id=59041, home/wasserzaehler, 344.6011
2021-04-10_09-18-32: task_autodoFlow - round done
2021-04-10_09-18-32: CPU Temperature: 53.9
2021-04-10_09-19-48: download_get_handler
2021-04-10_09-19-49: download_get_handler
2021-04-10_09-19-55: download_get_handler
2021-04-10_09-23-43: task_autodoFlow - next round - Round #3
2021-04-10_09-23-43:    FlowControll.doFlow - ClassFlowMakeImage
2021-04-10_09-23-51:    FlowControll.doFlow - ClassFlowAlignment
2021-04-10_09-24-52:    FlowControll.doFlow - ClassFlowDigit
2021-04-10_09-24-56:    FlowControll.doFlow - ClassFlowAnalog
2021-04-10_09-25-02:    FlowControll.doFlow - ClassFlowPostProcessing
2021-04-10_09-25-02:    FlowControll.doFlow - ClassFlowMQTT
2021-04-10_09-25-02: sent publish successful in MQTTPublish, msg_id=20332, home/wasserzaehler, 344.6011
2021-04-10_09-25-02: task_autodoFlow - round done
2021-04-10_09-25-03: CPU Temperature: 53.9
2021-04-10_09-29-43: task_autodoFlow - next round - Round #4
2021-04-10_09-29-43:    FlowControll.doFlow - ClassFlowMakeImage
2021-04-10_09-29-51:    FlowControll.doFlow - ClassFlowAlignment
2021-04-10_09-30-21:    FlowControll.doFlow - ClassFlowDigit
2021-04-10_09-30-25:    FlowControll.doFlow - ClassFlowAnalog
2021-04-10_09-30-31:    FlowControll.doFlow - ClassFlowPostProcessing
2021-04-10_09-30-31:    FlowControll.doFlow - ClassFlowMQTT
2021-04-10_09-30-32: sent publish successful in MQTTPublish, msg_id=36109, home/wasserzaehler, 344.6121
2021-04-10_09-30-32: task_autodoFlow - round done
2021-04-10_09-30-32: CPU Temperature: 53.3
2021-04-10_09-35-43: task_autodoFlow - next round - Round #5
2021-04-10_09-35-43:    FlowControll.doFlow - ClassFlowMakeImage
2021-04-10_09-35-51:    FlowControll.doFlow - ClassFlowAlignment
2021-04-10_09-36-21:    FlowControll.doFlow - ClassFlowDigit
2021-04-10_09-36-25:    FlowControll.doFlow - ClassFlowAnalog
2021-04-10_09-36-31:    FlowControll.doFlow - ClassFlowPostProcessing
2021-04-10_09-36-31:    FlowControll.doFlow - ClassFlowMQTT
2021-04-10_09-36-32: sent publish successful in MQTTPublish, msg_id=28472, home/wasserzaehler, 344.6121
2021-04-10_09-36-32: task_autodoFlow - round done
2021-04-10_09-36-32: CPU Temperature: 53.3
2021-04-10_09-41-43: task_autodoFlow - next round - Round #6
2021-04-10_09-41-43:    FlowControll.doFlow - ClassFlowMakeImage
2021-04-10_09-41-49:    FlowControll.doFlow - ClassFlowAlignment
2021-04-10_09-42-49:    FlowControll.doFlow - ClassFlowDigit
2021-04-10_09-42-53:    FlowControll.doFlow - ClassFlowAnalog
2021-04-10_09-43-00:    FlowControll.doFlow - ClassFlowPostProcessing
2021-04-10_09-43-00:    FlowControll.doFlow - ClassFlowMQTT
2021-04-10_09-43-00: sent publish successful in MQTTPublish, msg_id=12270, home/wasserzaehler, 7N837.0023

I think I might have a problem with the alignment process, because when I go to the settings and check my alignment the correct rotated image never shows up, and the alignment is sometimes off.

These are my two alignment points:

And this is what the check looks like:

Any Idea whats going on here?

[theslash]

This is the log output when I do the check:

08:13:53.527 -> handler_editflow uri: /editflow.html?task=test_align&host=http://192.168.1.57
08:13:56.333 -> Vor ft->FindTemplate(_temp1);  /sdcard/config/ref0.jpg
08:13:56.333 -> Anzahl 5292, avgDifSum -115904.000000d, avg -21.901739, SAD_neu: 1.313647d, _SAD_old: 1.193412, _SAD_crit:0.120235
08:13:59.329 -> E (697815) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:13:59.329 -> E (697815) task_wdt:  - IDLE0 (CPU 0)
08:13:59.329 -> E (697815) task_wdt: Tasks currently running:
08:13:59.329 -> E (697815) task_wdt: CPU 0: httpd
08:13:59.329 -> E (697815) task_wdt: CPU 1: IDLE1
08:14:02.304 -> E (700815) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:14:02.339 -> E (700815) task_wdt:  - IDLE0 (CPU 0)
08:14:02.339 -> E (700815) task_wdt: Tasks currently running:
08:14:02.339 -> E (700815) task_wdt: CPU 0: httpd
08:14:02.339 -> E (700815) task_wdt: CPU 1: IDLE1
08:14:05.303 -> E (703815) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:14:05.336 -> E (703815) task_wdt:  - IDLE0 (CPU 0)
08:14:05.336 -> E (703815) task_wdt: Tasks currently running:
08:14:05.336 -> E (703815) task_wdt: CPU 0: httpd
08:14:05.370 -> E (703815) task_wdt: CPU 1: IDLE1
08:14:08.333 -> E (706815) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:14:08.333 -> E (706815) task_wdt:  - IDLE0 (CPU 0)
08:14:08.333 -> E (706815) task_wdt: Tasks currently running:
08:14:08.333 -> E (706815) task_wdt: CPU 0: httpd
08:14:08.333 -> E (706815) task_wdt: CPU 1: IDLE1
08:14:11.302 -> E (709815) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:14:11.335 -> E (709815) task_wdt:  - IDLE0 (CPU 0)
08:14:11.335 -> E (709815) task_wdt: Tasks currently running:
08:14:11.335 -> E (709815) task_wdt: CPU 0: httpd
08:14:11.368 -> E (709815) task_wdt: CPU 1: IDLE1
08:14:14.323 -> E (712815) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:14:14.323 -> E (712815) task_wdt:  - IDLE0 (CPU 0)
08:14:14.323 -> E (712815) task_wdt: Tasks currently running:
08:14:14.323 -> E (712815) task_wdt: CPU 0: httpd
08:14:14.356 -> E (712815) task_wdt: CPU 1: IDLE1
08:14:17.314 -> E (715815) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:14:17.314 -> E (715815) task_wdt:  - IDLE0 (CPU 0)
08:14:17.348 -> E (715815) task_wdt: Tasks currently running:
08:14:17.348 -> E (715815) task_wdt: CPU 0: httpd
08:14:17.348 -> E (715815) task_wdt: CPU 1: IDLE1
08:14:18.406 -> Anzahl 5292, avgDifSum -171970.000000d, avg -32.496220, SAD_neu: 1.222944d, _SAD_old: 1.193412, _SAD_crit:0.029532
08:14:18.406 -> Vor ft->FindTemplate(_temp2);  /sdcard/config/ref1.jpg
08:14:18.439 -> Anzahl 5292, avgDifSum -123283.000000d, avg -23.296108, SAD_neu: 1.081303d, _SAD_old: 1.066861, _SAD_crit:0.014441
08:14:21.419 -> E (719925) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
08:14:21.454 -> E (719925) task_wdt:  - IDLE0 (CPU 0)
08:14:21.454 -> E (719925) task_wdt: Tasks currently running:
08:14:21.454 -> E (719925) task_wdt: CPU 0: httpd
08:14:21.454 -> E (719925) task_wdt: CPU 1: IDLE1
08:14:23.040 -> Alignment: dx 1 - dy 18 - rot 2.708143
08:14:23.079 -> 2021-04-10_08-14-23
08:14:23.218 -> uri: /img_tmp/rot.jpg
08:14:23.218 -> 1 uri: /img_tmp/rot.jpg, filename: rot.jpg, filepath: /sdcardrot.jpg
08:14:23.218 -> File to upload: rot.jpg
08:14:23.218 -> ClassFlowControll::GetJPGStream rot.jpg
08:14:23.218 -> uri: /img_tmp/rot.jpg
08:14:23.218 -> 1 uri: /img_tmp/rot.jpg, filename: rot.jpg, filepath: /sdcardrot.jpg
08:14:23.255 -> File to upload: /sdcard/img_tmp/rot.jpg
08:14:23.255 -> File is locked: /sdcard/img_tmp/rot.jpg - wait for 1
08:14:23.255 -> Logfile opened: /sdcard/log/message/log_2021-04-10.txt
08:14:24.238 -> E (722735) serverhelp: Failed to read existing file : /sdcard/img_tmp/rot.jpg
08:14:24.238 -> W (722735) httpd_txrx: httpd_resp_send_err: 500 Internal Server Error - Failed to read existing file
08:14:24.273 -> W (722755) httpd_uri: httpd_uri: uri handler execution failed
08:14:24.273 -> uri: /img_tmp/rot_roi.jpg
08:14:24.273 -> 1 uri: /img_tmp/rot_roi.jpg, filename: rot_roi.jpg, filepath: /sdcardrot_roi.jpg
08:14:24.273 -> File to upload: rot_roi.jpg
08:14:24.273 -> ClassFlowControll::GetJPGStream rot_roi.jpg
08:14:24.273 -> uri: /img_tmp/rot_roi.jpg
08:14:24.273 -> 1 uri: /img_tmp/rot_roi.jpg, filename: rot_roi.jpg, filepath: /sdcardrot_roi.jpg
08:14:24.306 -> File to upload: /sdcard/img_tmp/rot_roi.jpg
08:14:24.306 -> File is locked: /sdcard/img_tmp/rot_roi.jpg - wait for 1
08:14:24.306 -> Logfile opened: /sdcard/log/message/log_2021-04-10.txt
08:14:25.289 -> E (723795) serverhelp: Failed to read existing file : /sdcard/img_tmp/rot_roi.jpg
08:14:25.289 -> W (723795) httpd_txrx: httpd_resp_send_err: 500 Internal Server Error - Failed to read existing file
08:14:25.324 -> W (723815) httpd_uri: httpd_uri: uri handler execution failed
08:14:25.324 -> uri: /img_tmp/alg.jpg
08:14:25.324 -> 1 uri: /img_tmp/alg.jpg, filename: alg.jpg, filepath: /sdcardalg.jpg
08:14:25.324 -> File to upload: alg.jpg
08:14:25.324 -> ClassFlowControll::GetJPGStream alg.jpg
08:14:25.324 -> I (723825) flow_controll: Sending file : alg.jpg ...
08:14:26.708 -> I (725195) flow_controll: File sending complete

After a reboot (disconnect & connect, a software reboots often results in a bootloop) everything works again for 3-4 times:

[jomjol]

Hi - first of all great, that you have it up and running. Some remarks:

1. I would recommend one other alignment reference: instead of the "412" I would recommend the Q3-4 text next to it. It is a bit bigger and has not so many fine structures - so should be more reliable
2. Can you post your config.ini - so I can see which alignment features you are using? Please switch of the "FixedExposure" - this will lead in 5-10 seconds longer runs, but more stable image quality.

[theslash]

Hi, thank you for this great project. Until now I read my water meter with a TCRT5000 but this was kind of unreliable an complicated so this looks very promising.

1. I changed it and rebooted
2. Here is my config:

[MakeImage] LogImageLocation = /log/source LogfileRetentionInDays = 15 WaitBeforeTakingPicture = 5 ImageQuality = 5 ImageSize = VGA ;Brightness = -2 FixedExposure = false

[Alignment] InitialRotate = 181 /config/ref0.jpg 341 142 /config/ref1.jpg 91 248 SearchFieldX = 20 SearchFieldY = 20 InitialMirror = false AlignmentAlgo = Default

[Digits] Model = /config/dig0820s2q.tflite LogImageLocation = /log/digit LogfileRetentionInDays = 3 ModelInputSize = 20 32 digit2 151 131 31 56 digit3 192 132 27 48 digit1 229 130 32 58 ROI4 267 128 37 64 ROI5 309 132 32 50

[Analog] Model = /config/ana0700s1lq.tflite LogImageLocation = /log/analog LogfileRetentionInDays = 3 ModelInputSize = 32 32 analog4 351 192 79 79 analog3 322 286 78 78 analog2 231 325 78 78 analog1 137 288 80 80 ExtendedResolution = false

[PostProcessing] DecimalShift = 0 PreValueUse = false PreValueAgeStartup = 720 AllowNegativeRates = false MaxRateValue = 0.1 ErrorMessage = true CheckDigitIncreaseConsistency = false

[MQTT] Uri = mqtt://192.168.1.5:1883 Topic = home/wasserzaehler ;TopicError = wasserzaehler/error ClientID = wasseruhr user = mqtt password =**

[AutoTimer] AutoStart = true Intervall = 6

[Debug] Logfile = true LogfileRetentionInDays = 3

[System] TimeZone = CET-1CEST ;TimeServer = fritz.box ;Hostname = watermeter ;SetupMode = false

[Ende]

[jomjol]

The config looks okay. You could switch on the UsePreValue - this helps in case of sudden reboots to get to know the previous value (in case there is an intermediate number). A reboot from time to time is normal. After quite some work, we could not figure out why. My system is running stable for more than 200 runs, other systems do a reboot after a view turns.

One can observe, that a reboot is more likely after a html access and with some kind of SD-cards.

Therefore after a reboot the system should recover itself pretty well and as you only need a value every several numbers, in my case this not turns out to be a problem at all.

[theslash]

Yes I think my main problem is the alignment. This is what it looked like after 6 correct values:

Should I perhaps put a little marking or sticker with red paint on the glass to help the alignment? Or could the rotation be a problem?

[jomjol]

This is not a problem of the alignment - rather of the camera itself. It looks like, the image is rotated several times in a sequence. This might happen, if the taking of new images fails. Can you check your camera connection itself?

[theslash]

Yes, I just replaced the whole ESP32 but used the cam. if this also fails I will try another cam.

[theslash]

So, no idea what is happening here, tried different ESP & different cam, Connections are ok:

5good readings, these are the source files after that: Good:

Good:

Wrong orientation:

Wrong orientation and strange twisted:

Also the whole alignment check process is not working, could this be a general hardware issue?

[jomjol]

What is the difference between good and twisted? The two lines on the edges indicate, that there are two rotations happening. I have seen this once, but do not remeber the reason for this or the counter measure. Do you have the corresponding part of the log-file?

[theslash]

I meant the image seems to be half rotated.

I just now started new. New SD card, new camera, and set it up so that the image doesnt have to be rotated and I think I got a better image quality now too.

One question though: when aligning the Digital ROIs, is the thin or the thick frame supposed to be around the number?

Here he always recognizes a 9 instead of a 3 And what is the way to deal with numbers like the last digit where I now get a NaN Value?

Sorry if I ask dumb questions, and thank you for your effort

[jomjol]

The thin frame is supposed to be directly around the number. The thicker frame is than the later cut out image. The recognition works better if there is some spacing to the border of the image. If the 3/9 Problem is still there you can send me some of your wrong detected "3" images (just turn on the logging and check the sd-card). I will put them to the next training.

[theslash]

I got it working now with the 3 and 9s I think the problem yesterday was something with the sd card. Now the esp boots perfectly and it looks good.

I will wait how it works today and close this issue then, thank you very much for your help, I will report how it works .

[theslash]

So far everything works very good. What I did, for reference:

• Formatted the SD card completely new (slow formatting in windows)
• Moved only the standard files and folders and my wifi config on it and started completely new
• so I think the main problem was the sd card as so often

Had good readings for the past hours, and the system recovered from 2 reboots without problems

Thank you very much for your help