jomjol / AI-on-the-edge-device

Easy to use device for connecting "old" measuring units (water, power, gas, ...) to the digital world
https://jomjol.github.io/AI-on-the-edge-device-docs/
5.94k stars 648 forks source link

Add support for WS2813 as external flashlight #1575

Closed pvogt09 closed 1 year ago

pvogt09 commented 1 year ago

The Problem

I am struggling to get a WS2813 LED Strip working as an external flashlight. I set up the wiring according to https://github.com/jomjol/AI-on-the-edge-device/wiki/External-LED (with R=450) and changed the settings to use GPIO12 as external flashlight. When I open http://ESP-CAM-IP/capture_with_flashlight a picture is taken but the LEDs don't light up at all (is this intended?) and when the normal flow for reading numbers starts sometimes the LEDs start to light up in different colors and different patterns and stay illuminated until the next flow starts. I am using the LED strip from here. The LED strip is powered from a different source than the ESP32 to avoid brownout problems and break down the problem to the signal wire.

While trying to solve the problem, I found out that it might be necessary to level-shift the output voltage of GPIO12 to 5V but for some people it works without that. It would be nice if somebody could confirm that this is the case.I tried to do this with the two methods mentioned here, but the diode methods makes the ESP32 unbootable because it sets GPIO12 HIGH at boot which changes the flash supply voltage to 1.8V according to this and I probably wired the MOSFET method (with two MOSFETS to double invert) wrong because no LED was lit up at all when testing this.

If it is a software problem I would be willing to help making WS2813 work and test if you told me where the code for the flashlight can be found after it is made sure that it is not a hardware problem on my side.

The serial log might not be too helpful because it does not print anything about activating the LEDs or not and taking the picture works nevertheless.

Version

13.0.7

Logfile

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:6656
load:0x40078000,len:15300
ho 0 tail 12 room 4
load:0x40080400,len:3848
entry 0x40080694
I (29) boot: ESP-IDF 4.4.2 2nd stage bootloader
I (29) boot: compile time 19:41:11
I (29) boot: chip revision: 1
I (32) boot_comm: chip revision: 1, min. bootloader chip revision: 0
I (39) boot.esp32: SPI Speed      : 40MHz
I (43) boot.esp32: SPI Mode       : DIO
I (48) boot.esp32: SPI Flash Size : 4MB
I (52) boot: Enabling RNG early entropy source...
I (58) boot: Partition Table:
I (61) boot: ## Label            Usage          Type ST Offset   Length
I (69) boot:  0 nvs              WiFi data        01 02 00009000 00004000
I (76) boot:  1 otadata          OTA data         01 00 0000d000 00002000
I (84) boot:  2 phy_init         RF data          01 01 0000f000 00001000
I (91) boot:  3 ota_0            OTA app          00 10 00010000 001db000
I (99) boot:  4 ota_1            OTA app          00 11 001f0000 001db000
I (106) boot: End of partition table
I (110) boot_comm: chip revision: 1, min. application chip revision: 0
I (117) esp_image: segment 0: paddr=001f0020 vaddr=3f400020 size=4b89ch (309404) map
I (244) esp_image: segment 1: paddr=0023b8c4 vaddr=3ffb0000 size=04294h ( 17044) load
I (252) esp_image: segment 2: paddr=0023fb60 vaddr=40080000 size=004b8h (  1208) load
I (253) esp_image: segment 3: paddr=00240020 vaddr=400d0020 size=166ac4h (1469124) map
I (821) esp_image: segment 4: paddr=003a6aec vaddr=400804b8 size=1b1e0h (111072) load
I (869) esp_image: segment 5: paddr=003c1cd4 vaddr=50000000 size=00010h (    16) load
I (883) boot: Loaded app from partition at offset 0x1f0000
I (883) boot: Disabling RNG early entropy source...
I (895) psram: This chip is ESP32-D0WD
I (895) spiram: Found 64MBit SPI RAM device
I (896) spiram: SPI RAM mode: flash 40m sram 40m
I (900) spiram: PSRAM initialized, cache is in low/high (2-core) mode.
I (907) cpu_start: Pro cpu up.
I (911) cpu_start: Starting app cpu, entry point is 0x400828a8
I (904) cpu_start: App cpu up.
I (1809) spiram: SPI SRAM memory test OK
I (1817) cpu_start: Pro cpu start user code
I (1817) cpu_start: cpu freq: 160000000
I (1817) cpu_start: Application information:
I (1820) cpu_start: Project name:     esp32cam-server-only
I (1826) cpu_start: App version:      v13.0.7
I (1831) cpu_start: Compile time:     Dec 11 2022 22:36:48
I (1837) cpu_start: ELF file SHA256:  80306e6cd06c9dd4...
I (1843) cpu_start: ESP-IDF:          4.4.2
I (1849) heap_init: Initializing. RAM available for dynamic allocation:
I (1855) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (1862) heap_init: At 3FFC06E8 len 0001F918 (126 KiB): DRAM
I (1868) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (1874) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1881) heap_init: At 4009B698 len 00004968 (18 KiB): IRAM
I (1887) spiram: Adding pool of 4095K of external SPI memory to heap allocator
I (1896) spi_flash: detected chip: generic
I (1900) spi_flash: flash io: dio
I (1910) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (1915) spiram: Reserving pool of 40K of internal memory for DMA/internal allocations
I (1925) MAIN: 

I (1925) gpio: GPIO[32]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
f external SPI memory to heap allocator
I (1896) spi_flash: detected chip: generic
I (1900) spi_flash: flash io: dio
I (1910) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (1914) spiram: Reserving pool of 40K of internal memory for DMA/internal allocations
I (1924) MAIN: 

I (1924) gpio: GPIO[32]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
I (3934) gpio: GPIO[25]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:2 
I (3934) cam_hal: cam init ok
I (3934) sccb: pin_sda 26 pin_scl 27
I (3934) sccb: sccb_i2c_port=1

I (3944) gpio: GPIO[32]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
I (3974) camera: Detected camera at address=0x30
I (3974) camera: Detected OV2640 camera
I (3974) camera: Camera PID=0x26 VER=0x42 MIDL=0x7f MIDH=0xa2
I (4054) cam_hal: buffer_size: 32768, half_buffer_size: 4096, node_buffer_size: 2048, node_cnt: 16, total_cnt: 15
I (4054) cam_hal: Allocating 61440 Byte frame buffer in PSRAM
I (4064) cam_hal: cam config ok
I (4064) ov2640: Set PLL: clk_2x: 0, clk_div: 0, pclk_auto: 0, pclk_div: 8
I (6174) MAIN: Using SDMMC peripheral
Name: 00000
Type: SDHC/SDXC
Speed: 20 MHz
Size: 14910MB
I (6394) MAIN: =================================================
I (6434) MAIN: ==================== Startup ====================
I (6484) MAIN: =================================================
I (6534) MAIN: Branch: 'HEAD', Revision: 304b9e0, Date/Time: 2022-12-11 22:36, Web UI: Release: v13.0.7 (Commit: 304b9e0) 
I (6604) MAIN: Reset reason: Power-on event
I (6644) OTA: Start CheckOTAUpdateCheck...
I (6664) OTA: SHA-256 for the partition table: : 9d4a2809ae90fb20709a8ca3dd3ac3462e1e73286bcd31b689060c91de06fa3a
I (6684) OTA: SHA-256 for bootloader: : 21594efd6996902a3375ef5661f957086c5f33e01a1cb644f7cd561e8ad3723c
I (7234) OTA: SHA-256 for current firmware: : d55c4f7057765201a5d2a79606310051a63e176d16779a0db445bb7532174e71
I (7294) wifi:wifi driver task: 3ffd4e00, prio:23, stack:6656, core=0
I (7294) system_api: Base MAC address is not set
I (7294) system_api: read default base MAC address from EFUSE
I (7314) wifi:wifi firmware version: eeaa27d
I (7314) wifi:wifi certification version: v7.0
I (7314) wifi:config NVS flash: enabled
I (7314) wifi:config nano formating: disabled
I (7314) wifi:Init data frame dynamic rx buffer num: 16
I (7324) wifi:Init management frame dynamic rx buffer num: 16
I (7324) wifi:Init management short buffer num: 32
I (7334) wifi:Init static tx buffer num: 16
I (7334) wifi:Init tx cache buffer num: 16
I (7334) wifi:Init static rx buffer size: 1600
I (7344) wifi:Init static rx buffer num: 10
I (7344) wifi:Init dynamic rx buffer num: 16
I (7354) wifi_init: rx ba win: 6
I (7354) wifi_init: tcpip mbox: 32
I (7354) wifi_init: udp mbox: 6
I (7364) wifi_init: tcp mbox: 6
I (7364) wifi_init: tcp tx win: 5744
I (7374) wifi_init: tcp rx win: 5744
I (7374) wifi_init: tcp mss: 1440
I (7374) wifi_init: WiFi IRAM OP enabled
I (7384) wifi_init: WiFi RX IRAM OP enabled
I (7394) phy_init: phy_version 4670,719f9f6,Feb 18 2021,17:07:07
I (7504) wifi:mode : sta (xx:xx:xx:xx:xx:xx)
I (7504) wifi:enable tsf
I (7504) BLINK: Flash - start
I (7504) WIFI: Set Hostname to:ESP32-CAM-gasmeter
I (7504) WIFI: wifi_init_sta finished.
I (7514) wifi:new:<6,0>, old:<1,0>, ap:<255,255>, sta:<6,0>, prof:1
I (7514) wifi:state: init -> auth (b0)
I (7524) wifi:state: auth -> assoc (0)
I (7534) wifi:state: assoc -> run (10)
I (7564) wifi:connected with XXXXXXX, aid = 1, channel 6, BW20, bssid = xx:xx:xx:xx:xx:xx
I (7564) wifi:security: WPA2-PSK, phy: bgn, rssi: -68
I (7574) wifi:pm start, type: 1

W (7594) wifi:<ba-add>idx:0 (ifx:0, xx:xx:xx:xx:xx:xx), tid:6, ssn:2, winSize:64
I (7614) wifi:AP's beacon interval = 102400 us, DTIM period = 1
I (7704) BLINK: Flash - done
I (8284) esp_netif_handlers: sta ip: 192.168.178.54, mask: 255.255.255.0, gw: xxx.xxx.xxx.x
I (8284) WIFI: got ip:192.168.178.54
I (8284) BLINK: Flash - start
I (8284) WIFI: connected to ap SSID: XXXXXXX, password: XXXXXXX
W (8294) wifi:<ba-add>idx:1 (ifx:0, xx:xx:xx:xx:xx:xx), tid:0, ssn:0, winSize:64
I (10294) SNTP: Initializing SNTP
I (10294) SNTP: Time is not set yet. Getting time over NTP server pool.ntp.org
I (10334) SNTP: Waiting until we get a time from the NTP server pool.ntp.org
I (10384) SNTP: Status: Reset
I (12434) SNTP: Status: Reset
I (14484) SNTP: Status OK
I (14544) SNTP: Time is synced with NTP Server pool.ntp.org
I (14594) SNTP: Time zone set to CET-1CEST,M3.5.0,M10.5.0/3
I (14644) SNTP: The current date/time in Berlin is: 2022-12-13_18:08:22
I (14644) MAIN: =================================================
I (14694) MAIN: ================== Main Started =================
I (14734) MAIN: =================================================
I (16794) MAIN SERVER: Starting server on port: '80'
I (16794) MAIN SERVER: Registering URI handlers
I (16794) TFLITE SERVER: server_part_camera - Registering URI handlers
I (16794) OTA: Registering URI handlers
I (16804) GPIO: start GpioHandler
I (16804) GPIO: register GPIO Uri
I (16814) GPIO: server_GPIO - Registering URI handlers
I (16814) MAIN: Initialization completed successfully!
I (16894) ov2640: Set PLL: clk_2x: 0, clk_div: 0, pclk_auto: 0, pclk_div: 8
I (18294) BLINK: Flash - done
E (18574) SERVER HELP: Failed to read existing file: /sdcard/html/192.168.178.54
W (18574) httpd_txrx: httpd_resp_send_err: 404 Not Found - <pre>

        _
    .__(.)< ( oh oh! This page does not exist! )
    \___)

                You could try your <a href=index.html target=_parent>luck</a> here!</pre>
<script>document.cookie = "page=overview.html"</script>
W (18604) httpd_uri: httpd_uri: uri handler execution failed
E (18634) SERVER HELP: Failed to read existing file: /sdcard/html/192.168.178.54
W (18634) httpd_txrx: httpd_resp_send_err: 404 Not Found - <pre>

        _
    .__(.)< ( oh oh! This page does not exist! )
    \___)

                You could try your <a href=index.html target=_parent>luck</a> here!</pre>
<script>document.cookie = "page=overview.html"</script>
W (18664) httpd_uri: httpd_uri: uri handler execution failed
E (19124) SERVER HELP: Failed to read existing file: /sdcard/html/192.168.178.54
W (19124) httpd_txrx: httpd_resp_send_err: 404 Not Found - <pre>

        _
    .__(.)< ( oh oh! This page does not exist! )
    \___)

                You could try your <a href=index.html target=_parent>luck</a> here!</pre>
<script>document.cookie = "page=overview.html"</script>
W (19154) httpd_uri: httpd_uri: uri handler execution failed
W (19274) httpd_txrx: httpd_sock_err: error in send : 104
W (19274) httpd_txrx: httpd_sock_err: error in recv : 104
W (19554) httpd_txrx: httpd_sock_err: error in send : 104
E (19564) SERVER HELP: File sending failed!
W (19564) httpd_txrx: httpd_sock_err: error in send : 104
W (19564) httpd_txrx: httpd_resp_send_err: 500 Internal Server Error - Failed to send file
W (19574) httpd_txrx: httpd_resp_send_err: error calling setsockopt : 22
W (19584) httpd_txrx: httpd_sock_err: error in send : 128
W (19584) httpd_txrx: httpd_sock_err: error in send : 128
W (19594) httpd_uri: httpd_uri: uri handler execution failed
W (19604) httpd_txrx: httpd_sock_err: error in recv : 104
W (19714) httpd_txrx: httpd_sock_err: error in send : 104
E (19724) SERVER HELP: File sending failed!
W (19724) httpd_txrx: httpd_sock_err: error in send : 104
W (19724) httpd_txrx: httpd_resp_send_err: 500 Internal Server Error - Failed to send file
W (19734) httpd_txrx: httpd_resp_send_err: error calling setsockopt : 22
W (19744) httpd_txrx: httpd_sock_err: error in send : 128
W (19744) httpd_txrx: httpd_sock_err: error in send : 128
W (19754) httpd_uri: httpd_uri: uri handler execution failed
W (20544) httpd_txrx: httpd_sock_err: error in recv : 104
W (22464) httpd_txrx: httpd_sock_err: error in recv : 104
I (23814) LOGFILE: Log Level set to ERROR
I (23814) SNTP: Time zone set to CET-1CEST,M3.5.0,M10.5.0/3
I (23814) SNTP: Set SNTP-Server to xxx.xxx.xxx.x
I (23814) SNTP: Waiting until we get a time from the NTP server xxx.xxx.xxx.x
I (23824) SNTP: Status: Reset
I (25834) SNTP: Status: Reset
I (27834) SNTP: Status OK
I (27834) SNTP: Time is synced with NTP Server 192.168.178.1
I (27834) GPIO: read GPIO config and init GPIO
I (27854) GPIO: Enable GPIO12 in external-flash-ws281x mode
I (27854) gpio: GPIO[12]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
I (27854) GPIO: GPIO init completed, is enabled
E (27854) MQTT IF: Init with no Client_ID () or Last Will Topic (). Abort Init!
I (27874) TFLITE SERVER: Round #1 started
E (27914) MQTT IF: Init failed, skipping all MQTT publishings in this round!
E (27914) MQTT IF: Init with no Client_ID () or Last Will Topic (). Abort Init!
E (28004) MQTT IF: Init failed, skipping all MQTT publishings in this round!
I (28054) GPIO: init SmartLed: LEDNumber=10, GPIO=12
I (51564) ov2640: Set PLL: clk_2x: 0, clk_div: 0, pclk_auto: 0, pclk_div: 8
I (55214) CAM: JPG: 19KB 852ms
W (70124) CNN: Value Rejected due to Threshold (Fit: 0.492364Threshold: 0.500000)
W (75974) CNN: Value Rejected due to Threshold (Fit: 0.412414Threshold: 0.500000)
I (80284) POSTPROC: main: Raw: 7NN.2221, Value: , Status: Neg. Rate - Read:  - Raw: 7NN.2221 - Pre: 777.4333 
I (80354) LOGFILE: Remove old log files
I (80434) LOGFILE: Remove old data files
I (80444) TFLITE SERVER: Round #1 completed
I (109824) ov2640: Set PLL: clk_2x: 0, clk_div: 0, pclk_auto: 0, pclk_div: 8
I (112464) CAM: JPG: 18KB 96ms
I (156314) ov2640: Set PLL: clk_2x: 0, clk_div: 0, pclk_auto: 0, pclk_div: 8
I (159024) CAM: JPG: 22KB 168ms
I (207884) TFLITE SERVER: Round #2 started
E (207884) MQTT IF: Init with no Client_ID () or Last Will Topic (). Abort Init!
E (207934) MQTT IF: Init failed, skipping all MQTT publishings in this round!
I (260404) POSTPROC: main: Raw: 777.3433, Value: , Status: Neg. Rate - Read:  - Raw: 777.3433 - Pre: 777.4333 
I (260454) LOGFILE: Remove old log files
I (260464) LOGFILE: Remove old data files
I (260464) TFLITE SERVER: Round #2 completed
I (386324) ov2640: Set PLL: clk_2x: 0, clk_div: 0, pclk_auto: 0, pclk_div: 8
I (387874) TFLITE SERVER: Round #3 started
E (387874) MQTT IF: Init with no Client_ID () or Last Will Topic (). Abort Init!
E (387914) MQTT IF: Init failed, skipping all MQTT publishings in this round!
I (388974) CAM: JPG: 22KB 117ms
I (440254) POSTPROC: main: Raw: 777.4343, Value: 777.4343, Status: no error
I (440464) LOGFILE: Remove old log files
I (440474) LOGFILE: Remove old data files
I (440484) TFLITE SERVER: Round #3 completed
I (567874) TFLITE SERVER: Round #4 started
E (567874) MQTT IF: Init with no Client_ID () or Last Will Topic (). Abort Init!
E (567914) MQTT IF: Init failed, skipping all MQTT publishings in this round!

Expected Behavior

The LEDs should light up analogously to the built-in flashlight for a short amount of time in the color specified on the settings page.

Screenshots

No response

Additional Context

No response

jomjol commented 1 year ago

First: /capture_with_flashlight does not support external LEDStrips - it is an old function that should either be deactivated or updated. I'll make an issue for this. If you see different colors, then most propobly the driver for the LEDs is not working properly. Did you set the right type of LED: grafik

As already mentioned there, the WS2813 is not tested and might therefore not work.

pvogt09 commented 1 year ago

OK, good to know that it does not work in /capture_with_flashlight. I find that functionality handy to simply take a picture from the camera or is there a better way to do this so I would rather opt for keeping the function and updating it?

My configuration for the GPIOs is:

;IO0 = input disabled 10 false false 
;IO1 = input disabled 10 false false 
;IO3 = input disabled 10 false false 
;IO4 = built-in-led disabled 10 false false 
IO12 = external-flash-ws281x disabled 10 false false 
;IO13 = input-pullup disabled 10 false false 
LEDType = WS2813
LEDNumbers = 10
LEDColor = 255 50 50

That should be correct.

I read that it is untested but thought that it might just work anyways because somebody in #406 seemed to at least make his LEDs light up despite the instability he encountered.

Can I assist somehow on making the driver work and test?

jomjol commented 1 year ago

I need to revert: /capture_with_flashlight is supporting also the external LED-light - varyfied in code and with testing on my own systems. Problem must have a different reason.

Regarding the driver test, one would need the corresponding LEDs. But instead it might be easier to switch to tested versions (WS2812)

pvogt09 commented 1 year ago

I have added a diode to VCC of the LEDs to get the voltage level compatible to 3.3V and played around with the timings T0H, T0L, T1H and T1L here and can confirm that the /capture_with_flashlight uses the LEDs. I managed to get the right color on some of the LEDs and they are turned of as well after the picture is taken but not all of them light up as expected. How did you determine the timing values for the other types? The datasheet has rather large intervals of allowed times (e.g. 580ns up to 1600ns) and I don't know which values to are suitable apart from simply testing and reflashing all the time. I also bought a supported strip and hope that it arrives next week, but it would nevertheless be a good thing to get the WS2813 working as well.

jomjol commented 1 year ago

I used a avalable library, so I did not determine the timing by myself

caco3 commented 1 year ago

I think the timing is defined here: https://github.com/jomjol/AI-on-the-edge-device/blob/069aac5723d28dfb63bf81c8c29d696765787235/code/components/jomjol_controlGPIO/SmartLeds.h#L98

If you get it working, please create a Pull request and also remove the "not tested" at https://github.com/jomjol/AI-on-the-edge-device/blob/45a3138d28a4b1e3c0cbd84b4b34ef2e5cbccbf0/sd-card/html/edit_config_param.html#L1136

pvogt09 commented 1 year ago

I have tested a little bit more and it seems that (at least) for the LED product I have, it is necessary to use a level shifter to get things working. I am using the circuit in the picture and with that it is possible to use the WS2813 without changes in the implemented timing values. Sometimes the first LED stays off or the whole chain stays illuminated when it is supposed to be off, but that might be a problem of the hardware. WS2813 To make this circuit work it is necessary to burn the efuse value VDD_SDIO to 3.3V because otherwise the ESP will get stuck in a boot loop since the PSRAM is powered with 1.8V then. I cannot say whether this applies to all types of WS2813 boards or strips or only to the one I currently have, but it might be worth adding a hint to the documentation.

jomjol commented 1 year ago

Problem has been solved on the hardware side.