Can't set Wifi SSID/password for Panlee project

[Rx7man]

Followed all the instructions in flashing the ESP32-S3 dev kit (Panlee is the only project available for it it seems) and I'm not able to set my wifi SSID/password, no dialog comes up, I've reset the device, reloaded the browser, etc (Chrome) and no dice

Here's the console output

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x44c
load:0x403c9700,len:0xbe4
load:0x403cc700,len:0x2a68
entry 0x403c98d4
E (268) esp_core_dump_flash: No core dump partition found!
E (268) esp_core_dump_flash: No core dump partition found!
Replacing Idle Tasks with TaskManager...
(I) (PrintOutputHeader)(C1) NightDriverStrip
(I) 
(I) (PrintOutputHeader)(C1) ------------------------------------------------------------------------------------------------------------
(I) (PrintOutputHeader)(C1) M5STICKC: 0, USE_M5DISPLAY: 0, USE_OLED: 0, USE_TFTSPI: 1, USE_LCD: 0, USE_AUDIO: 0, ENABLE_REMOTE: 0
(I) (PrintOutputHeader)(C1) ESP32 PSRAM Init: OK
(I) (PrintOutputHeader)(C1) Version 40: Wifi SSID: "" - ESP32 Free Memory: 287632, PSRAM:2095055, PSRAM Free: 2094659
(I) (PrintOutputHeader)(C1) ESP32 Clock Freq : 240 MHz
(I) (setup)(C1) Startup!
(I) (setup)(C1) Starting DebugLoopTaskEntry
>> Launching Debug Thread.  Mem: 287632, LargestBlk: 262132, PSRAM Free: 2094659/2095055, >> Launching JSON Writer Thread.  Mem: 279064, LargestBlk: 262132, PSRAM Free: 2094623/2095039, (W) (DeviceConfig)(C1) DeviceConfig could not be loaded from JSON, using defaults
(W) (NotifyJSONWriterThread)(C1) >> Notifying JSON Writer Thread
(W) (setup)(C1) Starting ImprovSerial
(W) (ReadWiFiConfig)(C1) Retrieved SSID and Password from NVS: , ********
(W) (setup)(C1) Creating TFT Screen
[  1057][E][esp32-hal-gpio.c:102] __pinMode(): Invalid pin selected
E (793) gpio: gpio_set_level(226): GPIO output gpio_num error
[  1068][E][esp32-hal-gpio.c:102] __pinMode(): Invalid pin selected
E (805) gpio: gpio_set_level(226): GPIO output gpio_num error
E (816) gpio: gpio_set_level(226): GPIO output gpio_num error
E (837) gpio: gpio_set_level(226): GPIO output gpio_num error
(W) (InitializeHardware)(C1) Allocating LEDStripGFX for channel 0
(I) (AddLEDsToFastLED)(C1) Adding LEDs to FastLED...
(I) (AddLEDsToFastLED)(C1) Adding 1152 LEDs to pin 5 from channel 0 on FastLED.
(W) (SetupBufferManagers)(C1) Reserving 312 LED buffers for a total of 1088256 bytes...
(W) (InitEffectsManager)(C1) InitEffectsManager...
(I) (InitEffectsManager)(C1) Creating EffectManager using default effects
>> Launching Drawing Thread.  Mem: 265900, LargestBlk: 249844, PSRAM Free: 986607/2084863, (W) (DrawLoopTaskEntry)(C1) >> DrawLoopTaskEntry
(W) 
(W) (DrawLoopTaskEntry)(C1) Entering main draw loop!
>> Launching Screen Thread.  Mem: 261428, LargestBlk: 245748, PSRAM Free: 986607/2084863, (I) (setup)(C1) Calling ConnectToWifi()
(I) 
(I) (ConnectToWiFi)(C1) Setting host name to NightDriverStrip...WL_NO_SHIELD
(W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect.
(W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set...
(W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect.
(W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set...
(W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect.
(W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set...
(W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect.
(W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set...

On a side note, if you have the text size increased in your browser you can't see the whole page and there are no scroll bars.. I got that figured out pretty quick though

[rbergen]

@Rx7man Thanks for flagging this. Lacking the device in question, I unfortunately can't replicate or debug the WiFi credential setting issue at my end.

What I can say is that the logging looks "healthy" to me, at least to the point that setting SSID and password should be possible - specifically the line stating that ImprovSerial is starting is relevant in this context.

The mention of the text sizing/scrollbar issue is very useful. I will look into that.

[smiliea]

I would start by deleting the cached .json files in the .pio directory. I believe they get stored (encrypted?) in one of these files the first time the pio project is compiled, and if you change the SSID or password, the old SSID gets used.

I was having similar issues for a few weeks, however my solution was to do a factory reset on my wireless router. Hopefully you won’t need to do this! I had done all I could think of and that was my last resort.

On Wed, Sep 13, 2023 at 3:19 AM Rutger van Bergen @.***> wrote:

@Rx7man https://github.com/Rx7man Thanks for flagging this. Lacking the device in question, I unfortunately can't replicate or debug the WiFi credential setting issue at my end.

What I can say is that the logging looks "healthy" to me, at least to the point that setting SSID and password should be possible - specifically the line stating that ImprovSerial is starting is relevant in this context.

The mention of the text sizing/scrollbar issue is very useful. I will look into that.

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1717164650, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF2ATVBDLUUUZPYFJMDVVYTX2FUCHANCNFSM6AAAAAA4V2725Q . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

[Rx7man]

I would start by deleting the cached .json files in the .pio directory. I believe they get stored (encrypted?) in one of these files the first time the pio project is compiled, and if you change the SSID or password, the old SSID gets used. I was having similar issues for a few weeks, however my solution was to do a factory reset on my wireless router. Hopefully you won’t need to do this! I had done all I could think of and that was my last resort. … On Wed, Sep 13, 2023 at 3:19 AM Rutger van Bergen @.> wrote: @Rx7man https://github.com/Rx7man Thanks for flagging this. Lacking the device in question, I unfortunately can't replicate or debug the WiFi credential setting issue at my end. What I can say is that the logging looks "healthy" to me, at least to the point that setting SSID and password should be possible - specifically the line stating that ImprovSerial is starting is relevant in this context. The mention of the text sizing/scrollbar issue is very useful. I will look into that. — Reply to this email directly, view it on GitHub <#425 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF2ATVBDLUUUZPYFJMDVVYTX2FUCHANCNFSM6AAAAAA4V2725Q . You are receiving this because you are subscribed to this thread.Message ID: @. com>

Where do I find this file? It's the first time I do anything with web install

This is the specific device I got, https://www.amazon.ca/dp/B0B6HT7V7P?psc=1&ref=ppx_yo2ov_dt_b_product_details

I tried installing it on a different model (ESP32-wroom) and I get the same thing, no place to ever enter the SSID

That device doesn't reconnect to the terminal automatically when I reset it so I wasn't getting the boot messages, connected with Putty and got them..

Above my pay grade to implement, but would it be possible to enter the SSID/Password over serial?

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:1184
load:0x40078000,len:13232
load:0x40080400,len:3028
entry 0x400805e4
E (585) esp_core_dump_flash: No core dum▒▒▒ѥѥ▒▒▒found!
E (585) esp_core_dump_flash: No core dump partition found!
Replacing Idle Tasks with TaskManager...
                                        (I) (PrintOutputHeader)(C1) NightDriverStrip
(I)
(I) (PrintOutputHeader)(C1) ------------------------------------------------------------------------------------------------------------
(I) (PrintOutputHeader)(C1) M5STICKC: 0, USE_M5DISPLAY: 0, USE_OLED: 0, USE_TFTSPI: 0, USE_LCD: 0, USE_AUDIO: 0, ENABLE_REMOTE: 0
(I) (PrintOutputHeader)(C1) Version 40: Wifi SSID: "" - ESP32 Free Memory: 215336, PSRAM:0, PSRAM Free: 0
(I) (PrintOutputHeader)(C1) ESP32 Clock Freq : 240 MHz
(I) (setup)(C1) Startup!
(I) (setup)(C1) Starting DebugLoopTaskEntry
>> Launching Debug Thread.  Mem: 215336, LargestBlk: 110580, PSRAM Free: 0/0, >> Launching JSON Writer Thread.  Mem: 206732, LargestBlk: 110580, PSRAM Free: 0/0, (W) (DeviceConfig)(C1) DeviceConfig could not be loaded from JSON, using defaults
(W) (NotifyJSONWriterThread)(C1) >> Notifying JSON Writer Thread
(W) (setup)(C1) Starting ImprovSerial
(W) (ReadWiFiConfig)(C1) Retrieved SSID and Password from NVS: , ********
(W) (InitializeHardware)(C1) Allocating LEDStripGFX for channel 0
(I) (AddLEDsToFastLED)(C1) Adding LEDs to FastLED...
(I) (AddLEDsToFastLED)(C1) Adding 1152 LEDs to pin 5 from channel 0 on FastLED.
(W) (SetupBufferManagers)(C1) Reserving 12 LED buffers for a total of 41856 bytes...
(W) (InitEffectsManager)(C1) InitEffectsManager...
(I) (InitEffectsManager)(C1) Creating EffectManager using default effects
>> Launching Drawing Thread.  Mem: 151212, LargestBlk: 106484, PSRAM Free: 0/0, (W) (DrawLoopTaskEntry)(C1) >> DrawLoopTaskEntry
(W)
(W) (DrawLoopTaskEntry)(C1) Entering main draw loop!
(I) (setup)(C1) Calling ConnectToWifi()
(I)
(I) (ConnectToWiFi)(C1) Setting host name to NightDriverStrip...WL_NO_SHIELD
(W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect.
(W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set...
(W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect.
(W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set...

[smiliea]

I don’t know why this went over my head, but I didn’t realize that your original post was regarding the web installer. I have used the web install interface only a few times, and I am pretty sure I ran into this same issue with the SSID and Wi-Fi password, so I just moved to using VSCode and PlatformIO and didn’t spend too much time on it. I think you will find you have much more control setting up, building and deploying going this route. However, this definitely needs to be addressed. The web install is a great tool and is going to get people started with this project quick.

I will see if I can replicate this issue with one of my “HiLetgo ESP-32 Wi-Fi+Bluetooth” dev boards and see if I can help out.

On Wed, Sep 13, 2023 at 10:54 AM Rx7man @.***> wrote:

I would start by deleting the cached .json files in the .pio directory. I believe they get stored (encrypted?) in one of these files the first time the pio project is compiled, and if you change the SSID or password, the old SSID gets used. I was having similar issues for a few weeks, however my solution was to do a factory reset on my wireless router. Hopefully you won’t need to do this! I had done all I could think of and that was my last resort. … <#m_8780031045454978923m-835555331415377564_> On Wed, Sep 13, 2023 at 3:19 AM Rutger van Bergen @.> wrote: @Rx7man https://github.com/Rx7man https://github.com/Rx7man https://github.com/Rx7man Thanks for flagging this. Lacking the device in question, I unfortunately can't replicate or debug the WiFi credential setting issue at my end. What I can say is that the logging looks "healthy" to me, at least to the point that setting SSID and password should be possible - specifically the line stating that ImprovSerial is starting is relevant in this context. The mention of the text sizing/scrollbar issue is very useful. I will look into that. — Reply to this email directly, view it on GitHub <#425 (comment) https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1717164650>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF2ATVBDLUUUZPYFJMDVVYTX2FUCHANCNFSM6AAAAAA4V2725Q https://github.com/notifications/unsubscribe-auth/AF2ATVBDLUUUZPYFJMDVVYTX2FUCHANCNFSM6AAAAAA4V2725Q . You are receiving this because you are subscribed to this thread.Message ID: @. com>

Where do I find this file? It's the first time I do anything with web install

This is the specific device I got, https://www.amazon.ca/dp/B0B6HT7V7P?psc=1&ref=ppx_yo2ov_dt_b_product_details

I tried installing it on a different model (ESP32-wroom) and I get the same thing, no place to ever enter the SSID [image: image] https://user-images.githubusercontent.com/79332079/267712800-431282dd-fca4-43c2-b1a2-944d8d3b866f.png [image: image] https://user-images.githubusercontent.com/79332079/267714810-644c2988-ff4e-425c-a0ef-dd1e5759a773.png

That device doesn't reconnect to the terminal automatically when I reset it so I wasn't getting the boot messages, connected with Putty and got them..

Above my pay grade to implement, but would it be possible to enter the SSID/Password over serial?

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0030,len:1184 load:0x40078000,len:13232 load:0x40080400,len:3028 entry 0x400805e4 E (585) esp_core_dump_flash: No core dum▒▒▒ѥѥ▒▒▒found! E (585) esp_core_dump_flash: No core dump partition found! Replacing Idle Tasks with TaskManager... (I) (PrintOutputHeader)(C1) NightDriverStrip (I) (I) (PrintOutputHeader)(C1) ------------------------------------------------------------------------------------------------------------ (I) (PrintOutputHeader)(C1) M5STICKC: 0, USE_M5DISPLAY: 0, USE_OLED: 0, USE_TFTSPI: 0, USE_LCD: 0, USE_AUDIO: 0, ENABLE_REMOTE: 0 (I) (PrintOutputHeader)(C1) Version 40: Wifi SSID: "" - ESP32 Free Memory: 215336, PSRAM:0, PSRAM Free: 0 (I) (PrintOutputHeader)(C1) ESP32 Clock Freq : 240 MHz (I) (setup)(C1) Startup! (I) (setup)(C1) Starting DebugLoopTaskEntry

Launching Debug Thread. Mem: 215336, LargestBlk: 110580, PSRAM Free: 0/0, >> Launching JSON Writer Thread. Mem: 206732, LargestBlk: 110580, PSRAM Free: 0/0, (W) (DeviceConfig)(C1) DeviceConfig could not be loaded from JSON, using defaults (W) (NotifyJSONWriterThread)(C1) >> Notifying JSON Writer Thread (W) (setup)(C1) Starting ImprovSerial (W) (ReadWiFiConfig)(C1) Retrieved SSID and Password from NVS: , **** (W) (InitializeHardware)(C1) Allocating LEDStripGFX for channel 0 (I) (AddLEDsToFastLED)(C1) Adding LEDs to FastLED... (I) (AddLEDsToFastLED)(C1) Adding 1152 LEDs to pin 5 from channel 0 on FastLED. (W) (SetupBufferManagers)(C1) Reserving 12 LED buffers for a total of 41856 bytes... (W) (InitEffectsManager)(C1) InitEffectsManager... (I) (InitEffectsManager)(C1) Creating EffectManager using default effects Launching Drawing Thread. Mem: 151212, LargestBlk: 106484, PSRAM Free: 0/0, (W) (DrawLoopTaskEntry)(C1) >> DrawLoopTaskEntry (W) (W) (DrawLoopTaskEntry)(C1) Entering main draw loop! (I) (setup)(C1) Calling ConnectToWifi() (I) (I) (ConnectToWiFi)(C1) Setting host name to NightDriverStrip...WL_NO_SHIELD (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set...

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1717902314, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF2ATVAZMVK3TEDFIFRBQO3X2HJM3ANCNFSM6AAAAAA4V2725Q . You are receiving this because you commented.Message ID: @.***>

[rbergen]

A fundamental challenge with the Web Installer is that the web application is entirely dependent on the ESP Web Tools project for the actual flashing of images and the subsequent configuration of WiFi details. That is to say that "at the browser side", everything beyond the logic to select a device and project is effectively a black box.

The configuration of WiFi details requires the implementation of the "improv_serial" protocol in the NightDriverStrip image running on the board. The implementation that's in place (in the file improvserial.h to be precise), is based on the improv_serial implementation in ESPHome, itself based on C++ SDK for Improv Wi-Fi. I'm not sure exactly what version of the ESPHome implementation ours is currently based on, but it's possible that ours has become misaligned with the current state of ESP Web Tools over time.

I think assessing this would require a "line-by-line" comparison of our improv_serial code to the one in ESPHome. That's a job I'd personally describe as "not a lot of fun when you're at it, but very rewarding if it solves the problem."

The other approach - which one could apply first - is adding some low-level debug logging to our improvserial.h as it stands. That should bring to the surface any situations where we expect a different set of bytes to come in than we're getting. That might lead us on a path to fixing that, maybe by increasing our fault tolerance and/or improving our recovery logic.

[Rx7man]

I'm not in a rush to play with this, so it's a back burner project.. part of the reason I figured trying the web installer would be a neat thing to try. I haven't played much with VS Code, only used Visual studio with Visual Micro for Arduino programming.

[rbergen]

I'm happy to pick this up myself to the extent that I can make the changes blind. The "problem" is that I can't test the scenarios that are now failing. That is to say that all the devices I have do allow their WiFi credentials to be set via the Web Installer. Without someone willing and able to test the changes I (or anyone else) would make, making them is pretty pointless.

[Rx7man]

I'd test them.. just seems strange that neither of my devices work seeing they're the most common ESP32's

[KeiranHines]

I noticed when I flashed my mesmerizer I had to connect it to the web installer. Close the dialog and reopen it a second time for the wifi credentials option to show, sometimes it only showed the open logs button.

You could possibly try connecting to the web installer, leaving the device connected for a minute or two, then trying again.

[robertlipe]

If Rutger (or another with an appetite for grubbing around in systems stuff) had a device of an approximately similar pedigree (i.e. an "esp32 s3" - regardless of which pins were connected to what on the TTGO vs. Panlee or whatever) would it be possible to take a "core dump"of the affected device, ship it over the internet to the inspector[s], and study the image on another ESP-32? I think that board is pretty close to these https://www.aliexpress.us/item/2251832649206668.html and I have a few like that. If I had a device that could repro it - especially one I could modify for testing and then roll back to original state if I needed it - I'd help take a look, though my web-updater-fu is weak.

Would something approximately like: esptool -p /dev/cu.usb*101 read_flash 0x0 0x400000 dump-4M.bin and then offering dump-4M allow Rutger, Dave, or, if you're truly desperate, myself, to load that first 4MB of core which would contain things like the partition table as well as the beginning of the filesystems, etc. for study on a sibling device? Having access to the build tree for that exact image to correlate addresses to source would, of course, be super helpful, but i don't know if those are kept for web images.

This kind of looks like flash/filesystem corruption. The 8+MB (common on S3) had a change https://github.com/PlummersSoftwareLLC/NightDriverStrip/commit/ad6f72846f10f373fa1efb16d1558674cdd0f0c1 to the partitioning scheme and depending on the order of installation date, OTA upgrade cycle hitting on A or B partition (a couple of submits in that same approximate time frame, it probably possible to have had a system that lived in one partition but then OTAed or web updated to another version that depended on a different partition table that had the filesystem starting in a different place and was therefore trashed.

It would be a total bear to debug, but if that's the case, an esptool [ erase the device ] followed by a re-installation would be likely to both "cure" the affected device and destroy the evidence trail needed to confirm that the changed partition table was to blame.

I don't know if the ESPHome stuff is inherently better or worse than Espressif's own tools like Rainmaker https://rainmaker.espressif.com/docs/ota.html or web https://github.com/espressif/esp-idf/issues/9434 updater. It does seem like a pretty established project in widespread use.

Rx7Man, you mentioned your boards, the ESP32-S3-DevKitC-1-N8R2, being the most popular boards around. That may be true for some projects, but I've had teh feeling that S3's were under-represented here, perhaps because the official mesmerizer hardware uses the ESP32-nothing (man, I hate their names....) or because running small numbers of LEDs on a strip just doesn't require the memory or bus advances (the memory and SPI subsystems are just WAY faster) of the S3 and most people were just using the base form because they were so inexpensive. So I'm not sure the population of S3's in this project is representative of the popularity of the S3 in the advanced hacker/hobbyist market.

So, if you have yet another S3 on hand and can confirm that transferring a memory dump from the failing unit to a new device under test, I'd be interested in both A) seeing a copy of that failing image and B) seeing if an esptool.py erase_flash "heals" it enough that it can then be recovered.

On Thu, Sep 14, 2023 at 3:23 PM Keiran Hines @.***> wrote:

I noticed when I flashed my mesmerizer I had to connect it to the web installer. Close the dialog and reopen it a second time for the wifi credentials option to show, sometimes it only showed the open logs button.

You could possibly try connecting to the web installer, leaving the device connected for a minute or two, then trying again.

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1720092644, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD354YEJME56UW6FF4ETX2NRTTANCNFSM6AAAAAA4V2725Q . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

[Rx7man]

I have both the ESP32-wroom and the ESP32-S3-wroom-1 (a couple of each) and neither work.. I never mentioned it but I am on the latest version of Chrome and Windows 11

I don't know how to do a memory dump but I'll gladly do that if it helps and you walk me through it

[rbergen]

This may just be an oddness in my personal wiring, but I think that for me it will be most helpful right now if I can see exactly what comes into the board from the browser at the (improv_)serial level, and what it tries to send out. There is debug logging code in the relevant files at the moment, but that's currently not switched on.

When I have the time, I'll create a branch with low-level improv_serial debug logging switched on in a way that it outputs to the same channel as the rest of the logging we've been looking at so far (i.e. the console). Once we can see what the firmware receives and doesn't from the browser at a character/byte level, we can hopefully figure out what's misaligned with what at the moment.

[rbergen]

I've spent a few more brain cycles on this and it dawned on me that just enabling debug logging is not going to work, because when Improv is supposedly trying to do its thing concerning setting up WiFi etc., no logging will actually come through. So, what I'll do instead is have the Improv logging persisted to SPIFFS (when told to do so), and update the on-board web server to allow that log to be retrieved. As this is something we may very well need again in the future, I want to implement it "properly". That does mean it will take me a little longer to finish this.

I'm currently finishing off some work to add a new set of effects to the Mesmerizer project, and I'll pick this up after.

[rbergen]

I just merged #426 which fixes the "missing scrollbar" sidenote issue. It'll show in the Web Installer when CI finishes - it includes the deployment to GitHub Pages.

[rbergen]

@Rx7man I've created an "add-improv-logging" branch in my fork (https://github.com/rbergen/NightDriverStrip/tree/add-improv-logging) that includes the ability to write Improv debug logging to SPIFFS, and retrieve it via the webserver.

The capability can be enabled by defining ENABLE_IMPROV_LOGGING towards the top of improvserial.h, or passing -DENABLE_IMPROV_LOGGING=1 as a build flag.
Once the log has been generated, it can be retrieved from the device using http://&lt;device IP>/improv.log

Disclaimer: it's untested - there's not much point in me doing so, because with my gear Improv basically works as it should.

Maybe you can give it a run with ENABLE_IMPROV_LOGGING set to 1 when you have a few minutes, try to config WiFi with the Web Installer, and let me know whatever logging that comes out of that?

[Rx7man]

I am a total noob at this, but I tried installing again and now the logs only say "Terminal disconnected: Break received" I'm not sure how to set flags, etc on the web installer This is what I got. I unplugged the device and did a hard reboot, same thing

[Rx7man]

Tried it on the ESP32-S3 and got the "can't connect"

esp-web-tools-logs (1).txt

[rbergen]

My apologies, I think we need to take a step back here.

The version of NightDriverStrip that I created can't be installed using the Web Installer. It needs to be built and uploaded via PlatformIO. That means that:

• The source code needs to be downloaded from the "add-improv-logging" branch in my fork. You can downloaded it in a ZIP file from the URL in my previous comment: https://github.com/rbergen/NightDriverStrip/tree/add-improv-logging
• After unpacking the ZIP file, the 0 in line 46 of file include/improvserial.h (#define ENABLE_IMPROV_LOGGING 0) needs to be changed to a 1.
• PlatformIO has to be installed and used to build your downloaded and now modified version of NightDriverStrip. You can find information about this in the "Build pointers" section in the README in this project. Note that your WiFi network details need to be specified before the build, using the instructions in the "Wifi setup" section in the same README.
• The firmware image that the build creates needs to be uploaded to your device over USB using PlatformIO as well.

When this is done, your device should boot with the right version of NightDriverStrip, and connect to WiFi on startup. With all that in place and it (still) connected to USB, you can now access the Web Installer again, and try to (re)configure WiFi on your device using it. While you do this, a log file should be created on your device that documents exactly what data is sent back and forth between your web browser and the device. When you're done testing, that log file can be retrieved using the IP address your device was issued by your WiFi access point, via the following URL: http:///improv.log. That's the log file I would need to hopefully be able to analyze what's going on.

[Rx7man]

I'm really not good with command line stuff, but I just can't get it to find PIO.exe, followed all the instructions to add it to the path and it's just not working.. I have some things to do today and I'll get back to this later.. I did make a copy of secrets.h and put my wifi info into it, as well as modified the improvserial.h to debug= 1

[rbergen]

If you install the PlatformIO IDE (i.e. the VSCode plugin) then you don't need to separately make the command-line tools available. They'll be part of the package and available if you open a PlatformIO Core CLI terminal through the plugin in VSCode.

Even better: with the VSCode/PlatformIO combo, you can do the whole thing of building (including environment/project selection) and uploading (including port selection) from within VSCode.

[Rx7man]

I did install platformIO through VScode.. I'm also getting the red squiggly line under the improv.h include.. I see it there in the directory with everything else

[Rx7man]

It sees Hexdump.h but not improv.h

[rbergen]

I've learned that red squigglies (or the absence of them) don't mean too much until you've actually performed a build. So, my recommendation would be to use VSCode/PlatformIO to create a build for the environment/project of your choosing, and see how far you get with that.

[Rx7man]

I'm not sure where to go from here and what to do.. perhaps this is a little over my pay grade

[dosipod]

We are working a lot with open source firmware and can confirm that the site https://plummerssoftwarellc.github.io/NightDriverStrip/ does not work with the same issue the OP has posted , this is the issue faced with multiple esp32. To compare ( or for devs to borrow the correct code for web installer ) try the following open source web installer

WLED https://install.wled.me/v2.htm or https://wled-install.github.io/ , source is (https://github.com/Aircoookie/WLED and web installer source is https://github.com/Aircoookie/WLED-WebInstaller )

Tasmota https://tasmota.github.io/install/

[rbergen]

I'm not sure where to go from here and what to do.. perhaps this is a little over my pay grade

@Rx7man That's ok. I'll just have to wait for someone else with a non-functioning chip to take note of this issue and generate the logging I'm asking for.

Thanks for giving it a shot, regardless!

[robertlipe]

I've admittedly not followed every post (that quote seems to be trendy) but I see rope endings in sight. Is this believed to be an issue specific to this board (e.g. some state it's in that a flash erase won't cure or a chip that's just plain broken or a model that requires an external that doesn't have one) or specific to this model of board (if one of us has that exact model delivered, would we see the same?) or is it specific to something about rx7man's Wifi?

From rx7man's profile pic, he and I may be in the same country. (I'm in Nashville, TN, US.) I won't make any guesses about relative pay grades, but I may have a different experience level in something relevant. If there's some way I can repro it here, I'm willing to be Rutger's fingers and eyes. ... dupont-ing up and tunneling a debugger if that's necessary/helpful.

I'm willing to toss a few bucks into shipping costs if it makes sense to send it here or a few more bucks if this is one of the 'bare development board' variety but not really (sorry) if it's a $60 jobber with a touch screen and a toaster oven.

If it really is a ESP32-S3-DevKitC-1 as shown in the picture, my next-to-last Ali order included https://www.aliexpress.us/item/3256805600763040.html which I've not really messed with, but looking at the pictures, they can't be THAT different - WROOM with LVDO, an on-board WS2812, and an inexplicable external UART. I've been using them (and some YD-something's that are similar) on another project and haven't really messed with them in this code, but can if there's a reason.

I'm willing to try repro if someone can communicate the needed steps precisely. Honestly, looking at the logs, I'm skeptical that it's hardware specific - it's doing way too much "computer stuff" to just be broken.

Now, for a bunch of "have you tried plugging it in" questions that may have already been tried before, but just as a pair of fresh eyes...

Rx7man, does your Wifi environment have anything weird going on? A password that's weird (e.g. multi-byte characters or control characters) WiFi security levels that block new devices, like a MAC filter or one of those 'guest' networks that's not a "real" login, but that takes you to a Hotel-style guest form to fill out? Password or name over 63 characters long? Remember that these are only 2.4Ghz devices. Some APs use different name/password pairs to connect to 2.4 and 5Ghz network. Are you sure the password is the 2.4Ghz one and verified it on another IoT-class device with similar limitations?

If you (gasp) disable the WiFi password, will this device connect?

Rutger, does "WL_NO_SHIELD" seem alarming? That might make some sense in an Arduino environment where you have to add a board (and they can't call a board a board, it's a "shield" in their programs, err, sketches.) but the absence of finding networking hardware on an ESP32 is pretty alarming since that's literally their claim to fame - including WiFi on the chip.

    debugW("Retrieved SSID and Password from NVS: %s, %s",

WiFi_ssid.c_str(), "****"); but the info printed a blank for the first. There's no SSID!?!?

Rutger, is there any code in this path that should show nearby APs and signal levels just to confirm it's able to see the network radio space at all?

Rx7, If you flash similar projects like ESPHOME or WLED, are the results similar? I think they use the same basic installation library.

LMK out of band (my gmail address is my fullname) if you need a shipping address. On projects like this, it's depressing to spend more on shipping than these things are worth, but we've all alreadyexceded the value of board just in debugging them, so let's get it figured out so we don't keep doing it in the future.

RJL

On Sat, Sep 23, 2023 at 7:29 PM Rutger van Bergen @.***> wrote:

I'm not sure where to go from here and what to do.. perhaps this is a little over my pay grade

@Rx7man https://github.com/Rx7man That's ok. I'll just have to wait for someone else with a non-functioning chip to take note of this issue and generate the logging I'm asking for.

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1732441875, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD34M7NGHCPNJOA26L6DX355FFANCNFSM6AAAAAA4V2725Q . You are receiving this because you commented.Message ID: @.***>

[Rx7man]

So just a little about me, I'm a farmer that's worked with PLCs in industrial environments, done a little bit of VB programming all my life, make gizmos for myself with arduino-ish stuff (VGT turbo controller, etc), but when it comes to "real" programming I am out of my depths I have both the ESP32-wroom and ESP32-S3 and neither of them even give me a prompt to enter the wifi SSID/password (Win11/chrome).. My wifi is terribly insecure as far as passoword strength.. because there's just no one around except me.. I could try this on my Windows 7 computer too I guess

I'm in Canada if that helps.. I think I'm behind 2 NAT's on my internet so I don't think I could set something up like remote desktop I'm just in the process of moving so my hands are pretty full, but I'd love to help where I can..

[robertlipe]

Heh. I always assumed from your name you were a pharmacist, but now I remember a certain Wankle-bearer...

So you have two that don't work. You said you have "a couple of each". Do you have any that DO, or do you just have a couple that are cursed?

If you're never being ASKED for a password, that loop that we see that's failing login is never going to work. ReadWiFiConfig() thinks there's a WiFi name (SSID) - though it doesn't DISPLAY one - and a password, which it intentionally doesn't show for security reasons.

I just offered to butt into the middle because domestic shipping is probably cheaper than the boards themselves. But if you have multiple boards failing, it's probably NOT the boards.

I failed to notice there are two distinct boots in your recent post. I started from a case where you're in ReadWiFiConfig() where it thinks you have a WifiSSID (the AP you're connecting to) but no password. It doesn't print the SSID and if you've never entered a password. So that's weird.

Looking just at the two logs, it looks like it jumps the rails at: (W) (setup)(C1) Starting ImprovSerial (W) (ReadWiFiConfig)(C1) Error (ESP_ERR_NVS_NOT_FOUND) opening NVS handle! (W) (W) (setup)(C1) Could not read WiFI Credentials (W) (WriteWiFiConfig)(C1) Stored SSID and Password to NVS: , ***

It can't find the Non Volatile Storage partition, so whatever SSID/pass it's trying to read is doomed. So it couldn't store SSID into the NVS and the SSID is empty anyway.

On the second boot, it IS able to open the NVS handle, but what it's reading is nonsense. So this time, we do have an NVS partition on the board.

So I was thinking it was actually trying to log into your WiFi, but things are going wrong way earlier.

I'm away from my boards, but I may grab my similar boards and see what happens and, more importantly for butting into someplace, learn more about who actually spawns the UI that asks you for an AP and see if I can guess on what that's not happening. Maybe I'll learn something.

You're using the Panlee configuration, right? Elecrow/Mesmerizer and Lilygo Tdisplay-S3-demo and liliygo tdisplay s3 demo look to be the only other stock s3 configurations.

Honestly, I think the S3 boards are mostly chosen by those of us that are comfortable hatching our own configurations because we're mixing boards with display/effect options.

If you get a shot, try those other choices. I'll get back to my board sometime this weekdend and see if I can match your (anti)-success.

On Sat, Sep 23, 2023 at 10:54 PM Rx7man @.***> wrote:

So just a little about me, I'm a farmer that's worked with PLCs in industrial environments, done a little bit of VB programming all my life, make gizmos for myself with arduino-ish stuff (VGT turbo controller, etc), but when it comes to "real" programming I am out of my depths I have both the ESP32-wroom and ESP32-S3 and neither of them even give me a prompt to enter the wifi SSID/password (Win11/chrome).. My wifi is terribly insecure as far as passoword strength.. because there's just no one around except me.. I could try this on my Windows 7 computer too I guess

I'm in Canada if that helps.. I think I'm behind 2 NAT's on my internet so I don't think I could set something up like remote desktop I'm just in the process of moving so my hands are pretty full, but I'd love to help where I can..

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1732469367, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD33LJLK4IA5SCNOLR73X36VGLANCNFSM6AAAAAA4V2725Q . You are receiving this because you commented.Message ID: @.***>

[Rx7man]

I haven't tested the others, but it's kinda strange that two different model board I have don't work (and I did upload some other arduino sketches to test the wifi and they work just fine) Yes, I am a spinning doritos guy I'll go through your entire message tomorrow when I'm a bit more awake

[rbergen]

I'm pretty sure there's nothing wrong with the boards. If memory serves me, there have been other reports of S3 owners running into trouble when trying to use the Web Installer. The report on the WROOM is new, IIRC.

I'm basing this on nothing but intuition (which has certainly been wrong before), but I'm guessing there is some "static on the line" between the browser/ESP Web Tools and the Improv protocol implementation in NDS. Maybe some characters get lost, maybe we're not processing a message type we should, maybe we're being confronted with an Improv protocol version we don't know and ignore, etc. etc.
That's what the new logging should tell us.

(W) (setup)(C1) Starting ImprovSerial (W) (ReadWiFiConfig)(C1) Error (ESP_ERR_NVS_NOT_FOUND) opening NVS handle! (W) (W) (setup)(C1) Could not read WiFI Credentials (W) (WriteWiFiConfig)(C1) Stored SSID and Password to NVS: , ***

The NVS partition isn't there because the Web Installer fully erased the board's storage before flashing it. Upon determining this (i.e. immediately after reporting the error), the NVS partition is created and formatted on the fly. Which is why the storing of the default empty SSID (and password) to NVS does work "4 lines later".

On subsequent boots the (empty) SSID is loaded from NVS, but obviously still equally useless.

In all cases, the fact that the SSID is empty then puts the networking logic in "waiting mode" for WiFi credentials to be provided via the Web Installer interface/Improv. And that's where things get stuck at the moment.

[rbergen]

Rutger, does "WL_NO_SHIELD" seem alarming? That might make some sense in an Arduino environment where you have to add a board (and they can't call a board a board, it's a "shield" in their programs, err, sketches.) but the absence of finding networking hardware on an ESP32 is pretty alarming since that's literally their claim to fame - including WiFi on the chip.

No, not in the particular spot it shows up. This is what I get on my perfectly functional Mesmerizer and M5StickC Plus as well. I think the code may be trying to set the hostname too soon for the WiFi library's liking, which is something I may look into further at some point.

      debugW("Retrieved SSID and Password from NVS: %s, %s", WiFi_ssid.c_str(), "********");

but the info printed a blank for the first. There's no SSID!?!?

Correct. The Web Installer flashes images with an empty SSID, because it can't know what SSID to use. With the Web Installer, the WiFi creds are supplied after the firmware flash. In fact, the absence of an SSID puts the board in a "waiting mode" for Web-Installer-supplied WiFi credentials.

Rutger, is there any code in this path that should show nearby APs and signal levels just to confirm it's able to see the network radio space at all?

As part of the Improv implementation that should be handling the setting of WiFi creds via the Web Installer, there is indeed logic to identify nearby APs (but I don't think signal levels; I'd have to look at improvserial.h to check). A list of nearby APs is actually sent back to the Web Installer so the user can choose which one to connect to, if everything works as it should.

(I noticed you mentioning me in your earlier comment more or less by accident. Tagging me with an "at handle" will work better - as in by using @rbergen instead of my first name. GitHub will then send me a specific "you were mentioned" notification.)

[rbergen]

I've gone over the Improv code for the umpteenth time just now, and there is a (small) chance I may have found something that can explain the erroneous behavior of the Web Installer with S3-based boards.

Long story short: within the context of ESP Web Tools, the ESP-S3 is a different chip family than the "plain" ESP32. At the moment, the Improv implementation in NDS always identifies itself as an ESP32, due to a fixed string being passed to the ImprovSerial::setup() function in main.cpp (line 325, to be precise). It's possible that after flashing, ESP Web Tools is looking for a device identifying as an ESP32-S3, finds nothing, and concludes there's nothing it can do.

I'll make some updates to my "add-improv-logging" branch so that S3 projects identify themselves as ESP32-S3 via Improv. I'll then open a PR to merge that branch. When that's done, @Rx7man can test if that allows him to configure WiFi after a reflash of his S3 with the Web Installer - this is a test that does not require a build or firmware flash to take place via PlatformIO.

To be continued!

[Rx7man]

I'll gladly test it!

I'm just wondering if there's something in my computer setup that's preventing the select SSID dialog box from popping up, I've disabled uBlock for the site though

[rbergen]

@Rx7man I appreciate you considering the possibility that the problem is at your end, but I'm pretty sure that's not where the issue lies. If the flash dialog does show up, the SSID dialog should also.

The new images have been pushed to the Web Installer, so you could give it another try. Before you do, please consider clearing your browser cache for the Web Installer domain (plummerssoftwarellc.github.io) at least, to ensure you're using the new binaries.

[Rx7man]

Nope, still not working

`Logs (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... ESP-ROM:esp32s3-20210327 Build:Mar 27 2021 rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT) SPIWP:0xee mode:DIO, clock div:1 load:0x3fce3808,len:0x44c load:0x403c9700,len:0xbe4 load:0x403cc700,len:0x2a68 entry 0x403c98d4 E (268) esp_core_dump_flash: No core dump partition found! E (268) esp_core_dump_flash: No core dump partition found! Replacing Idle Tasks with TaskManager... (I) (PrintOutputHeader)(C1) NightDriverStrip (I) (I) (PrintOutputHeader)(C1) ------------------------------------------------------------------------------------------------------------ (I) (PrintOutputHeader)(C1) M5STICKC: 0, USE_M5DISPLAY: 0, USE_OLED: 0, USE_TFTSPI: 1, USE_LCD: 0, USE_AUDIO: 0, ENABLE_REMOTE: 0 (I) (PrintOutputHeader)(C1) ESP32 PSRAM Init: OK (I) (PrintOutputHeader)(C1) Version 40: Wifi SSID: "" - ESP32 Free Memory: 287632, PSRAM:2095055, PSRAM Free: 2094659 (I) (PrintOutputHeader)(C1) ESP32 Clock Freq : 240 MHz (I) (setup)(C1) Startup! (I) (setup)(C1) Starting DebugLoopTaskEntry

Launching Debug Thread. Mem: 287632, LargestBlk: 262132, PSRAM Free: 2094659/2095055, >> Launching JSON Writer Thread. Mem: 279064, LargestBlk: 262132, PSRAM Free: 2094623/2095039, (W) (DeviceConfig)(C1) DeviceConfig could not be loaded from JSON, using defaults (W) (NotifyJSONWriterThread)(C1) >> Notifying JSON Writer Thread (W) (setup)(C1) Starting ImprovSerial for ESP32-S3 [ 1259][E][vfs_api.cpp:182] remove(): /improv.log does not exists or is directory (W) (ReadWiFiConfig)(C1) Retrieved SSID and Password from NVS: "", "****" (W) (setup)(C1) Creating TFT Screen [ 1268][E][esp32-hal-gpio.c:102] __pinMode(): Invalid pin selected E (1004) gpio: gpio_set_level(226): GPIO output gpio_num error [ 1279][E][esp32-hal-gpio.c:102] __pinMode(): Invalid pin selected E (1016) gpio: gpio_set_level(226): GPIO output gpio_num error E (1028) gpio: gpio_set_level(226): GPIO output gpio_num error E (1049) gpio: gpio_set_level(226): GPIO output gpio_num error (W) (InitializeHardware)(C1) Allocating LEDStripGFX for channel 0 (I) (AddLEDsToFastLED)(C1) Adding LEDs to FastLED... (I) (AddLEDsToFastLED)(C1) Adding 1152 LEDs to pin 5 from channel 0 on FastLED. (W) (SetupBufferManagers)(C1) Reserving 312 LED buffers for a total of 1088256 bytes... (W) (InitEffectsManager)(C1) InitEffectsManager... (I) (InitEffectsManager)(C1) Creating EffectManager using default effects Launching Drawing Thread. Mem: 265896, LargestBlk: 249844, PSRAM Free: 986607/2084863, (W) (DrawLoopTaskEntry)(C1) >> DrawLoopTaskEntry (W) (W) (DrawLoopTaskEntry)(C1) Entering main draw loop! Launching Screen Thread. Mem: 261424, LargestBlk: 245748, PSRAM Free: 986607/2084863, (I) (setup)(C1) Calling ConnectToWifi() (I) (I) (ConnectToWiFi)(C1) Setting host name to NightDriverStrip...WL_NO_SHIELD (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 1 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (NotifyJSONWriterThread)(C1) >> Notifying JSON Writer Thread (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (I) (WriteCurrentEffectIndexFile)(C0) Number of bytes written to file /current.cfg: 1 (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (NotifyJSONWriterThread)(C1) >> Notifying JSON Writer Thread (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (I) (WriteCurrentEffectIndexFile)(C0) Number of bytes written to file /current.cfg: 1 (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (W) (NotifyJSONWriterThread)(C1) >> Notifying JSON Writer Thread (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect. (W) (ConnectToWiFi)(C1) Waiting for WiFi credentials to be set. Pass 2 of 99... (I) (WriteCurrentEffectIndexFile)(C0) Number of bytes written to file /current.cfg: 1 (W) (ConnectToWiFi)(C1) WiFi credentials not set, cannot connect.`

[rbergen]

This is where someone like me says "ah well, it was worth a shot".

I think we're back to waiting for someone with an S3 who can generate the logging that has now also been enabled. Regardless, thanks very much for trying!

[Rx7man]

as a contribution to the project I'd send you one if that would help

[rbergen]

That's very kind of you! If you're sure (and please make sure that you are) then I think it's best to reach out to me via email. My address is my GitHub handle followed by "at" and xs4all.nl.

[Rx7man]

mine is my handle at gmail

[Rx7man]

I don't know if anything has changed in the ESP32-wroom side of things, but I got 2 other, very similar wroom boards in the mail yesterday, and both of those, and the one I was using before are getting "Terminal disconnected - Break received" messages on the logs, and then something is hanging the serial port, after some combination of resetting the device and reloading the page or pressing the BOOT button I'll finally be able to get it back.. Of course now that I've typed that it actually worked.. once

Here's the 3 boards, on the left is the original one I was using, then two variants

[rbergen]

Ok, I think I found the problem, and it's an interesting one. In short:

• Panlee is a PlatformIO project (environment) that defines LEDSTRIP to be 1.
• The respective block in globals.h in turn defines WAIT_FOR_WIFI to be 1.
• When WAIT_FOR_WIFI is 1, the functions that tries to connect to WiFi during setup (ConnectToWiFi() in network.cpp) blocks and loops (invoking delay()), explicitly waiting for credentials until WiFi is connected.
• This means that the main setup() function never finishes, and the loop() function is never invoked.
• That means Improv commands will never come in as they are polled for in the main loop() function, and those commands are the only means to set WiFi credentials on a device that doesn't have any configured already.

So, when WAIT_FOR_WIFI is 1, the device gets caught in a catch 22 that it can never get out of.

In my eyes, there are three ways to solve this:

• Remove all projects that lead to WAIT_FOR_WIFI being 1 from the web-flashable projects. That does mean that such projects are dead in the water if no (valid) WiFI credentials are configured through secrets.h.
• Add a separate task (thread) for the Improv loop, and kicking it off before the WiFi connect in setup() is started.
• Stop blocking setup() if WiFi can't connect, potentially effectively removing the WAIT_FOR_WIFI feature.

My preference would be the latter as I think it's the cleanest, but there are very specific reasons @davepl implemented things this way so I'll have to align with him.

All of this also means that devices will become (mostly) unresponsive to Improv while the (first) WiFi connection is attempted (which may well be why the "show log and back out" approach often works), or if the WiFi credentials that were entered are wrong. Fixing this would again require a separate Improv thread or an overhaul of the WiFi connection logic. I have some unfinished thoughts on the latter, which I'll develop further when I have the time.

To be continued.

[robertlipe]

Wow. INteresting detective work. Can I propose another? (It's another of those awkward questions/suggestions that I know you love, but, again, I'm willing to help sling code for this...) It's a breaking change, but it seems less fragile.

4: Kick improvserial to the curb and drop in something like WifiManager.

https://tzapu.com/esp8266-wifi-connection-manager-library-arduino-ide/ seems the best known. https://github.com/zhouhan0126/WIFIMANAGER-ESP32 has better doc https://github.com/cotestatnt/esp-fs-webserver is the kitchen sync, but it integrates with other lis we use. Mostly I'm talking about the first one; I'm just saying that there are many alternatives to Improv. This is what projects like Brian https://www.youtube.com/watch?v=Errh7LEEug0&ab_channel=BrianLough Lough's do, and he's in a pretty similar orbit to ours.

If we don't have a valid SSID/login pair, drop back to AP mode, advertise an SSID of our own, await a connection to it, prompt for WiFI credentials, then save them and reboot. Lots of projects do this. It's an Arduino library that wraps features indigenous to the ESP32, so it's a pretty natural fit for this code. It handles the Arduino state machine of setup/loop.

There is the problem that a human has to know to grab an internet terminal (phone, tablet, real computer, etc.) and connect to an access point named "NightDriverLED needs attention" (or something) to connect to it and feed AP choice and credentials. We could probably blink out some kind of telemetry signaling that we're awaiting credentials.

The prose describing it at https://dronebotworkshop.com/wifimanager/ is a good introduction. The WiFiManager library is probably just lipstick over the native https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-reference/provisioning/wifi_provisioning.html and/or https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/network/esp_smartconfig.html to make it aware of Arduino-ness.

The UX is perhaps not as magical as Improv when it works well, especially freshly out of a flash write, but it looks like it has the advantage of working better (IF you know to connect to it - and that's admittedly the weakness than Improv was trying to solve) if you just turn the device on in a random concert hall or pumpkin patch at a halloween event or any other case where you're not necessarily bouncing straight off an installation but "just" need to reconfigure it from your phone while you're miles away from where you installed and tested the firmware. I think this is something that most IoT users are just generally used to doing.

The Git commit log https://github.com/tzapu/WiFiManager/commits/master doesn't reek of professionalism (Welcome to Arduino...) but it looks like the project is active. The >500 open issues is a cry for help, but there are a LOT of those open issues that I (as a ruthless project owner) would have just tossed out as irrelevant or insufficient info. There are definitely many that should be closed, so the absolute number isn't the whole story.

This may be "throwing out the baby with the bathwater", but I think this would almost drop in and may be less fraile. Maybe.

Selfishly, this isn't a problem I have and there are other things in this project that I'd rather work on, but I recognize the need to address this for the project at large. At a low priority, I'm willing to prototype it and/or pair-program on this if this at least passes the foul order filter and is perceived as a possible step to improvement. I won't hard-sell it.

If you think it's worth chasing, LMK.

On Mon, Oct 23, 2023 at 4:42 AM Rutger van Bergen @.***> wrote:

Ok, I think I found the problem, and it's an interesting one. In short:

• Panlee is a PlatformIO project (environment) that defines LEDSTRIP to be 1.
• The respective block in globals.h in turn defines WAIT_FOR_WIFI to be 1.
• When WAIT_FOR_WIFI is 1, the functions that tries to connect to WiFi during setup (ConnectToWiFi() in network.cpp) blocks and loops (invoking delay()), explicitly waiting for credentials until WiFi is connected.
• This means that the main setup() function never finishes, and the loop() function is never invoked.
• That means Improv commands will never come in as they are polled for in the main loop() function, and those commands are the only means to set WiFi credentials on a device that doesn't have any configured already.

So, when WAIT_FOR_WIFI is 1, the device gets caught in a catch 22 that it can never get out of.

In my eyes, there are three ways to solve this:

• Remove all projects that lead to WAIT_FOR_WIFI being 1 from the web-flashable projects. That does mean that such projects are dead in the water if no (valid) WiFI credentials are configured through secrets.h.
• Add a separate task (thread) for the Improv loop, and kicking it off before the WiFi connect in setup() is started.
• Stop blocking setup() if WiFi can't connect, potentially effectively removing the WAIT_FOR_WIFI feature.

My preference would be the latter as I think it's the cleanest, but there are very specific reasons @davepl https://github.com/davepl implemented things this way so I'll have to align with him.

All of this also means that devices will become (mostly) unresponsive to Improv while the (first) WiFi connection is attempted (which may well be why the "show log and back out" approach often works), or if the WiFi credentials that were entered are wrong. Fixing this would again require a separate Improv thread or an overhaul of the WiFi connection logic. I have some unfinished thoughts on the latter, which I'll develop further when I have the time.

To be continued.

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1774810810, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD366JOEZ6JWWFTID2A3YAY3ZHAVCNFSM6AAAAAA4V2725SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZUHAYTAOBRGA . You are receiving this because you commented.Message ID: @.***>

[rbergen]

@robertlipe By now I have refactored the WiFi connection logic to the point it doesn't block setup() and hence allows Improv to work (the code is in my fork's branch called "polish-webprojects", if you want to have a look). However, Improv now insists on being a pain in the butt by not processing response packets sent by the device, when it should. In other words: the bytes that should be received from the browser are received, and the bytes that should be sent by the device are sent, but the browser currently refuses to see the latter when it should. (They are confirmed to be transmitted though, which makes the whole situation even more "interesting").

...and that may seem like an implicit endorsement of your idea to ditch Improv, but the choice for that over the AP scenario was indeed very deliberate. When it works, the "connect device via cable, load known web page, flash" scenario really is a great user experience. And as I have to assume the issues I am now experiencing are because of me missing something, I don't think we should abandon that flow - just yet.

[robertlipe]

It's not impossible to imagine that before loop() gets called that there's some interrupt that's not enabled or some thread/timer event that's responsible for moving data between layers (hardware to ring buffers to canonical queues to ...) or whatever that's just not running.

It's not helpful that I don't know what it IS in this case (sorry) but that I can imagine it only limping along in some zombie state - neither dead nor alive - until it's fully ready to run.

In the UNIX kernels, we had completely different serial drivers: one that was super-synchronous, low performance, byte-oriented that handled boot, panic messages, early log messages, etc. vs the high-performance ones that let you attach 1,792+ (no kidding) terminals and printers and whatever to the "real" system. "Interesting", indeed, but I've seen lots of OS/runtimes that share that basic trait.

Can you maybe get a logic analyzer on the line to see if it's really really transmitted? It might be helpful to know that they are leaving the UART, but at 2400bps or something that the receiver just doesn't even recognize. Right now I'm trying to debug some ESP32 board builds and just being able to see on a scope that the chip is hocking up the "rst:0x1 (POWERON_RESET), boot:"square waves on the tx is helpful compared to chasing it completely through the stupid UART circuit. I know the chip is booting - it's just not talking to ME.

I think we're agreeing that when Improv works, the combination of coming right out of a firmware install into Improv is pretty awesome. The "go into access point mode" solutions have much more of a speed bump here. The user case that might be better - which we've seen raised a few times in the group, but not by a huge number - is the case of configuring it in a nice, controlled, environment that's slathered in rich, foamy WiFi and then moving it with different (or no) APs and no WiFi at all. Being able to connect to the NDL device from your phone while in the nightclub, warehouse, pumpkin field, or whatever and being able to change the settings has a lot going for it. We just don't have WLED's level of runtime configurability, for example, but being able to change the device's credentials so you can spark up a web server to change the effects list has something going for it. ("Why can't we just stay in Access Point mode and run a web server there?" Good question that we might need a better answer for, but that's a couple of chess moves away.)

It sounds like you have a tourniquet on the bleeding. I know my discussion of amputation is harsh. You know it's an option, so my work here is done...

... unless you want to (╯°□°)╯︵ ┻━┻, set fire to ImprovSerial, and try something else. Then I'll step in and at least try to help.

I totally get that torching ImprovSerial shouldn't be taken lightly. I'm just saying that there are options.

From your description, https://github.com/PlummersSoftwareLLC/NightDriverStrip/compare/main...rbergen:NightDriverStrip:polish-webprojects#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R348 doesn't seem painful. You barely notice that it's trying to outsmart the Arduino global state from that code - that's the danger of making things look easy. :-)

String WiFi_ssid = command.ssid.c_str();

Are you getting command.ssid as Strings, asking for the c_str() const char* string_view, and then promoting that back to a String class?

RJL

On Tue, Oct 24, 2023 at 5:38 AM Rutger van Bergen @.***> wrote:

@robertlipe https://github.com/robertlipe By now I have refactored the WiFi connection logic to the point it doesn't block setup() and hence allows Improv to work (the code is in my fork's branch called "polish-webprojects", if you want to have a look). However, Improv now insists on being a pain in the butt by not processing response packets sent by the device, when it should. In other words: the bytes that should be received from the browser are received, and the bytes that should be sent by the device are sent, but the browser currently refuses to see the latter when it should. (They are confirmed to be transmitted though, which makes the whole situation even more "interesting").

...and that may seem like an implicit endorsement of your idea to ditch Improv, but the choice for that over the AP scenario was indeed very deliberate. When it works, the "connect device via cable, load known web page, flash" scenario really is a great user experience. And as I have to assume the issues I am now experiencing are because of me missing something, I don't think we should abandon that flow - just yet.

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1776957782, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD33AZHY5A4BRU72QC5TYA6LB5AVCNFSM6AAAAAA4V2725SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZWHE2TONZYGI . You are receiving this because you were mentioned.Message ID: @.***>

[rbergen]

The problem with Improv is not that the response packets don't get through. I know for sure that they do, because when I open the Logs & Console window in the web installer after the logging on SPIFFS confirms that the response packets have been sent, they somehow do show up in the logging shown. Why the actual Improv logic in the web browser doesn't pick them up before I push that button I don't know.
I can't help thinking I must be overlooking something that's glaringly obvious, which is quite frustrating.

Are you getting command.ssid as Strings, asking for the c_str() const char* string_view, and then promoting that back to a String class?

Yes, and I thought you might take me up on it. 🙂 This is where I chose to stick to the "we store strings in Strings" de facto project approach, except where I had a specific reason not to (that being the ConnectToWifi() arguments, where it was useful to be able to give nullptr meaning). I note that I may still change my mind on this one, as this is still WIP until it actually does what it should.

[robertlipe]

My next couple of days are kind of full, but I'll try to build up an environment and try to repro it and be additional eyes.

At least we're reaching harmony if we can predict what the other will find distasteful. :-) I won't shout that down, but if you don't have a reason to round-trip them through a StringView or a Span or whatever we're calling it now and just keep it completely in the String domain, drop the c_str() and make it easier on the constructors and explicit constructors - making a String from a String should be easier than making a String from a zchar* that you made from a Strong. But, yeah, we'll do that code review in time.

On Tue, Oct 24, 2023 at 7:51 AM Rutger van Bergen @.***> wrote:

The problem with Improv is not that the response packets don't get through. I know for sure that they do, because when I open the Logs & Console window in the web installer after the logging on SPIFFS confirms that the response packets have been sent, they somehow do show up in the logging shown. Why the actual Improv logic in the web browser doesn't pick them up before I push that button I don't know. I can't help thinking I must be overlooking something that's glaringly obvious, which is quite frustrating.

Are you getting command.ssid as Strings, asking for the c_str() const char* string_view, and then promoting that back to a String class?

Yes, and I thought you might take me up on it. 🙂 This is where I chose to stick to the "we store strings in Strings" de facto project approach, except where I had a specific reason not to (that being the ConnectToWifi() arguments, where it was useful to be able to give nullptr meaning). I note that I may still change my mind on this one, as this is still WIP until it actually does what it should.

— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/425#issuecomment-1777145023, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD3Y4R7FUMWGVECADY4DYA62TNAVCNFSM6AAAAAA4V2725SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZXGE2DKMBSGM . You are receiving this because you were mentioned.Message ID: @.***>

[rbergen]

Well, talking about things I don't like: I just decided to reboot my laptop, after which Improv seems to work like a charm - <insert Windows reboot joke here>.

I've pushed the latest state of polish-webprojects to my staging branch, from where it will be pushed to my fork's web installer in an hour and a half or so. I'll test from scratch using that, and hope to confirm I just ran into a glitch in the system yesterday - which may well have been caused by the Improv implementation on the ESP32 misbehaving until I fixed it.

If that confirms things actually work, I'll reconsider the String ➡️ const char * ➡️ String deal, and see if I can find a less cringeworthy solution for that one. (Conceptually std::optional comes to mind, but storing references in that is not possible and solving that with std::reference_wrapper seems overly complex. Maybe I'll just use pointers to Strings. Anyway, as one can tell the jury is still out.)

[rbergen]

Alright. I've done a reflash-with-erase of Panlee on the ESP32 S3 using the staging web installer, and I've been able to set up WiFi on it after the flash. It did require a reboot of the device and a browser restart, but that's not out of the ordinary.

I'll confirm Improv also works on the M5StickC Plus and Mesmerizer using this particular image. If that works I'll do any code clean-up I should as a good citizen and open a PR to merge this afterwards.

[rbergen]

Improv confirmed working on the M5StickC Plus and Mesmerizer as well.