EspPixelstick UI goes unresponsive, You can connect once and that is it

[patdelaney]

--------- Instructions -------- Please provide answers directly below each section. --------- Instructions ---------

ESPixelStick Firmware Version The last view builds. The one I just tested was the ci build from 7/24/2023

Hardware Version ESPPixelstick V3 or Artstick with SD card

Binary release or compiled yourself? CI build 7/24/2023

Operating System (and version)

Web Browser (and version) Chrome Version 114.0.5735.248 (Official Build) (64-bit)

Access Point

Describe the bug Ever since I got back from Christmas Expo 7/172023 I have been having issues navigating the pixelstick UI. You may be able to connect once, for a bit, then pixelstick spontaneously reboots. When it comes back the webui is unresponsive, but you can ping the stick. The sticks I was using were fine at expo. I suspect a chrome update. When I tested with a new laptop and tried to connect via chrome, it downloaded a file. I will attach to ticket.

On another one when I clicked on diagnostics, it downloaded a file and the stick is unresponsive

it never appears to finish the download

Unconfirmed 158206.zip

[MartinMueller2003]

Is this using the original ESP8266 mini Processor board?

[MartinMueller2003]

I have been testing the ESPS V3 ESP8266 build for an hour now. Nothing odd going on. Diags page shows the expected output (no file downloaded). Config page works fine. Status page works fine. The one difference here is that I uploaded the FS and image via VS code. I will try the flashtool next.

FYI: The only time the ESP will load a file to your computer is when you ask to download the config.

[MartinMueller2003]

Been running on the Diags page showing a random effect output on 100 pixels for the past few hours. Have switch pages on occasion and everything is working fine.

[MartinMueller2003]

So I just kept clicking around and ended up in a situation where the esps.css file was somehow corrupted. It looks like something happened to the FileSystem.

[bloojoop]

I was able to reproduce what Pat is seeing. I am using build from July 25 version ESPixelStick_Firmware-4.0-ci5659517986. I have a esplixelstick v3 hardware with a D1 mini. I can flash it fine, and the web ui is still responsive.. I can reboot it just fine and still seems responsive. I then configured input 1 as DDP, and saved. It was fine until I rebooted, once I rebooted after configuring the first input, then it would also hang up and not respond. Here are my log files from the various reboots while the flasher was still connected.

ui-hang.log

[MartinMueller2003]

DDP is the key. DDP on and a reboot (which the ESP8266 does a lot) results in no UI. No sure why but this is something that can be fixed

[MartinMueller2003]

The UI fails on this error:

Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'browsing-topics'.

[bloojoop]

Hm, Likely same issue but does not seem to be limited to DDP... I just configured ws2811 output with no inputs... and changed universe size to 510... saved, was fine until reboot.. Now that I rebooted it is hanging the same way... Likely the same underlying issue just wanted to report that it seems to not be limited to choosing DDP as an input.

[MartinMueller2003]

When development mde is in use, everything works. When everything is compressed and munged things stop working

[patdelaney]

any chance you are running into a memory or or buffer limitation when compressing and decompressing?

Pat

On Wed, Jul 26, 2023 at 8:21 PM Martin Mueller @.***> wrote:

When development mde is in use, everything works. When everything is compressed and munged things stop working

— Reply to this email directly, view it on GitHub https://github.com/forkineye/ESPixelStick/issues/643#issuecomment-1652775721, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6FEAZUYNSEEQXNAY3F4PTXSG7CDANCNFSM6AAAAAA2XZYW44 . You are receiving this because you authored the thread.Message ID: @.***>

[MartinMueller2003]

that is something i am looking at

[MartinMueller2003]

Still digging into this. It looks like there is a problem serving the esps.js and esps.css files. There is no V3 code involved in this operation. It is handled entirely by the web server library.

[MartinMueller2003]

any chance you are running into a memory or or buffer limitation when compressing and decompressing? Pat … On Wed, Jul 26, 2023 at 8:21 PM Martin Mueller @.> wrote: When development mde is in use, everything works. When everything is compressed and munged things stop working — Reply to this email directly, view it on GitHub <#643 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6FEAZUYNSEEQXNAY3F4PTXSG7CDANCNFSM6AAAAAA2XZYW44 . You are receiving this because you authored the thread.Message ID: @.>

Nope. Not even getting to the point where we use the web socket. This is just generic web hosting.

[patdelaney]

I just tried the CI build on esp8266 and got the same issue again

[MartinMueller2003]

I have to create a pull request and Shelby has to review it and then the CI builds will get kicked off to get you thos images.

[MartinMueller2003]

New dist.zip is at https://drive.google.com/drive/folders/1bGrkXSMoGjWtX5iWspPFRogM2G6c1VXu?usp=sharing

[MartinMueller2003]

Fixed in 645

[forkineye]

It's merged, thanks Martin. I've been completely slammed with work lately.

[patdelaney]

Sorry it's still broken (the CI Build) I'll test Martins build as well

[patdelaney]

I tried Martins build on a different PC and it works. I'll try clearing the browser cache. But then if I try and connect from another browser, it breaks again

[bloojoop]

I download and flashed the lateset build ( FW Version4.0-ci5696732341Build DateJul 28 2023 - 22:17:29 ) and configured ddp for primary, fpp remote for secondary, and 2811 for output... so far it appears fairly stable for me.

[patdelaney]

try connecting from a different browser at the same time

[patdelaney]

I may be headed into the weeds on this, but is it possible that when flashing a new image, it doesn't completely wipe a formerly bad version? Could something be left behind. Does the flash tool erase/wipe/zero out everything before writing the new image?

[patdelaney]

it's still very unstable IMHO. I am currently testing with an Artstick that has SD card I test with and without the card and also just with the esp8266 stand alone and get unpredictable results.

[MartinMueller2003]

I may be headed into the weeds on this, but is it possible that when flashing a new image, it doesn't completely wipe a formerly bad version? Could something be left behind. Does the flash tool erase/wipe/zero out everything before writing the new image?

The flash tool does a complete erase of all flash memory (at least it tries to do that) and then verifies that all memory locations have been reset.

[MartinMueller2003]

I dont know about the CI build, but the dist image I sent a link to is working just fine. I just flashed an old device and it is working fine. Keep in mind that the ESP8266 is not very stable when the UI is getting a lot of use. It will reboot regularly due to internal memory leaks in the libs we are using. But it always comes back.

[MartinMueller2003]

You may be right. I just spent a few minutes clicking from one menu to another until the device crashed. After reboot it again failed to load esps.js. This is getting annoying.

[MartinMueller2003]

Broke up the esps.js file into its parts. Consistently fails to load on a js file but not a specific file.

[patdelaney]

baby steps but at least I am not crazy

[MartinMueller2003]

I put in a heap monitor. The moment th ui tries to connect, the available heap drops to less than 3000 bytes. that is too small to support an http connection and the transfer of js files halts.

[MartinMueller2003]

More data. Even if you get the number of concurrent connections down to a point where the heap is not exhausted, the Async Web server refuses to send data whe the heap is under 10K. So you get a connection but no data phase. Need to find out if the trip level is adjustable and also find out if we can free up more ram.

[patdelaney]

Since I'm not sure what we could remove to save on "ram", let me toss thses ideas out. What is we remove some of the edge case protocals, Would that save RAM? If it would, How much work would it be to spin protol specific builds? Maybe 1 build for 2811 and GECE, and then a second build for Serial DMX and Renard?

[cybercop23]

FWIW I tested the latest one on 2 identical ESP32s/AE+ Quindor/DrZzzs special ESP32 with the AE+ hat to get 3 more ports and the SD card. One works without any issues, the 2nd one experiences the same issues as Pat. All I did was turn on Outputs 1&2 and set first 5 to WS2811

[MartinMueller2003]

try this version https://drive.google.com/drive/folders/1bGrkXSMoGjWtX5iWspPFRogM2G6c1VXu?usp=drive_link

[patdelaney]

I am at the office. I'll check tonight

On Thu, Aug 3, 2023 at 11:50 AM Martin Mueller @.***> wrote:

try this version

https://drive.google.com/drive/folders/1bGrkXSMoGjWtX5iWspPFRogM2G6c1VXu?usp=drive_link

— Reply to this email directly, view it on GitHub https://github.com/forkineye/ESPixelStick/issues/643#issuecomment-1664316435, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6FEA2IHHNAFFV57RRKIPLXTPJGRANCNFSM6AAAAAA2XZYW44 . You are receiving this because you modified the open/close state.Message ID: @.***>

[bloojoop]

I loaded todays version on a d1 mini. Just using it powered up via the usb cable. network appears to be very unstable. attached is a just a continuous ping showing mass packet loss. I could not do anything the browser interface.

aug3rd ping.txt

aug3rd flash logs.txt

[cybercop23]

Ditto. I get a bare bone html page with a spinning loading page. looks like the css hasn't been applied

On Thu, Aug 3, 2023 at 6:15 PM Ron Armstrong @.***> wrote:

I loaded todays version on a d1 mini. Just using it powered up via the usb cable. network appears to be very unstable. attached is a just a continuous ping showing mass packet loss. I could not do anything the browser interface.

aug3rd ping.txt https://github.com/forkineye/ESPixelStick/files/12255399/aug3rd.ping.txt

— Reply to this email directly, view it on GitHub https://github.com/forkineye/ESPixelStick/issues/643#issuecomment-1664709328, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFUGO2N64Y65CF3E53JNXVDXTQPGNANCNFSM6AAAAAA2XZYW44 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[patdelaney]

I am using the build martin Posted and It's connecting and occasionally I get reboots. But it comes back each time. This test is an artstick with no sd card installed. I saw this after configuring DDP in and ws2811 out:

18:29:15: [WebMgr] WS client connect - 2 18:29:16: [FileMgr] ERROR: Failed to allocate memory for the GetListOfSdFiles web request response. 18:29:16: [FileMgr] ERROR: Failed to allocate memory for the GetListOfSdFiles web request response.

[cybercop23]

DugUno with ESP32 + AE+ hat with SD card. Before this one it was loading... [image: image.png]

On Thu, Aug 3, 2023 at 12:52 PM patdelaney @.***> wrote:

I am at the office. I'll check tonight

On Thu, Aug 3, 2023 at 11:50 AM Martin Mueller @.***> wrote:

try this version

https://drive.google.com/drive/folders/1bGrkXSMoGjWtX5iWspPFRogM2G6c1VXu?usp=drive_link

— Reply to this email directly, view it on GitHub < https://github.com/forkineye/ESPixelStick/issues/643#issuecomment-1664316435>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AB6FEA2IHHNAFFV57RRKIPLXTPJGRANCNFSM6AAAAAA2XZYW44>

. You are receiving this because you modified the open/close state.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/forkineye/ESPixelStick/issues/643#issuecomment-1664318540, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFUGO2KNQVGNSCRY3B4VODTXTPJM5ANCNFSM6AAAAAA2XZYW44 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[MartinMueller2003]

I am using the build martin Posted and It's connecting and occasionally I get reboots. But it comes back each time. This test is an artstick with no sd card installed. I saw this after configuring DDP in and ws2811 out:

18:29:15: [WebMgr] WS client connect - 2 18:29:16: [FileMgr] ERROR: Failed to allocate memory for the GetListOfSdFiles web request response. 18:29:16: [FileMgr] ERROR: Failed to allocate memory for the GetListOfSdFiles web request response.

Thanks Pat. Yes it is still pretty low on memory. Still looking to see where I can get more memory. May have to rework the Web Socket interface to get rid of the buffer. That means changing a lot on the UI to make it more REST like.

[patdelaney]

the Pixel stick I put your latest image on lst night, I set it for WLED rainbow and test let it run. Today I tried to openm the web UI and it hanging. But the Rainbow effect is still running. I tried ruinning a postman WS call http://192.168.1.170/fppjson.php?command=getHostNameInfo and it returned { "HostName": "shelby-sd6", "HostDescription": "ESPixelStick" } than a few more retrys of the same call and it hangs

[MartinMueller2003]

Still working on this. I had to find a bunch of memory. This caused me to look at the WebSocket server. It turns out the WS Server used 10KB of memory (buffers etc). I have now eliminated all WebSocket support and replaced all of the functionality with HTTP operations. All work on the ESP8266. The output_config download for ESP32 does not seem to complete. The last chunk seems to get lost.

[bloojoop]

I appreciate all the work and effort on this... using d1 minis on espixelstick hardware (and espartstick) the last stable version that I've been able to use reliably is FW Version

4.0-ci2259677003 Build Date

May 2 2022 - 18:28:27 Architecture

ESP8266

I have 3 of these deployed full time and very little issues... I tried a version from 11/2022 and then more recent versions here in July and have had various stability issues.. Hopefully this move to DTTP operations will help quite a bit :) Thanks again for all your help.

On Fri, Aug 11, 2023 at 4:31 PM Martin Mueller @.***> wrote:

Still working on this. I had to find a bunch of memory. This caused me to look at the WebSocket server. It turns out the WS Server used 10KB of memory (buffers etc). I have now eliminated all WebSocket support and replaced all of the functionality with DTTP operations. All work on the ESP8266. The output_config download for ESP32 does not seem to complete. The last chunk seems to get lost.

— Reply to this email directly, view it on GitHub https://github.com/forkineye/ESPixelStick/issues/643#issuecomment-1675422199, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVCLUGBD77RDPTBA7ZESI4TXU2QCRANCNFSM6AAAAAA2XZYW44 . You are receiving this because you commented.Message ID: @.***>

[patdelaney]

I downloaded last nights CI build and I am testing on Artstick with SD card. (currently no card in slot) After I configured it the Web UI was very responsive and no reboots. At boot up the Heap was 24456. I then configured it for E1.21 input and WS2811 output 100 pixels 20% brightness. Now Heap is at 23568. Saved the config and then noticed this in the serial output window. and it write it out every second:

07:56:33: [WebMgr] ERROR: Status Doc is too small 07:56:34: [WebMgr] ERROR: Status Doc is too small 07:56:34: [WebMgr] ERROR: Status Doc is too small

Note that there is no sd card in the slot I'm guessing that is the issue

[MartinMueller2003]

Nope. That means one of my constants is too low for the status response on the home page. That is an easy fix. and will be in an upcoming push to my fork.

WARNING: Only ESP8266 is working. The ESP32 TCP stack has a bug that prevents the output config from getting saved.

[MartinMueller2003]

New dist with ESP32 support and file upload / management

https://drive.google.com/drive/folders/1bGrkXSMoGjWtX5iWspPFRogM2G6c1VXu?usp=sharing

[patdelaney]

Not reading contents of the SD card

But when you reboot the serial output can see the files 18:56:45: [FileMgr] SD Card Size: 14860MB 18:56:45: [FileMgr] > System Volume Information 18:56:45: [FileMgr] IndexerVolumeGuid - 76 18:56:45: [FileMgr] WPSettings.dat - 12 18:56:45: [FileMgr] FPP-Matrix_Demo.fseq - 108072 18:56:45: [FileMgr] pole_butterfly.fseq - 306072 18:56:45: [FileMgr] pole_random.fseq - 306072 18:56:45: [FileMgr] MyPlayList.pl - 998

[MartinMueller2003]

And mine looks fine on both ESP8266 and ESP32

[MartinMueller2003]

press F12 key and look at the console tab. It may be throwing errors that would help figure out what is going on.

[patdelaney]