Unable to save rules in the browser window

[issleiber]

Since the last release, I'm unable to save rules in a browser windows. Sometimes it works properly, and the rules are saved correctly.

When I click "save" ... this is happened Error:

[TD-er]

Which build? What platform? ESP8266/ESP32? Does the ESP reboot or reconnect when this happens? How much free memory do you have on the ESP?

[issleiber]

Build: ESP_Easy_mega_20221105_neopixel_ESP32_4M316k Nov 5 2022

ESP32

Note: No reboot or anything else, when this happend. I only can't save the rules. espeasy works fine, except this "save" issue.

This is happend, when I click on "save" in the rules (see developerTools in my browser). There is also no "submitted" notice, after clicking. And this is independent from the browser (it happend on safari, Chrome ...).

[TD-er]

And after a new reboot, can you then save it? (maybe also a power cycle?)

Thing is, you're now the 3rd person to mention these save issues on ESP32. There must be some kind of pattern here, but I can't put my finger on it...

Can you also try this latest test build: https://github.com/letscontrolit/ESPEasy/actions/runs/3591951789 ?

[chromoxdor]

This is happend, when I click on "save" in the rules (see developerTools in my browser). There is also no "submitted" notice, after clicking. And this is independent from the browser (it happend on safari, Chrome ...).

You will not see a submit message because according to your last screenshot it states "nothing to save". This appears when you didn´t change anything in the rules and then hit submit. Or is this part of the issue?

I just tired to reproduce your error messages with the same build but so far no success...

[chromoxdor]

there is a lot of other stuff on the client side that could cause things like that i would think... can you rule out bad firewall, antivirus, proxy settings?

[issleiber]

• No firewall issue (i'm networker), no AV or proxy settings.
• This problem exists on different Laptops/Computer and different Browser. A
• After a reboot, same behavior.
• And yes the "nothing to save" came after 20-50 times clicking on the save button, and sometimes after 20-50 times clicking, the rules are saves and then you can see the message "nothing to save".
• And yes, I've changed the rules, any time a try to save.

--> I've tried the "ESP_Easy_mega20221201" ... same problem.

--> I roll back to ESP_Easy_mega_20220616_neopixel_ESP32_4M316k, and this works fine.

[chromoxdor]

Did you try to reset the device to defaults before testing?

[issleiber]

hmmmm .... okay, let me try this.

[issleiber]

• Okay, I had reset to "factory setting".
• update to ESP_Easy_mega_20221201
• loaded my config.dat & rules1 (&2)
• and ... same issue. Cannot save the rules. I can load rules via filebrowser, but not edit them.

Result: any version below mega-20221105 works fine

[chromoxdor]

I meant to try it without loading the config.dat... Just enter you WiFi credentials and nothing more.

[tonhuisman]

Has the browser cache already been force-cleared somewhere during testing? It might be a crappy downloaded .js file from CDN in your cache that is causing this. The Uncaught... messages could point in that direction

[TD-er]

Can you upload the files to the SPIFFS file system? If it is indeed related to the CDN not working, then serving it from the file system would fix it.

See: https://espeasy.readthedocs.io/en/latest/Reference/ExternalHostedStaticFiles.html?highlight=cdn#external-hosted-static-files

[issleiber]

Uploading (especially) the following files:

• espeasy_default.css (also rename it to esp.css)
• rules_save.js

... doesn't fix the problem.

Another weird behaviour I got:

• After a reboot the "saving of the rules" works sometimes 1-2 times properly , after that, I get an error again.

[issleiber]

Now I used a second (exactly same) esp32 board. With this board, I can save rules!

The release and config and also the rules are exactly the same.

1.) ESP32 Board which works properly (saving rules):

Build:⋄
20221201  - Mega32
System Libraries:⋄
ESP32 SDK v4.4.3
Git Build:⋄
mega_2b6dbc6
Plugin Count:⋄
52 [Normal][NeoPixel]
Build Time:⋄
Dec  1 2022 11:15:31
Binary Filename:⋄
ESP_Easy_mega_20221201_neopixel_ESP32_4M316k
Build Platform:⋄
Linux-5.15.0-1023-azure-x86_64-with-glibc2.31
Git HEAD:⋄
mega_2b6dbc6

2.) ESP32 Board which doesn't work properly (saving rules):

Build:⋄
20221201  - Mega32
System Libraries:⋄
ESP32 SDK v4.4.3
Git Build:⋄
mega_2b6dbc6
Plugin Count:⋄
52 [Normal][NeoPixel]
Build Time:⋄
Dec  1 2022 11:15:31
Binary Filename:⋄
ESP_Easy_mega_20221201_neopixel_ESP32_4M316k
Build Platform:⋄
Linux-5.15.0-1023-azure-x86_64-with-glibc2.31
Git HEAD:⋄
mega_2b6dbc6

I increased the webLogLevel to 'max'. When I try to save the rules, I can see an error in the WebLog:

>> Failed to fetch <<

AddOn: Now it doesn't work an the second ESP32 too. I don't know why, for several time I was able to save rules, and now the error is back :-(

I give up ... and for me, the older version "Easy_mega_20220616" works well and now I use this one as "my personell" stable version.

[chromoxdor]

And did you try it with a clean install without your settings?

[TD-er]

Can you also post info about the used flash chip? If it by any chance vendor id 0x20 ? (XMC)

[issleiber]

Flash Chip ID:	Vendor: 0xC8 Device: 0x4016
Flash Chip Real Size:	4096 kB
Flash IDE Size:	4096 kB
Flash Chip Speed:	80 MHz
Flash Writes:	0 daily / 0 boot
Sketch Size:	1328 kB (1856 kB free)
Max. OTA Sketch Size:	1856 kB (1900544 bytes)
SPIFFS Size:	283 kB (124 kB free)
Page size:	256
Block size:	8192
Number of blocks:	35

 

ESP Chip ID:	1889788 (0x1CD5FC)
ESP Chip Frequency:	240 MHz
ESP Chip Model:	ESP32-D0WDQ6
ESP Chip Revision:	1
ESP Chip Cores:	2
ESP Board Name:	Espressif Generic 1 4M Flash ESPEasy 1810k Code/OTA 316k FS

[TD-er]

The only "suspicious" thing I can see is that it has ESP chip rev. 1. There are some "silicon bugs" in revisions prior to rev. 3. which are being dealt with in the recent SDK. For example when accessing flash, the SDK now adds some NOP operations (no-op as in "do nothing") to make sure no data is being lost on some very timing critical stuff where data may not yet be guaranteed to be ready.

But to be honest, I'm not sure whether we need to be looking in that direction.

When you're unable to save the rules, are you able to save some settings? For example change the name of a task, or the name of tast values? You need to do it on at least 2 tasks, to make sure the cached values are being read from the file system again.

If that's possible, then I guess we need to have a look at either serving the JavaScript to save the rules, or the functions handling the file upload call in the background. For example, what if the file was updated, but the window reload did load the old version?

[issleiber]

Thank's.

Now I used one more time the mega-20221105 build. And I see the same issue. Only rules cannot be saved properly. But I can change all other things/settings and it's saved as expected (Unit name, timeouts, devices etc. etc.)

[TD-er]

Just to be sure, you did try to save the JavaScript file used to save the rules to the file system on the ESP?

[TD-er]

OK, I've been looking at the code quite a bit more and so far I don't see where it might have changed in such a way to cause these issues. However, based on your screenshots, I wonder if we may have to deal with a timeout? How do you access your ESP? (typical ping/response times) Is there something else also connecting to the ESP on a regular basis? Is the ESP sometimes quite busy? Does it also fail if you have some ping from just any host in your network running to the ESP?

[issleiber]

Network and ping: I'm a networker, and no, there is no issue. The average PING delay is between 5-8ms. The same ESP32 running with mega-202206xx (and older one) works well (with the same via DHCP leased IP address). The related AccessPoint is very close to the ESP32 (5m) and I have a RSSI of about 54-60 dbm. I also use MQTT with this ESP32. This also works well.

CPU load: average 30-40%, no visible peaks.

[TD-er]

The reason I asked to let a ping run is not just for diagnosing the network latency, but on an ESP it really makes a difference in how the ESP behaves if it has to answer a ping every second. The ESP may lower its power consumption by increasing the DTIM. When this happens, it may miss some ARP requests and as a result it may no longer be accessible as the switches/AP no longer knows how to route packets to the ESP's MAC address. A larger DTIM may also cause larger delays in answering requests, which may play a role here as the JavaScript to save the rules first tries to fetch the current rules file and then compares it with the text in the rules window. If they are the same, no upload will be performed. But if the fetch of the rules fails, also no upload. If the Javascript cannot be loaded by the browser... well you get the idea ;)

About the "how you connect to it"... If for example you were accessing it via some 4G hotspot over a VPN. Then this would at least introduce come latency.

Now back to the question on the load. If the ESP does need to handle a lot of queries, it may also have run out of open slots as it can only handle a fixed number of concurrent requests (I think it is 5???) So what if one of such requests is not terminated? E.g. due to timeout, or just being used.

I think a 'fix' could be to extend the JavaScript to also perform an upload when it failed to fetch the file. On the other hand, it may also just corrupt the rules if the ESP is too busy handling other stuff. At least it should show something on the screen to indicate a failure.

[issleiber]

A ping to the ESP32 ...

... ping 192.168.100.60
PING 192.168.100.60 (192.168.100.60): 56 data bytes
64 bytes from 192.168.100.60: icmp_seq=0 ttl=255 time=4.420 ms
64 bytes from 192.168.100.60: icmp_seq=1 ttl=255 time=4.515 ms
64 bytes from 192.168.100.60: icmp_seq=2 ttl=255 time=4.729 ms
64 bytes from 192.168.100.60: icmp_seq=3 ttl=255 time=4.568 ms
64 bytes from 192.168.100.60: icmp_seq=4 ttl=255 time=3.978 ms
64 bytes from 192.168.100.60: icmp_seq=5 ttl=255 time=3.873 ms
64 bytes from 192.168.100.60: icmp_seq=6 ttl=255 time=3.902 ms
>> from here, I pressed the save button in rules several times <<
64 bytes from 192.168.100.60: icmp_seq=7 ttl=255 time=6.776 ms
64 bytes from 192.168.100.60: icmp_seq=8 ttl=255 time=3.750 ms
64 bytes from 192.168.100.60: icmp_seq=9 ttl=255 time=4.427 ms
64 bytes from 192.168.100.60: icmp_seq=10 ttl=255 time=4.221 ms
>> end pressing save button <<
64 bytes from 192.168.100.60: icmp_seq=11 ttl=255 time=4.549 ms
64 bytes from 192.168.100.60: icmp_seq=12 ttl=255 time=4.321 ms
64 bytes from 192.168.100.60: icmp_seq=13 ttl=255 time=4.619 ms
64 bytes from 192.168.100.60: icmp_seq=14 ttl=255 time=5.051 ms
64 bytes from 192.168.100.60: icmp_seq=15 ttl=255 time=4.961 ms
64 bytes from 192.168.100.60: icmp_seq=16 ttl=255 time=4.302 ms
64 bytes from 192.168.100.60: icmp_seq=17 ttl=255 time=4.592 ms
64 bytes from 192.168.100.60: icmp_seq=18 ttl=255 time=7.422 ms
64 bytes from 192.168.100.60: icmp_seq=19 ttl=255 time=4.677 ms
64 bytes from 192.168.100.60: icmp_seq=20 ttl=255 time=3.960 ms
64 bytes from 192.168.100.60: icmp_seq=21 ttl=255 time=4.458 ms
64 bytes from 192.168.100.60: icmp_seq=22 ttl=255 time=6.807 ms
64 bytes from 192.168.100.60: icmp_seq=23 ttl=255 time=4.409 ms
64 bytes from 192.168.100.60: icmp_seq=24 ttl=255 time=4.104 ms
64 bytes from 192.168.100.60: icmp_seq=25 ttl=255 time=4.039 ms

One more clue (not sure, if that is important): When I press the "save" button in rules several times (maybe 3 times a second) I can see a "nothing to save" after clicking x-times, but no "submitted" note, that I expected. The rules is saved properly after clicking x-times (>> 10 times).

[TD-er]

What the JavaScript tries to do is something like this:

• Fetch the rules as stored on the ESPEasy file system
• Fetch all from the text box
• Compare both to see if they have changed.
• When changed, create a POST request to "upload" a new version of that file

However the console errors you show, seem to indicate sometimes the first step (fetching/downloading) the rules file fails and in other attempts the POST fails.

Could it be the browser either doesn't close the "download" connection, or maybe way too fast here?

Which browser are you using?

Can you also look at the waterfall diagram drawn in the inspect window, with the "network" tab selected and then try to save again? (with a change of course)

[issleiber]

What the JavaScript tries to do is something like this:

• Fetch the rules as stored on the ESPEasy file system
• Fetch all from the text box
• Compare both to see if they have changed.
• When changed, create a POST request to "upload" a new version of that file

However the console errors you show, seem to indicate sometimes the first step (fetching/downloading) the rules file fails and in other attempts the POST fails.

Could it be the browser either doesn't close the "download" connection, or maybe way too fast here?

Which browser are you using?

Can you also look at the waterfall diagram drawn in the inspect window, with the "network" tab selected and then try to save again? (with a change of course)

As mentioned ... I use different browser (Safari MacOS only) , Chrome (Windows) , IPad ... this issue is on all browsers I use. I also downloaded opera to try out wether or not there is an effect on this behaviour, but is not.

second try with clicking "save" in rules (the 5'th click succeeded an I got an "submitted" note ).