Board not reachable

[Secarius]

Hardware

ESP32 WT32-ETH01

Firmware version

v2.3.3

Application version

v2.3.3

What happened? What did you expect to happen?

The board with the ETS works great, I have 2 of them in operation at different locations. This morning both were no longer accessible, neither via Home Assistant nor via the web interface. The network interface still showed activity, but it was only accessible again after a power cycle. I've had this a few times, does anyone recognise this phenomenon?

I operate the WT32-ETH01 at 5V with a 3A power supply unit.

Otherwise the whole thing works really well.

How to reproduce it (setp by step)

Don't know how to reproduce

Logs

No response

[rstrouse]

I have 2 WT32-ETH01. One of them (the other has never done it) will occasionally be offline. It is always accessible via the serial port and watching the logs it is operating fine. It just does not have any network access. It seems to come back by unplugging the network cable and plugging it back in.

[Jezza34000]

I have WT32-ETH01 too, and never noticed this problem @Secarius How about IP ping when board is unavailable ?

[011V32]

Since the last (or last 2 to a maximum of 3) updates, I have also had the board no longer be accessible after a few days. After a restart it works again. I didn't have this problem with older versions. I use a LilyGO T-Internet POE.

[rstrouse]

I have set up the WT32-ETH01 that has dropped off the network in the past with a serial monitor open to catch it if it fails. It is running the pre-release as I put some additional logging when it checks the connection. I'll let you know the results but so far the heap remains steady. I also have set up my dev HA to move some of the entities every few minutes.

[rstrouse]

Ok so I have caught the failure on the PHY on that board after 110 hours. The failure mode never identifies that it dropped the link. Yes it is dropping the link but the reason for this is unclear. It could be as simple as the lease expiring on the IP or some other network event for which it doesn't automatically recover. Again it is very odd that the other WT32-ETH01 did not drop. Both of these units had the exact same 32 shade backup file and occasional movement.

Disconnecting the cable and plugging it back in makes it start again. But the reason it does that is that it releases the adapter and thus notifies the ESP32 that the connection is stopped. In this condition without any reload of the software it began working again as if nothing ever happened. So the perfect scenario is to detect the condition and reset the connection. I dug into the bowels of the ETH module and managed to find some bits that indicate whether the interface has a proper link that I am going to try. If I were to guess, there is a link condition that the ETH library doesn't always report and thus does not reset its buffers.

I have published these changes into the v2.4.0 pre-release and reset the Ethernet ESP32s to run with that code. If you would like to play along please install the v2.4.0 pre-release and lets see if I managed to capture the failure and recover.

[011V32]

I have updated the firmware. But the problem usually only occurs for me after 10 days at the earliest, so I can't give quick feedback... The funny thing is that I never had a problem with previous versions.

[rstrouse]

The funny thing is that I never had a problem with previous versions.

I am afraid that this is likely coincidental. The failure mode has to do with the PHY not reporting that it needs to re-establish the link. There are lots of conditions that can require it and time isn't one of them but things like lease timing on the ip can make it seem like it is time constrained. This happens with wifi as well but that seems to report reliably and registration is made within a few ms of the occurrence.

Interestingly, on older versions of the esp32 libraries there was an event ARDUINO_EVENT_ETH_LOST_IP that was broadcast on this failure mode but that was removed. Many of these libraries were reworked with 2.0.14+ of the ESP32 libraries so this may actually have been handled differently in earlier versions. These Arduino libraries were updated to 2.0.14 on the v2.2.0 transition.

[Secarius]

So as i understand, a Quick fix also would be to use a static ip?

[rstrouse]

So as i understand, a Quick fix also would be to use a static ip?

No that is not how a static IP works. A static IP is still negotiated with the gateway so the link has to be established no matter what or the gateway will poop on the packets. As a result of failed link negotiation though, the IP will be lost and I suspect the re-negotiation is where the library is not reporting or re-establishing the link. It is sitting there as if the network is fine and never reporting that it lost the link to the software that is consuming its service.

In the past I think that it likely ended up bubbling out to a dropped IP where the esp32 would reset it. The pre-release inspects the link to make sure it is still up.

[Secarius]

Today it stuck again, with the beta firmware. After Powercycle it came back working fine, but replugging the ethernet cable only did not help.

[rstrouse]

Install the release v2.4.0. I have added some additional checks.

[Secarius]

I already have it on:

[rstrouse]

Unless you just updated an hour ago you are running the pre-release of v2.4.0.

[Secarius]

ah ok, i will update :)

[Secarius]

Hi, even after the update, it just stopped working again.

[rstrouse]

Hmmm the one that was disconnecting hasn't since I updated it.

[rstrouse]

On a hunch I removed the event processor from the ethernet network connection. In looking at the underlying code for the library there are a couple of spots that made me question whether it was thread safe. The ethernet module operates on a separate core and while the event model was convenient, I think it comes with a wart or two. Please install the pre-release v2.4.1 and lets see if changing the processing makes a difference.

[011V32]

Mine became also unavailable 6 hours ago. I will try it 👍

[Craig-97]

Also been having the same issue for a few versions now. Will give the v2.4.1 a try and report back

[Secarius]

Hi, it just came unreachable again. :)

[011V32]

Mine also :(

[Craig-97]

Same for me, happened yesterday

[rstrouse]

So weird. I have been monitoring heap fragmentation and the results on that front are conclusive. There is no memory leakage over time with any of the modules. That doesn't mean that there is some network condition that makes the ESP32 libraries for v2.0.14 break down when it hits a specific condition. I have been running both of my WT32-ETH01 modules since the 3rd without incident. I will continue to monitor while connected to the serial logging to see if there is something that sends it south.

The key difference between earlier ESPSomfy RTS versions and the current version is an update from v2.0.12 to v2.0.14 of the ESP32 libraries.

[011V32]

I have an LiLiGo PoE board. What have the others?

[Secarius]

WT32-ETH01 https://www.aliexpress.com/item/1005005967733312.html

[011V32]

Today the module was not available again. But I have now noticed that at least 1 day before Home Assistant no longer has access, the website was no longer available. Could this information be relevant? Should I try reinstalling the module via USB?

[Secarius]

My one Was also Not available today in the morning but came back in the afternoon by itself

[Craig-97]

Today the module was not available again. But I have now noticed that at least 1 day before Home Assistant no longer has access, the website was no longer available. Could this information be relevant? Should I try reinstalling the module via USB?

I get the same thing, the site is unavailable and home assistant still responds but is flakey until it fully stops working.

[Jumperm]

Mine is not available since 2 Hours in HA and also via DHCP IP. How can i help you to debug?

WT32-ETH01

[rstrouse]

Can you get to the serial logs? I need to know what is displaying on this log. I think there may be a leak in the lwip for the ethernet libraries.

[Secarius]

@rstrouse is it possible to log it with a second esp mcu ? If so i would connect it and start logging

[rstrouse]

Unfortunately, if there is I don't know how to do it. I have managed to have mine fail again after about 2 weeks while it was connected to the serial monitor.

In that instance the error from the board reports E (470611489) esp.emac: no mem for receive buffer. No commands were running at the time and only one socket was open monitoring traffic. I monitor the heap and under normal usage there is no reduction in this memory so it isn't leaking from there so I can only assume that somewhere in the lwip it is allocating memory and does not always free it.

At the time it had everything turned on including MQTT and SSDP support so I turned those off to see if I can get the failure to happen again.

[Jumperm]

In the Logs of HA I see every 2 days disconnects. I try to Connect an FTDI to the ESPSomfy. The disconnect today has selfhealed after 9hours.

[Secarius]

I think i managed to get a second esp to log the serial of the wt32-eth01. As soon as a crash happens, i will post the log.

[Jumperm]

I think i managed to get a second esp to log the serial of the wt32-eth01. As soon as a crash happens, i will post the log.

What outputs did you get. I log some data with the FTDI at TX/RX and a terminal session over the esphome-flasher "View Logs". But after a Reboot of the ESPSomfy I get something like that:

[19:55:54][19:55:54]Chip Model ESP32-[19:55:54][19:55:54]
[19:55:57][19:55:57][19:55:57]Min Heap: 179416
[19:56:00][19:56:00][HTTPS] GET... code: 200 - -1
[19:56:01][19:56:01]Fru

and then it stops. Trying to connect again show "Serial port closed!" Did you get more output?

[rstrouse]

I have some news on this front. I set up a WT32-ETH01 with a heap dump every few seconds and watch the memory usage to ensure there was no leakage. Guess what,... there of course was no leakage and everything was flawless. However, in going through all this I noticed that the binary size from the CI workflows on the site were smaller than the binary size from compiling with the Arduino IDE despite using the same library versions.

As it turns out there were some additional compiler flags that existed in the IDE that were not being applied to the workflow. These were related to the SPI interface on the board. I have released v2.4.1 that changes the compiler flags and may have been the goofiness we were experiencing all along since the CI workflow were added a few versions ago.

[Edition30]

Hallo welche Version ist eigentlich für den WT32-ETH01 gedacht ? esp32s3?

[Jumperm]

Hallo welche Version ist eigentlich für den WT32-ETH01 gedacht ? esp32s3?

Ich hab die ganz normale ESP32 drauf installiert.

I installed the normal ESP32.

(SomfyController.onboard.esp32.bin.zip

I have some news on this front. I set up a WT32-ETH01 with a heap dump every few seconds and watch the memory usage to ensure there was no leakage. Guess what,... there of course was no leakage and everything was flawless. However, in going through all this I noticed that the binary size from the CI workflows on the site were smaller than the binary size from compiling with the Arduino IDE despite using the same library versions.

As it turns out there were some additional compiler flags that existed in the IDE that were not being applied to the workflow. These were related to the SPI interface on the board. I have released v2.4.1 that changes the compiler flags and may have been the goofiness we were experiencing all along since the CI workflow were added a few versions ago.

That's very good news. Thanks! I'll test the new version!

[Secarius]

Today the same again:

Log-Inhalt
28.3.2024 22:59:59 - CMD:Down ADDR:744163 RCODE:415 REPEAT:0
28.3.2024 22:59:59 - Enabled receive on Pin #35 Timing: 0
28.3.2024 23:36:36 - Setting Last Rolling code 3234
28.3.2024 23:36:37 - Timing Somfy: 541ms
29.3.2024 08:59:59 - Sending Shade Command
29.3.2024 08:59:59 - Received:{"shadeId":1,"command":"up"}
29.3.2024 08:59:59 - CMD:Up ADDR:744161 RCODE:393 REPEAT:0
29.3.2024 08:59:59 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:00:04 - Sending Shade Command
29.3.2024 09:00:04 - Received:{"shadeId":2,"command":"up"}
29.3.2024 09:00:04 - CMD:Up ADDR:744162 RCODE:310 REPEAT:0
29.3.2024 09:00:04 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:00:09 - Sending Shade Command
29.3.2024 09:00:09 - Received:{"shadeId":3,"command":"up"}
29.3.2024 09:00:09 - CMD:Up ADDR:744163 RCODE:416 REPEAT:0
29.3.2024 09:00:09 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:00:14 - Sending Shade Command
29.3.2024 09:00:14 - Received:{"shadeId":4,"command":"up"}
29.3.2024 09:00:14 - CMD:Up ADDR:744164 RCODE:305 REPEAT:0
29.3.2024 09:00:14 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:26:20 - Setting Last Rolling code 2843
29.3.2024 09:26:40 - Sending Shade Command
29.3.2024 09:26:40 - Received:{"shadeId":5,"target":90}
29.3.2024 09:26:40 - Moving to 90.00% from 100.00% using Up
29.3.2024 09:26:40 - CMD:Up ADDR:744165 RCODE:391 REPEAT:0
29.3.2024 09:26:40 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:26:42 - CMD:My ADDR:744165 RCODE:392 REPEAT:0
29.3.2024 09:26:42 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:26:42 - Timing Somfy: 206ms
29.3.2024 09:26:56 - Sending Shade Command
29.3.2024 09:26:56 - Received:{"shadeId":5,"target":80}
29.3.2024 09:26:56 - Moving to 80.00% from 90.00% using Up
29.3.2024 09:26:56 - CMD:Up ADDR:744165 RCODE:393 REPEAT:0
29.3.2024 09:26:56 - Enabled receive on Pin #35 Timing: 0
29.3.2024 09:26:58 - CMD:My ADDR:744165 RCODE:394 REPEAT:0
29.3.2024 09:26:58 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:26:58 - Timing Somfy: 210ms
29.3.2024 09:27:14 - Sending Shade Command
29.3.2024 09:27:14 - Received:{"shadeId":5,"target":90}
29.3.2024 09:27:14 - Moving to 90.00% from 80.00% using Down
29.3.2024 09:27:14 - CMD:Down ADDR:744165 RCODE:395 REPEAT:0
29.3.2024 09:27:14 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:27:16 - CMD:My ADDR:744165 RCODE:396 REPEAT:0
29.3.2024 09:27:16 - Enabled receive on Pin #35 Timing: 1
29.3.2024 09:27:16 - Timing Somfy: 223ms
29.3.2024 09:28:28 - Sending Shade Command
29.3.2024 09:28:28 - Received:{"shadeId":5,"command":"up"}
29.3.2024 09:28:28 - CMD:Up ADDR:744165 RCODE:397 REPEAT:0
29.3.2024 09:28:28 - Enabled receive on Pin #35 Timing: 1
29.3.2024 11:05:05 - Sending Shade Command
29.3.2024 11:05:05 - Received:{"shadeId":5,"target":50}
29.3.2024 11:05:05 - Moving to 50.00% from 0.00% using Down
29.3.2024 11:05:05 - CMD:Down ADDR:744165 RCODE:398 REPEAT:0
29.3.2024 11:05:05 - Enabled receive on Pin #35 Timing: 1
29.3.2024 11:05:13 - CMD:My ADDR:744165 RCODE:399 REPEAT:0
29.3.2024 11:05:13 - Enabled receive on Pin #35 Timing: 1
29.3.2024 11:05:13 - Timing Somfy: 689ms
29.3.2024 12:21:30 - Check github for updates...
29.3.2024 12:21:30 - Max Heap: 36852
29.3.2024 12:21:30 - Free Heap: 78972
29.3.2024 12:21:30 - Min Heap: 42604
29.3.2024 12:21:30 - Internet is Unavailable: -1: 42ms
29.3.2024 12:38:14 - Setting Last Rolling code 5673
29.3.2024 14:40:38 - Sending Shade Command
29.3.2024 14:40:38 - Received:{"shadeId":5,"target":90}
29.3.2024 14:40:38 - Moving to 90.00% from 100.00% using Up
29.3.2024 14:40:38 - CMD:Up ADDR:744165 RCODE:400 REPEAT:0
29.3.2024 14:40:38 - Enabled receive on Pin #35 Timing: 1
29.3.2024 14:40:40 - CMD:My ADDR:744165 RCODE:401 REPEAT:0
29.3.2024 14:40:40 - Enabled receive on Pin #35 Timing: 1
29.3.2024 14:40:40 - Timing Somfy: 214ms
29.3.2024 14:41:29 - Sending Shade Command
29.3.2024 14:41:29 - Received:{"shadeId":5,"target":80}
29.3.2024 14:41:29 - Moving to 80.00% from 90.00% using Up
29.3.2024 14:41:29 - CMD:Up ADDR:744165 RCODE:402 REPEAT:0
29.3.2024 14:41:29 - Enabled receive on Pin #35 Timing: 1
29.3.2024 14:41:31 - CMD:My ADDR:744165 RCODE:403 REPEAT:0
29.3.2024 14:41:31 - Enabled receive on Pin #35 Timing: 1
29.3.2024 14:41:31 - Timing Somfy: 221ms
29.3.2024 14:41:50 - Sending Shade Command
29.3.2024 14:41:50 - Received:{"shadeId":5,"command":"up"}
29.3.2024 14:41:50 - CMD:Up ADDR:744165 RCODE:404 REPEAT:0
29.3.2024 14:41:50 - Enabled receive on Pin #35 Timing: 1
29.3.2024 18:04:20 - Setting Last Rolling code 3235
29.3.2024 18:04:26 - Setting Last Rolling code 3236
29.3.2024 18:26:18 - Setting Last Rolling code 1688
29.3.2024 19:03:27 - Setting Last Rolling code 3237
29.3.2024 19:24:15 - Sending Shade Command
29.3.2024 19:24:15 - Received:{"shadeId":1,"command":"down"}
29.3.2024 19:24:15 - CMD:Down ADDR:744161 RCODE:394 REPEAT:0
29.3.2024 19:24:15 - Enabled receive on Pin #35 Timing: 1
29.3.2024 19:24:20 - Sending Shade Command
29.3.2024 19:24:20 - Received:{"shadeId":2,"command":"down"}
29.3.2024 19:24:20 - CMD:Down ADDR:744162 RCODE:311 REPEAT:0
29.3.2024 19:24:20 - Enabled receive on Pin #35 Timing: 1
29.3.2024 19:24:24 - Setting Last Rolling code 5674
29.3.2024 19:24:25 - Sending Shade Command
29.3.2024 19:24:25 - Received:{"shadeId":4,"command":"down"}
29.3.2024 19:24:25 - CMD:Down ADDR:744164 RCODE:306 REPEAT:0
29.3.2024 19:24:25 - Enabled receive on Pin #35 Timing: 1
29.3.2024 19:24:30 - Sending Shade Command
29.3.2024 19:24:30 - Received:{"shadeId":5,"command":"down"}
29.3.2024 19:24:30 - CMD:Down ADDR:744165 RCODE:405 REPEAT:0
29.3.2024 19:24:30 - Enabled receive on Pin #35 Timing: 1
29.3.2024 19:24:32 - Timing Somfy: 523ms
29.3.2024 19:24:35 - Sending Shade Command
29.3.2024 19:24:35 - Received:{"shadeId":6,"command":"down"}
29.3.2024 19:24:35 - CMD:Down ADDR:744166 RCODE:213 REPEAT:0
29.3.2024 19:24:36 - Enabled receive on Pin #35 Timing: 1
29.3.2024 22:59:59 - Sending Shade Command
29.3.2024 22:59:59 - Received:{"shadeId":3,"command":"down"}
29.3.2024 22:59:59 - CMD:Down ADDR:744163 RCODE:417 REPEAT:0
29.3.2024 22:59:59 - Enabled receive on Pin #35 Timing: 1
29.3.2024 23:00:06 - Sending Shade Command
29.3.2024 23:00:06 - Received:{"shadeId":3,"command":"up"}
29.3.2024 23:00:06 - CMD:Up ADDR:744163 RCODE:418 REPEAT:0
29.3.2024 23:00:06 - Enabled receive on Pin #35 Timing: 1
30.3.2024 07:23:00 - Setting Last Rolling code 3045
30.3.2024 07:23:00 - Setting Last Rolling code 2539
30.3.2024 07:23:06 - Setting Last Rolling code 1689
30.3.2024 07:23:22 - Timing Somfy: 541ms
30.3.2024 08:35:58 - Setting Last Rolling code 2844
30.3.2024 08:36:05 - Setting Last Rolling code 5675
30.3.2024 08:36:06 - Schlafzimmer Processing MY after 0 repeats
30.3.2024 08:36:07 - Setting Last Rolling code 5676
30.3.2024 08:51:24 - Setting Last Rolling code 5677
30.3.2024 08:59:59 - Sending Shade Command
30.3.2024 08:59:59 - Received:{"shadeId":1,"command":"up"}
30.3.2024 08:59:59 - CMD:Up ADDR:744161 RCODE:395 REPEAT:0
30.3.2024 08:59:59 - Enabled receive on Pin #35 Timing: 1
30.3.2024 09:00:04 - Sending Shade Command
30.3.2024 09:00:04 - Received:{"shadeId":2,"command":"up"}
30.3.2024 09:00:04 - CMD:Up ADDR:744162 RCODE:312 REPEAT:0
30.3.2024 09:00:05 - Enabled receive on Pin #35 Timing: 1
30.3.2024 09:00:10 - Sending Shade Command
30.3.2024 09:00:10 - Received:{"shadeId":3,"command":"up"}
30.3.2024 09:00:10 - CMD:Up ADDR:744163 RCODE:419 REPEAT:0
30.3.2024 09:00:10 - Enabled receive on Pin #35 Timing: 1
30.3.2024 09:00:15 - Sending Shade Command
30.3.2024 09:00:15 - Received:{"shadeId":4,"command":"up"}
30.3.2024 09:00:15 - CMD:Up ADDR:744164 RCODE:307 REPEAT:0
30.3.2024 09:00:15 - Enabled receive on Pin #35 Timing: 1
30.3.2024 09:38:58 - E (508655296) esp.emac: no mem for receive buffer
30.3.2024 09:38:58 - E (508655297) esp.emac: no mem for receive buffer
30.3.2024 12:21:31 - Check github for updates...
30.3.2024 12:21:31 - Max Heap: 36852
30.3.2024 12:21:31 - Free Heap: 58700
30.3.2024 12:21:31 - Min Heap: 40976
30.3.2024 12:21:31 - Internet is Unavailable: -1: 35ms
30.3.2024 12:44:36 - Setting Last Rolling code 5678
30.3.2024 13:08:45 - Sending Shade Command
30.3.2024 13:08:45 - Received:{"shadeId":5,"command":"up"}
30.3.2024 13:08:45 - CMD:Up ADDR:744165 RCODE:406 REPEAT:0
30.3.2024 13:08:45 - Enabled receive on Pin #35 Timing: 1
30.3.2024 13:43:57 - Setting Last Rolling code 5679
30.3.2024 15:10:09 - E (528526472) esp.emac: no mem for receive buffer
30.3.2024 15:57:23 - E (531359993) esp.emac: no mem for receive buffer
30.3.2024 15:57:23 - E (531359994) esp.emac: no mem for receive buffer
30.3.2024 15:57:23 - E (531359995) esp.emac: no mem for receive buffer
30.3.2024 16:01:05 - Sending Shade Command
30.3.2024 16:01:05 - Received:{"shadeId":5,"command":"up"}
30.3.2024 16:01:05 - CMD:Up ADDR:744165 RCODE:407 REPEAT:0
30.3.2024 16:01:06 - Enabled receive on Pin #35 Timing: 1
30.3.2024 16:05:10 - E (531827326) esp.emac: no mem for receive buffer
30.3.2024 16:30:11 - E (533327705) esp.emac: no mem for receive buffer
30.3.2024 16:30:11 - E (533327707) esp.emac: no mem for receive buffer
30.3.2024 16:30:11 - E (533327707) esp.emac: no mem for receive buffer
30.3.2024 16:48:11 - E (534408022) esp.emac: no mem for receive buffer
30.3.2024 17:02:37 - E (535274354) esp.emac: no mem for receive buffer
30.3.2024 17:02:37 - E (535274355) esp.emac: no mem for receive buffer
30.3.2024 17:02:37 - E (535274356) esp.emac: no mem for receive buffer
30.3.2024 17:02:37 - E (535274364) esp.emac: no mem for receive buffer
30.3.2024 17:02:37 - E (535274367) esp.emac: no mem for receive buffer
30.3.2024 17:02:37 - E (535274367) esp.emac: no mem for receive buffer
30.3.2024 17:25:11 - E (536628368) esp.emac: no mem for receive buffer
30.3.2024 18:15:57 - E (539674497) esp.emac: no mem for receive buffer
30.3.2024 18:15:57 - E (539674497) esp.emac: no mem for receive buffer
30.3.2024 18:15:57 - E (539674499) esp.emac: no mem for receive buffer
30.3.2024 18:15:57 - E (539674590) esp.emac: no mem for receive buffer
30.3.2024 18:15:57 - E (539674590) esp.emac: no mem for receive buffer
30.3.2024 18:15:57 - E (539674597) esp.emac: no mem for receive buffer
30.3.2024 18:16:04 - Timing WebServer: 6944ms
30.3.2024 18:16:14 - Timing Socket: 8720ms
30.3.2024 18:16:24 - Timing Socket: 9996ms
30.3.2024 18:16:34 - Timing Socket: 9996ms
30.3.2024 18:16:44 - Timing Socket: 9996ms
30.3.2024 18:16:54 - Timing Socket: 9996ms
30.3.2024 18:17:04 - Timing Socket: 9996ms
30.3.2024 18:17:14 - Timing Socket: 9996ms
30.3.2024 18:17:24 - Timing Socket: 9996ms
30.3.2024 18:17:34 - Timing Socket: 9996ms
30.3.2024 18:17:44 - Timing Socket: 9996ms
30.3.2024 18:17:54 - Timing Socket: 9996ms
30.3.2024 18:18:04 - Timing Socket: 9996ms
30.3.2024 18:18:14 - Timing Socket: 9996ms
30.3.2024 18:18:24 - Timing Socket: 9996ms
30.3.2024 18:18:34 - Timing Socket: 9996ms
30.3.2024 18:18:44 - Timing Socket: 9996ms
30.3.2024 18:18:54 - Timing Socket: 9996ms
30.3.2024 18:19:04 - Timing Socket: 9996ms
30.3.2024 18:19:14 - Timing Socket: 9996ms
30.3.2024 18:19:24 - Timing Socket: 9996ms
30.3.2024 18:19:34 - Timing Socket: 9996ms
30.3.2024 18:19:44 - Timing Socket: 9996ms
30.3.2024 18:19:54 - Timing Socket: 9996ms
30.3.2024 18:20:04 - Timing Socket: 9996ms
30.3.2024 18:20:14 - Timing Socket: 9996ms
30.3.2024 18:20:24 - Timing Socket: 9996ms
30.3.2024 18:20:34 - Timing Socket: 9996ms
30.3.2024 18:20:44 - Timing Socket: 9996ms
30.3.2024 18:20:54 - Timing Socket: 9996ms
30.3.2024 18:21:04 - Timing Socket: 9996ms
30.3.2024 18:21:14 - Timing Socket: 9996ms
30.3.2024 18:21:24 - Timing Socket: 9996ms
30.3.2024 18:21:34 - Timing Socket: 9996ms
30.3.2024 18:21:44 - Timing Socket: 9996ms
30.3.2024 18:21:54 - Timing Socket: 9996ms
30.3.2024 18:22:04 - Timing Socket: 9996ms
30.3.2024 18:22:14 - Timing Socket: 9996ms
30.3.2024 18:22:24 - Timing Socket: 9995ms
30.3.2024 18:22:34 - Timing Socket: 9996ms
30.3.2024 18:22:44 - Timing Socket: 9996ms
30.3.2024 18:22:54 - Timing Socket: 9996ms
30.3.2024 18:23:04 - Timing Socket: 9996ms
30.3.2024 18:23:14 - Timing Socket: 9996ms
30.3.2024 18:23:24 - Timing Socket: 9996ms
30.3.2024 18:23:34 - Timing Socket: 9996ms
30.3.2024 18:23:44 - Timing Socket: 9996ms
30.3.2024 18:23:54 - Timing Socket: 9996ms
30.3.2024 18:24:04 - Timing Socket: 9996ms
30.3.2024 18:24:14 - Timing Socket: 9996ms
30.3.2024 18:24:24 - Timing Socket: 9996ms
30.3.2024 18:24:34 - Timing Socket: 9996ms
30.3.2024 18:24:44 - Timing Socket: 9996ms
30.3.2024 18:24:54 - Timing Socket: 9996ms
30.3.2024 18:25:04 - Timing Socket: 9996ms
30.3.2024 18:25:14 - Timing Socket: 9996ms
30.3.2024 18:25:24 - Timing Socket: 9996ms
30.3.2024 18:25:34 - Timing Socket: 9996ms
30.3.2024 18:25:44 - Timing Socket: 9996ms
30.3.2024 18:25:54 - Timing Socket: 9996ms
30.3.2024 18:26:04 - Timing Socket: 9996ms
30.3.2024 18:26:14 - Timing Socket: 9996ms
30.3.2024 18:26:24 - Timing Socket: 9996ms
30.3.2024 18:26:34 - Timing Socket: 9996ms
30.3.2024 18:26:44 - Timing Socket: 9996ms
30.3.2024 18:26:54 - Timing Socket: 9996ms
30.3.2024 18:27:04 - Timing Socket: 9996ms
30.3.2024 18:27:14 - Timing Socket: 9996ms
30.3.2024 18:27:24 - Timing Socket: 9996ms
30.3.2024 18:27:34 - Timing Socket: 9996ms
30.3.2024 18:27:44 - Timing Socket: 9996ms
30.3.2024 18:27:54 - Timing Socket: 9996ms
30.3.2024 18:28:04 - Timing Socket: 9996ms
30.3.2024 18:28:14 - Timing Socket: 9996ms
30.3.2024 18:28:24 - Timing Socket: 9996ms
30.3.2024 18:28:34 - Timing Socket: 9995ms
30.3.2024 18:28:44 - Timing Socket: 9996ms
30.3.2024 18:28:54 - Timing Socket: 9996ms
30.3.2024 18:29:04 - Timing Socket: 9996ms
30.3.2024 18:29:14 - Timing Socket: 9996ms
30.3.2024 18:29:24 - Timing Socket: 9996ms
30.3.2024 18:29:34 - Timing Socket: 9996ms
30.3.2024 18:29:44 - Timing Socket: 9996ms
30.3.2024 18:29:54 - Timing Socket: 9996ms
30.3.2024 18:30:04 - Timing Socket: 9996ms
30.3.2024 18:30:14 - Timing Socket: 9996ms
30.3.2024 18:30:24 - Timing Socket: 9996ms
30.3.2024 18:30:34 - Timing Socket: 9996ms
30.3.2024 18:30:44 - Timing Socket: 9996ms
30.3.2024 18:30:54 - Timing Socket: 9996ms
30.3.2024 18:31:04 - Timing Socket: 9996ms
30.3.2024 18:31:14 - Timing Socket: 9996ms
30.3.2024 18:31:24 - Timing Socket: 9996ms
30.3.2024 18:31:34 - Timing Socket: 9996ms
30.3.2024 18:31:44 - Timing Socket: 9996ms
30.3.2024 18:31:54 - Timing Socket: 9996ms
30.3.2024 18:32:04 - Timing Socket: 9996ms
30.3.2024 18:32:14 - Timing Socket: 9996ms
30.3.2024 18:32:24 - Timing Socket: 9996ms
30.3.2024 18:32:34 - Timing Socket: 9996ms
30.3.2024 18:32:44 - Timing Socket: 9996ms
30.3.2024 18:32:54 - Timing Socket: 9996ms
30.3.2024 18:33:04 - Timing Socket: 9996ms
30.3.2024 18:33:14 - Timing Socket: 9996ms
30.3.2024 18:33:24 - Timing Socket: 9996ms
30.3.2024 18:33:34 - Timing Socket: 9996ms
30.3.2024 18:33:44 - Timing Socket: 9996ms
30.3.2024 18:33:54 - Timing Socket: 9996ms
30.3.2024 18:34:04 - Timing Socket: 9996ms
30.3.2024 18:34:14 - Timing Socket: 9996ms
30.3.2024 18:34:24 - Timing Socket: 9996ms
30.3.2024 18:34:34 - Timing Socket: 9996ms
30.3.2024 18:34:44 - Timing Socket: 9995ms
30.3.2024 18:34:54 - Timing Socket: 9996ms
30.3.2024 18:35:04 - Timing Socket: 9996ms
30.3.2024 18:35:14 - Timing Socket: 9996ms
30.3.2024 18:35:24 - Timing Socket: 9996ms
30.3.2024 18:35:34 - Timing Socket: 9996ms
30.3.2024 18:35:44 - Timing Socket: 9996ms
30.3.2024 18:35:54 - Timing Socket: 9996ms
30.3.2024 18:36:04 - Timing Socket: 9996ms
30.3.2024 18:36:14 - Timing Socket: 9996ms
30.3.2024 18:36:23 - Timing Socket: 9996ms
30.3.2024 18:36:33 - Timing Socket: 9996ms
30.3.2024 18:36:43 - Timing Socket: 9996ms
30.3.2024 18:36:53 - Timing Socket: 9996ms
30.3.2024 18:37:03 - Timing Socket: 9996ms
30.3.2024 18:37:13 - Timing Socket: 9996ms
30.3.2024 18:37:23 - Timing Socket: 9996ms
30.3.2024 18:37:33 - Timing Socket: 9996ms
30.3.2024 18:37:43 - Timing Socket: 9996ms
30.3.2024 18:37:53 - Timing Socket: 9996ms
30.3.2024 18:38:03 - Timing Socket: 9996ms
30.3.2024 18:38:13 - Timing Socket: 9996ms
30.3.2024 18:38:23 - Timing Socket: 9996ms
30.3.2024 18:38:33 - Timing Socket: 9996ms
30.3.2024 18:38:43 - Timing Socket: 9996ms
30.3.2024 18:38:53 - Timing Socket: 9996ms
30.3.2024 18:39:03 - Timing Socket: 9996ms
30.3.2024 18:39:13 - Timing Socket: 9996ms
30.3.2024 18:39:23 - Timing Socket: 9996ms
30.3.2024 18:39:33 - Timing Socket: 9996ms
30.3.2024 18:39:43 - Timing Socket: 9996ms
30.3.2024 18:39:53 - Timing Socket: 9996ms
30.3.2024 18:40:03 - Timing Socket: 9996ms
30.3.2024 18:40:13 - Timing Socket: 9996ms
30.3.2024 18:40:23 - Timing Socket: 9996ms
30.3.2024 18:40:33 - Timing Socket: 9996ms
30.3.2024 18:40:43 - Timing Socket: 9996ms
30.3.2024 18:40:53 - Timing Socket: 9995ms
30.3.2024 18:41:03 - Timing Socket: 9996ms
30.3.2024 18:41:13 - Timing Socket: 9996ms
30.3.2024 18:41:23 - Timing Socket: 9996ms
30.3.2024 18:41:31 - Timing Socket: 7996ms
30.3.2024 18:42:01 - Socket [0] Disconnected!
26.3.2024 19:19:30 - Received:{"shadeId":1,"command":"down"}
26.3.2024 19:19:30 - CMD:Down ADDR:744161 RCODE:388 REPEAT:0
26.3.2024 19:19:30 - Enabled receive on Pin #35 Timing: 1
26.3.2024 19:19:30 - Timing WebServer: 223ms
26.3.2024 19:19:35 - Sending Shade Command
26.3.2024 19:19:35 - Received:{"shadeId":2,"command":"down"}
26.3.2024 19:19:35 - CMD:Down ADDR:744162 RCODE:305 REPEAT:0
26.3.2024 19:19:36 - Enabled receive on Pin #35 Timing: 1
26.3.2024 19:19:41 - Sending Shade Command
26.3.2024 19:19:41 - Received:{"shadeId":4,"command":"down"}
26.3.2024 19:19:41 - CMD:Down ADDR:744164 RCODE:300 REPEAT:0
26.3.2024 19:19:41 - Enabled receive on Pin #35 Timing: 1
26.3.2024 19:19:46 - Sending Shade Command
26.3.2024 19:19:46 - Received:{"shadeId":5,"command":"down"}
26.3.2024 19:19:46 - CMD:Down ADDR:744165 RCODE:383 REPEAT:0
26.3.2024 19:19:46 - Enabled receive on Pin #35 Timing: 1
26.3.2024 19:19:51 - Sending Shade Command
26.3.2024 19:19:51 - Received:{"shadeId":6,"command":"down"}
26.3.2024 19:19:51 - CMD:Down ADDR:744166 RCODE:210 REPEAT:0
26.3.2024 19:19:51 - Enabled receive on Pin #35 Timing: 1
26.3.2024 22:59:59 - Sending Shade Command
26.3.2024 22:59:59 - Received:{"shadeId":3,"command":"down"}
26.3.2024 22:59:59 - CMD:Down ADDR:744163 RCODE:411 REPEAT:0
26.3.2024 23:00:00 - Enabled receive on Pin #35 Timing: 1
27.3.2024 02:48:29 - Setting Last Rolling code 3232
27.3.2024 07:15:18 - Setting Last Rolling code 2841
27.3.2024 07:15:21 - Setting Last Rolling code 3043
27.3.2024 07:15:22 - Setting Last Rolling code 2537
27.3.2024 07:15:27 - Setting Last Rolling code 1686
27.3.2024 07:15:35 - Timing Somfy: 536ms
27.3.2024 07:16:50 - Sending Shade Command
27.3.2024 07:16:50 - Received:{"shadeId":5,"command":"up"}
27.3.2024 07:16:50 - CMD:Up ADDR:744165 RCODE:384 REPEAT:0
27.3.2024 07:16:51 - Enabled receive on Pin #35 Timing: 1
27.3.2024 08:59:59 - Sending Shade Command
27.3.2024 08:59:59 - Received:{"shadeId":1,"command":"up"}
27.3.2024 08:59:59 - CMD:Up ADDR:744161 RCODE:389 REPEAT:0
27.3.2024 09:00:00 - Enabled receive on Pin #35 Timing: 1
27.3.2024 09:00:05 - Sending Shade Command
27.3.2024 09:00:05 - Received:{"shadeId":2,"command":"up"}
27.3.2024 09:00:05 - CMD:Up ADDR:744162 RCODE:306 REPEAT:0
27.3.2024 09:00:05 - Enabled receive on Pin #35 Timing: 1
27.3.2024 09:00:10 - Sending Shade Command
27.3.2024 09:00:10 - Received:{"shadeId":3,"command":"up"}
27.3.2024 09:00:10 - CMD:Up ADDR:744163 RCODE:412 REPEAT:0
27.3.2024 09:00:10 - Enabled receive on Pin #35 Timing: 1
27.3.2024 09:00:15 - Sending Shade Command
27.3.2024 09:00:15 - Received:{"shadeId":4,"command":"up"}
27.3.2024 09:00:15 - CMD:Up ADDR:744164 RCODE:301 REPEAT:0
27.3.2024 09:00:15 - Enabled receive on Pin #35 Timing: 1
27.3.2024 11:36:38 - Setting Last Rolling code 5669
27.3.2024 12:21:29 - Check github for updates...
27.3.2024 12:21:29 - Max Heap: 77812
27.3.2024 12:21:29 - Free Heap: 119556
27.3.2024 12:21:29 - Min Heap: 77400
27.3.2024 12:21:31 - Internet is Available: 1905ms
27.3.2024 12:21:32 - [HTTPS] GET... code: 200
27.3.2024 12:21:32 - [HTTPS] GET... code: 200 - -1
27.3.2024 12:21:33 - Max Heap: 38900
27.3.2024 12:21:33 - Free Heap: 116956
27.3.2024 12:21:33 - Min Heap: 62492
27.3.2024 12:21:33 - Timing WebServer: 3804ms
27.3.2024 14:48:02 - Sending Shade Command
27.3.2024 14:48:02 - Received:{"shadeId":5,"target":80}
27.3.2024 14:48:02 - Moving to 80.00% from 100.00% using Up
27.3.2024 14:48:02 - CMD:Up ADDR:744165 RCODE:385 REPEAT:0
27.3.2024 14:48:02 - Enabled receive on Pin #35 Timing: 1
27.3.2024 14:48:06 - CMD:My ADDR:744165 RCODE:386 REPEAT:0
27.3.2024 14:48:06 - Enabled receive on Pin #35 Timing: 1
27.3.2024 14:48:06 - Timing Somfy: 208ms
27.3.2024 14:52:17 - Sending Shade Command
27.3.2024 14:52:17 - Received:{"shadeId":5,"command":"up"}
27.3.2024 14:52:17 - CMD:Up ADDR:744165 RCODE:387 REPEAT:0
27.3.2024 14:52:17 - Enabled receive on Pin #35 Timing: 1
27.3.2024 19:17:28 - Setting Last Rolling code 5670
27.3.2024 19:21:05 - Sending Shade Command
27.3.2024 19:21:05 - Received:{"shadeId":1,"command":"down"}
27.3.2024 19:21:05 - CMD:Down ADDR:744161 RCODE:390 REPEAT:0
27.3.2024 19:21:06 - Enabled receive on Pin #35 Timing: 1
27.3.2024 19:21:11 - Sending Shade Command
27.3.2024 19:21:11 - Received:{"shadeId":2,"command":"down"}
27.3.2024 19:21:11 - CMD:Down ADDR:744162 RCODE:307 REPEAT:0
27.3.2024 19:21:11 - Enabled receive on Pin #35 Timing: 1
27.3.2024 19:21:16 - Sending Shade Command
27.3.2024 19:21:16 - Received:{"shadeId":4,"command":"down"}
27.3.2024 19:21:16 - CMD:Down ADDR:744164 RCODE:302 REPEAT:0
27.3.2024 19:21:16 - Enabled receive on Pin #35 Timing: 1
27.3.2024 19:21:21 - Sending Shade Command
27.3.2024 19:21:21 - Received:{"shadeId":5,"command":"down"}
27.3.2024 19:21:21 - CMD:Down ADDR:744165 RCODE:388 REPEAT:0
27.3.2024 19:21:21 - Enabled receive on Pin #35 Timing: 1
27.3.2024 19:21:26 - Sending Shade Command
27.3.2024 19:21:26 - Received:{"shadeId":6,"command":"down"}
27.3.2024 19:21:26 - CMD:Down ADDR:744166 RCODE:211 REPEAT:0
27.3.2024 19:21:26 - Enabled receive on Pin #35 Timing: 1
27.3.2024 19:21:27 - Timing Somfy: 529ms
27.3.2024 23:00:00 - Request /api/config 404-1 GET Sending Shade Command
27.3.2024 23:00:00 - Received:{"shadeId":3,"command":"down"}
27.3.2024 23:00:00 - CMD:Down ADDR:744163 RCODE:413 REPEAT:0
27.3.2024 23:00:00 - Enabled receive on Pin #35 Timing: 1
28.3.2024 04:07:33 - Setting Last Rolling code 3233
28.3.2024 07:09:20 - Setting Last Rolling code 2842
28.3.2024 07:10:17 - Setting Last Rolling code 3044
28.3.2024 07:10:17 - Setting Last Rolling code 2538
28.3.2024 07:10:28 - Setting Last Rolling code 1687
28.3.2024 07:10:44 - Timing Somfy: 519ms
28.3.2024 09:00:00 - Sending Shade Command
28.3.2024 09:00:00 - Received:{"shadeId":1,"command":"up"}
28.3.2024 09:00:00 - CMD:Up ADDR:744161 RCODE:391 REPEAT:0
28.3.2024 09:00:00 - Enabled receive on Pin #35 Timing: 1
28.3.2024 09:00:05 - Sending Shade Command
28.3.2024 09:00:05 - Received:{"shadeId":2,"command":"up"}
28.3.2024 09:00:05 - CMD:Up ADDR:744162 RCODE:308 REPEAT:0
28.3.2024 09:00:05 - Enabled receive on Pin #35 Timing: 1
28.3.2024 09:00:10 - Sending Shade Command
28.3.2024 09:00:10 - Received:{"shadeId":3,"command":"up"}
28.3.2024 09:00:10 - CMD:Up ADDR:744163 RCODE:414 REPEAT:0
28.3.2024 09:00:10 - Enabled receive on Pin #35 Timing: 1
28.3.2024 09:00:15 - Sending Shade Command
28.3.2024 09:00:15 - Received:{"shadeId":4,"command":"up"}
28.3.2024 09:00:15 - CMD:Up ADDR:744164 RCODE:303 REPEAT:0
28.3.2024 09:00:15 - Enabled receive on Pin #35 Timing: 1
28.3.2024 09:00:18 - Sending Shade Command
28.3.2024 09:00:18 - Received:{"shadeId":5,"command":"up"}
28.3.2024 09:00:18 - CMD:Up ADDR:744165 RCODE:389 REPEAT:0
28.3.2024 09:00:18 - Enabled receive on Pin #35 Timing: 1
28.3.2024 12:21:30 - Check github for updates...
28.3.2024 12:21:30 - Max Heap: 55284
28.3.2024 12:21:30 - Free Heap: 98384
28.3.2024 12:21:30 - Min Heap: 62492
28.3.2024 12:21:31 - Internet is Available: 1909ms
28.3.2024 12:21:33 - [HTTPS] GET... code: 200
28.3.2024 12:21:33 - [HTTPS] GET... code: 200 - -1
28.3.2024 12:21:33 - Max Heap: 36852
28.3.2024 12:21:33 - Free Heap: 95740
28.3.2024 12:21:33 - Min Heap: 42604
28.3.2024 12:21:33 - Timing WebServer: 3792ms
28.3.2024 12:21:38 - Setting Last Rolling code 5671
28.3.2024 14:22:48 - Setting Last Rolling code 5672
28.3.2024 19:22:39 - Sending Shade Command
28.3.2024 19:22:39 - Received:{"shadeId":1,"command":"down"}
28.3.2024 19:22:39 - CMD:Down ADDR:744161 RCODE:392 REPEAT:0
28.3.2024 19:22:40 - Enabled receive on Pin #35 Timing: 1
28.3.2024 19:22:45 - Sending Shade Command
28.3.2024 19:22:45 - Received:{"shadeId":2,"command":"down"}
28.3.2024 19:22:45 - CMD:Down ADDR:744162 RCODE:309 REPEAT:0
28.3.2024 19:22:45 - Enabled receive on Pin #35 Timing: 1
28.3.2024 19:22:50 - Sending Shade Command
28.3.2024 19:22:50 - Received:{"shadeId":4,"command":"down"}
28.3.2024 19:22:50 - CMD:Down ADDR:744164 RCODE:304 REPEAT:0
28.3.2024 19:22:50 - Enabled receive on Pin #35 Timing: 1
28.3.2024 19:22:55 - Sending Shade Command
28.3.2024 19:22:55 - Received:{"shadeId":5,"command":"down"}
28.3.2024 19:22:55 - CMD:Down ADDR:744165 RCODE:390 REPEAT:0
28.3.2024 19:22:55 - Enabled receive on Pin #35 Timing: 1
28.3.2024 19:23:00 - Sending Shade Command
28.3.2024 19:23:00 - Received:{"shadeId":6,"command":"down"}
28.3.2024 19:23:00 - CMD:Down ADDR:744166 RCODE:212 REPEAT:0
28.3.2024 19:23:00 - Enabled receive on Pin #35 Timing: 1
28.3.2024 20:58:20 - Socket [0] Disconnected!
28.3.2024 21:13:21 - Discovery Requested
28.3.2024 21:13:21 - Socket [0] Connected from 10.11.12.142 url: /
28.3.2024 22:59:59 - Sending Shade Command
28.3.2024 22:59:59 - Received:{"shadeId":3,"command":"down"}  

[rstrouse]

Please turn off the GitHub update check and monitor.

[Secarius]

Do you mean the serial Monitor which i use to log the serial output?

[rstrouse]

Uncheck Auto Check for Updates on the options tab and press Save.

EDIT: The update check looks like it is not releasing memory right away.

[tonoho]

My ESPSomfy-RTS also becomes unreachable every few weeks running v2.4.1. It's a standard ESP32 running on wifi, so no ethernet board involved. I disabled the auto update checking now to see if it keeps up longer then. Could it be the same issue or should I try connecting some pins for logging?

[mvantschip]

I have the same problems as @tonoho. I have also turned off auto-update. Just chiming in to say that if you need any additional logs or actions from my side to help alleviate this problem, I would be happy to help! Big fan of the project :)

[rstrouse]

@tonoho & @mvantschip it would be great if you guys turn off the auto-check for updates. I am doing my own monitoring as well with 6 different devices to see if I can get them to fail. I don't think I need any more logs at this point since I have narrowed this down to the ESP32 network libraries. The failure mode seems to indicate everything is working except the one thing that is required to operate the motors and report their status... the network.

I have seen other posts that seem to indicate similar issues with the socket and network lwip issues on ESP32 and am attempting to dig into the libraries to see if the software creates a condition where the socket does not close properly. I suspect this is actually what is going on since it can serve the files but is not receiving the requests or not responding to them.

[Secarius]

Uncheck Auto Check for Updates on the options tab and press Save.

EDIT: The update check looks like it is not releasing memory right away.

Did Not help, it went off 3 days ago and Till now it did Not come back on

[rstrouse]

Please install the v2.4.2 pre-release I have removed the JSON serialization libraries. These libraries use a lot of memory when they are writing to the socket so I decided that I didn't need them and am writing to the socket using my own process. My iffy WT32-ETH01 has been solid for several days.

[Jumperm]

I have installed the pre-Release yesterday and will Test it. also i disabled the check update.

[rstrouse]

Have all of you updated to v2.4.2 if so have there been any outages?

[Secarius]

Yes already updated and so far no issues