Ethernet disconnects when relay is switching 240V

[nagyrobi]

My ESP32-EVB exhibits a strange behavior. I have 2 consumers connected to the relays, one relay is switching a circuit feedig a small fishtank oxgen bubbles pump at 12V, the other relay is swithing a heating cable of 90W, at 240V. The board is powered by a quality 5V 2A MeanWell type power source.

Occasionally the board looses Ethernet connection after switching on the relay in the 240V circuit. The MCU is still running because I can see the other relay doing its job (switching on and off every few minutes). This only happens when the 240V circuit is powered on, regardless of the relay: I tried to swap the relays (use the first one for 240V and the second one for 12V) and the problem moved with the 240V circuit. No problem when switching 12V. Ethernet only disconnects when 240V switching occurs.

This happens in 70% of the cases and the biggest issue is that the board has to be powered off and back on to recover connection which is painful. Sometimes I have to cut the power twice to get the board's Ethernet back.

Seems like I'm not the only one having this issue: https://esp32.com/viewtopic.php?t=8055 Somehow I have a feeling that there's an interference issue affecting the power_pin of the LAN chip...? (On ESP32-POE it's connected to the MCU...)

Do you have any ideas what could be done? Any ferrite beads added to the cables could help? A capacitor or a transil (supressor) diode maybe?

[TsvetanUsunov]

Please try to move away the cable with high voltage current from ESP32-EVB. The high voltage / high current pulses induct spikes to the LAN trafo and PHY tracks which confuse the communication. When you add the external board in practice you move away the high voltage/high current stuff away of LAN.

[nagyrobi]

OK I'll try to route that circuit as far as possible within the box, and let you know about the results.

[nagyrobi]

Please look at the picture below:

The cable marked with red arrow is the one carrying the 240V. It goes as far as possible from the PCB. Also note that this is shielded power cable (it has a wire mesh shielding below the insulation, with 2x1mm2 wires inside. Shield is connectedt to the earth ground together with the GND of the board (taken from the CAN connector).

This has brought some improvement, but still not 100% reliable.

I also noticed with an ohmmeter that the case of the RJ45 socket is not galvanically connected to GND. In the schematic it's only pulled down with two capacitors. Since I don't use shielded CAT cable, I also try to connect this galvanically. Transformers are in this socket, right?

Note that the currents are not high at all. At most 400mA on both 12VDC and 240VAC through the relays. I wouldn't think this small current would induce too much interference. As for the spikes, I think they may be more or less caused by the arching inside the relay, at the contacts. Because I was also able to reproduce failure at switching OFF, not only at switching ON. I agree that current spikes may happen at turn ON, but not really on turn OFF...

[nagyrobi]

Unfortunately it is still very unreliable and I really can't figure out how to fix this... things are excellent for days until the heating cable needs to be switched on.

[nagyrobi]

Uptime graph with relay acting: (graphs are misaligned due to web interface bug, it can be seen that after each green period a very short red appears, after which the gray shows it's offline). You see that in the latest tests the problem occurs when the relay turns OFF (breaks the circuit), not when it turns ON. This should mean the problem is not caused by high current pulses inducting spikes in the transformers of the LAN interface, as when the circuit breaks, no high high current pulses can occur.

[nagyrobi]

I added a second relay between ESP32-EVB and the 240V circuit, so that the onboard relays would only switch 12V. The problem is more rare, but still persists unfortunately.

Next experiment is to completely isolate the relay from ESP32-EVB, but keep the high voltage cabling in the box in the same position as the last photo above. I will use a Sonoff Mini which will connect galvanically only to the 240V circuit, ESP32-EVB will send its commands to it via MQTT (LAN port > switch > AP > WiFi). The Sonoff Mini will do nothing else bit switch the high voltage based on ON/OFF commands coming via MQTT from ESP32-EVB.

[nagyrobi]

Uptime graph when instead of local relay I employed a Sonoff Mini:

In the same box, same circuitry, same cables:

No problems neither at switching ON or OFF. Sonoff Mini also has a mechanical relay inside, so if any high current spikes are generated when a relay acts, these are present now too. But in this case, it doesn't effect the Ethernet...? (this is just a test, don't want the Sonoff there)

[nagyrobi]

Maybe: https://electronics.stackexchange.com/a/321355 ?

[TsvetanUsunov]

inductive loads generate high voltage pulses when the power is cut off. what is the load you switch on and off?

[nagyrobi]

As I stated above in my first post, it's only 90W. The Sonoff Mini has no such problems.

[TsvetanUsunov]

the problem is not the heater, but the bubble pump they are big inductor with vibrating magnet when the pump is switched off the inductor generates high voltage spike try to switch on.off only the heater and see if these problems still exist. the sonoff have no such problem but have no wired Ethernet as well

[nagyrobi]

It has no problem with the little pump. It's only 2W. That is still switched ON/OFF by ESP32-EVB and no problems occur.

Above I described that problem happens ONLY when the heater circuit switches OFF by relay on ESP32-EVB.

It happens exactly in that moment.

Any filter, surpressor recommendation?

[DanKoloff]

We will perform some tests, but need to know what is the hardware revision of your board exactly?

Also we did some empirical tests with a drill and vacuum cleaner with older revisions of ESP32-EVB two years ago, there was no issue back then, maybe newer revisions or components introduced such issue. Will know better when we test. This is what we tested at the end of 2018:

https://www.youtube.com/watch?v=iTl9EQknGjA

https://www.youtube.com/watch?v=x3e_u8LgnTs

[nagyrobi]

It's latest, revision I.

Perhaps try with a few hair dryers :-) Note that these devices may have some surpressor circuits built-in.

The heating cable we use is: https://www.bvfheating.hu/shop/futokabel/belteri-futokabel/bvf-wfd-10w-m-futokabel/ But only 9 meters long so that's 90W (measured) - it's 10W/m.

[nagyrobi]

I tried adding a MOV (VDR-14D431K) to absorb the energy but still no help:

[DanKoloff]

Try removing C31 and test if it hangs again. Just be aware that this will hinder further USB programming. You would need to place it back to program via USB.

[nagyrobi]

Were you able to reproduce? I'll try and let you know.

[nagyrobi]

OK removed C31 and at first looked promising, I was able to turn the load ON/OFF for about 10 times and after that problem re-appeared. After that, after power-cycling the system it was enough to turn the relay ON/OFF only 1 or 2 times to reach in problem. Tried re-adding the MOV but absolutely no difference with or without it.

No problem about ESP_EN without the capacitor, I see t's out on the EXT1 header so I can always put back a capacitor temporarily for during the flash with jumper wires.

[DanKoloff]

We did manage to bug the Ethernet by using the relays but can't say that it is a reproduction since the hardware setups are different. In fact, here removing C31 fixes the issue but it doesn't at your side, so there are differences, for sure. This also means that your feedback is important to solve this problem. We are looking into every possibility but so far can't find the reason for this behavior, including routing seems alright. We keep working and if there are further tests that you might carry I would let you know.

[nagyrobi]

Ok, let me know what extra feedback you need As a sidenote, the board is used at outdoor temperature (dry attic), currently -1-2C. Maybe first test after removing C31 was partially OK because it was just brought back there from room temp...?

[nagyrobi]

I tested wit an SSR directly on a separate GPIO, it has no problems when switching high load through that:

[TsvetanUsunov]

we found the root of the problem and work on solution now, will post the results later today

[TsvetanUsunov]

Hi, Please solder 0603 1 NF 50VDC capacitor on C18 footprint and let us know the result. Thanks

[nagyrobi]

So you want me to remove C18 which is 10uF, and put in place 1nF?

[TsvetanUsunov]

yes, I see on the picture above C18 is not populated

[nagyrobi]

I need some time for this now, I'll let you know later on

[nagyrobi]

Unfortunately I don't have the tools (and skills) to solder on SMD parts. I was able to remove C31 because it was at the edge of the board.

[DanKoloff]

Unfortunately I don't have the tools (and skills) to solder on SMD parts. I was able to remove C31 because it was at the edge of the board.

Alternatively, instead of placing it on C18 position, you can solder the extra 1nF capacitor at the bottom of the board on the PHY_RST1 jumper, this should make it easier. I attach a picture that shows how we've done it here:

Let me know if that is doable and how it goes.

[nagyrobi]

I can try soldering a classical capacitor not an SMD one. How will these modifications affect my warranty (including removal of C31)?

[DanKoloff]

Alright, PTH would also work, maybe consider NP0(C0G) one. If after the addition of this capacitor on the bottom, there are any problems with start up maybe try putting C31 back in its place.

If we instructed you to do hardware changes, as in this case, then the warranty is unaffected. It is not like you decided to do these yourself.

[nagyrobi]

Since I temporarily fixed this with an SSR on another GPIO pin (and that works stable), I had to postpone investigation due to other projects. But please don't close this issue as I want to fix this.

[nagyrobi]

Unfortunately I was not able to make it run stable with the 1nF capacitor soldered. It's a pity because otherwise it's a nice board. Sonoff Mini works...

Anyways, since I can't use the relays, I had to sacrifice 2 GPIO pins from the UEXT connector for a SSR and a FET (I wanted to add I2C screen and a rotary encoder to create a local UI for control, can't do that anymore over UEXT):

SSR type is MRA-23D2, can be driven directly by 3.3V (has opto built-in), and fits nicely on DIN rail:

No more Ethernet disconnects since. Relays remain unused.

[nagyrobi]

If I knew all this, an ESP32-GATEWAY-IND would have been enough... chose the EVB exactly because of the relays... I wonder, could I get a compensation/discount for my next purchase because of this?

[nagyrobi]

😠 👎

[peerweb-ant]

I tested wit an SSR directly on a separate GPIO, it has no problems when switching high load through that:

Hi, same problem. Used to open the door: is electric magnet 12v~. If I disconnect the magnet, everything OK. So I already ordered an ss-relay. Which gpio did you use? Not a lot available on this board. I use the sd-card, ethernet, wifi (for esp-now), I don't use CAN or IR. Tx in advance for your reply. Peter

[peerweb-ant]

With SSR it's OK. GPIO 5 and 12 (and 32 and 33) are OK as OUTPUT, 35 and 39 are only INPUT. Yes, a bit sad that AC-current trough onboard-relays kills ethernet. Could have ordered board without relays.

[nagyrobi]

Same experience.

[DanKoloff]

About a board without relays - ESP32-GATEWAY is quite similar but without relays and few extra free pins. Alternatively, ESP32-POE-ISO.

About ESP32-EVB and relays - it is just these are electromechanical relays and there are some engineering considerations to be made when using them. The fact that your issues are fixed by using solid state relay is further confirmation of that. Start by placing the board away from the door controller so they don't influence each other. Inductive AC and DC loads are hard for electromechanical relays where welding contacts cause problems. Normal way to solve this problem is to use clamp diodes or zener diodes in the poles of the load. However this lengthens the off delay and it does not totally eliminate the sparking in relay contacts. Switching the high DC voltages is also difficult for the electromechanical relays. I am sure there is a lot of solutions that might be suggested online. There are a number of workarounds or partial solutions but they depend on your exact design and setup. There is no solution that I can recommend 100% - you will have to try different things. And in some specific cases electromechanical relays are not suitable at all and can't be used for your goals.

We did a lot of tests of the relays when we designed the board, even some funny empirical ones can be found in these videos (I already shared these links further above):

https://www.youtube.com/watch?v=iTl9EQknGjA https://www.youtube.com/watch?v=x3e_u8LgnTs

[peerweb-ant]

Hi, Tx for your reply. It's not "spark-issue". I know that's also a risk. I have done a test by moving the board away from powerlines, power supplies, ... . Same problem. As soon as AC-current is switched by the relays, ethernet hangs. Board near or moved-away, if I disconnect doormagnet-wire, relay functions fine, ethernet keeps fine, although AC-voltage is nearby. So, something around the relays isn't happy with switching AC-current. I even did a test by using both relays, the 2nd one current in reverse direction (to get some sort of zero field). Didn't help.

Spark-issue (burn in) is another story. Relay displays 10A, it should be able to handle 1A, a doormagnet.

Whatever, it was cheap. So, it's fine. And now it works. I appreciate your explainings.

Peter

[nagyrobi]

Additionally, if I place anything else, like a Sonoff Mini or any other cheap chinese thing, that doesn't hang.

[DanKoloff]

@peerweb-ant What is the hardware revision of your ESP32-EVB? It should be printed at the bottom of the PCB, under the name of the board?

[peerweb-ant]

Rev K. But all OK now.

[DanKoloff]

Thanks, I understand that you found a workaround but we still investigate reports like that here.