serial communication error in master-slave configuration

[jaroslawp]

My 2 SmartEVSEs are configured in a Master-Slave configuration with a sensorbox connected on the bus.

When the slave charger is in negotiation with the car just after connecting the charging cable both SmartEVSE start flashing "ERROR NO SERIAL COMM CHECK WIRING" and both chargers stop. complete power cycle of both is needed to restore the fuctionality: sometimes it takes 2-3 times before it works again. So far this only happened while connecting slave one, connecting master one seems to work OK.

I am suspecting that slave evse is trying to communicate with sensorbox directly instead of getting data from master which could be causing the problem (in the past I ran in a kind of similar problem while trying to change mode directly on slave via API or connected button to set it to a different one that master: apparently in master-slave mode this should be done on master only - which then sets same mode on slave )

Not sure how can I debug this further ? (the http://smartevse/settings only shows "Communications error", error number: 2)

This started happening with 1.6.5 release (previously I was using 1.4.6 for a long time ... so not sure when the problem really could have started ?):

[dingo35]

1. Upgrade to latest version before posting bug reports
2. If you flash firmware.debug.bin you can telnet to your smartevse device to see what is going on on your modbus.

[dingo35]

EDIT: Your first post mentioned version 1.5.6, now it says 1.6.5?

[jaroslawp]

Yes, made a typo it is 1.6.5

[dingo35]

Ok, try to makea telnet log and post that here, since I only have 1 SmartEVSE I cannot reproduce your problem.

Are you sure none of the SmartEVSE's are configured to EM_API?

A structured method is to pick a release, test if it has the problem, go up or down a few releases, until you localized the release that introduces the problem.

[jaroslawp]

OK, thanks for info, will do that (will take some time .. need to discharge cars in mean time :-))

[jaroslawp]

Here is what I see when the communications error occur: on master:

D) (ModbusSend8)(C0) Sent packet(ModbusDecode)(C0) Received packet(ModbusDecode)(C0) Invalid modbus FC=04 packet
(D) (Timer100ms)(C0) ModbusRequest 3: Request MainsMeter Measurement
(D) (ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(MBhandleError)(C0) Error response: E0 - Timeout, address: 0a, function: 04, reg: 0000.
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Response
(D) (ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(Timer100ms)(C0) ModbusRequest 22: Request MainsMeter Import Active Energy Measurement
(D) (ModbusSend8)(C0) Sent packet(ModbusDecode)(C0) Received packet(ModbusDecode)(C0) Invalid modbus FC=04 packet
(D) (Timer100ms)(C0) ModbusRequest 3: Request MainsMeter Measurement
(D) (ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(MBhandleError)(C0) Error response: E0 - Timeout, address: 0a, function: 04, reg: 0000.
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Response
(D) (ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(Timer100ms)(C0) ModbusRequest 22: Request MainsMeter Import Active Energy Measurement

on slave:

loop)(C1) 0 clients running.
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Request
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Response
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Request
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Request
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Request
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Response
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Request
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Request
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0) Invalid modbus FC=04 packet
(D) (ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Request

I am not sure if this "FC=04 packet" message is correlated to the error, it also appears when no communication errors are reported .. the difference being that master reports while no errors this:

MBhandleError)(C0) Error response: E0 - Timeout, address: 09, function: 10, reg: 0020.
(D) (Timer100ms)(C0) ModbusRequest 3: Request MainsMeter Measurement
(D) (ModbusSend8)(C0) Sent packet(ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Response
(D) (ModbusSend8)(C0) Sent packet(ModbusDecode)(C0) Received packet(ModbusDecode)(C0)  Response
(D) (ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(ModbusSend8)(C0) Sent packet(Timer100ms)(C0) ModbusRequest 22: Request MainsMeter Import Active Energy Measurement
(D) (ModbusSend8)(C0) Sent packet(ModbusDecode)(C0) Received packet(ModbusDecode)(C0) Invalid modbus FC=04 packet
(D) (CalcBalancedCurrent)(C0) Balance: EVSE0:C(18.7A),EVSE1:B(18.7A),EVSE2:A(0.0A),EVSE3:A(0.0A),EVSE4:A(0.0A),EVSE5:A(0.0A),EVSE6:A(0.0A),EVSE7:A(0.0A),
(D) (ModbusWriteMultipleRequest)(C0) Sent packet(printStatus)(C0) STATE: C Error: 0 StartCurrent: -6 ChargeDelay: 0 SolarStopTimer: 0 NoCurrent: 0 Imeasured: 0.0 A IsetBalanced: 18.7 A
(I) (printStatus)(C0) L1: 0.0 A L2: 0.0 A L3: 0.0 A Isum: 0.0 A
(MBhandleError)(C0) Error response: E0 - Timeout, address: 09, function: 10, reg: 0020.

Looks like in error state the reply from Sensorbox for 'ModbusRequest 3: Request MainsMeter Measurement' is not received or processed ? logs are from 1.6.5, I have tried 1.6.2.1 / 1.6.3 and 1.6.4 versions - all show this error. Sometimes the communication error goes away by itself after few minutes, but few times I had to reboot both evses to get it cleared.

EV meter is set to disabled on both (and I think it should not appear in menu on the slave: MainsMeter is hidden when evse is set to slave as all settings are imposed by master). I will try 1.5.5 next.

[dingo35]

Ok great work; anxious to learn what release introduces the problem!

[jaroslawp]

1.5.5 seems to work correctly for me, I havent seen the error after multiple car (dis)connects, mode changes .. etc. therefore - would that be introduction of the TCP Modbus bridge which is causing the problem ? ( just looking at: https://github.com/serkri/SmartEVSE-3/blob/master/SmartEVSE-3/src/evse.cpp#L2679 - the 100ms timeout would be too short for processing master-slave comms after adding the bridge ? ... of course this is just guessing .. but: would it be possible to implement bridge setup as selectable on/off menu option, as for other features ?)

[dingo35]

It would surprise me if thats that problem; it was introduced in 1.6.3, so then 1.6.2.1 would work fine. Could you try 1.6.0 and 1.6.1 too? Upgrading of the libraries might be the problem.

Please focus on the error on the LCD screen, since the logging of errors on the modbus were introduced in v1.6.3; before that they were just ignored.. it doesnt mean they werent there!

[jaroslawp]

Ok, so: 1.6.0 works fine, 1.6.1 ends up with communications error after few minutes at max: not sure if this enough info ? (BTW: apparently the problem can also occur with only master powered on, while communicating with sensorbox, slave off at the time. so looks like modbus related ?) [ and additional observation: after downgrade from 1.6.1 to 1.6.0 I still got 'comm error': executing reboot from webui cleared it ]

[dingo35]

Ok there is one minor change in there that affects modbus traffic, it is commit a40885c65.

Are you able to compile yourself to test before/after this commit?

[jaroslawp]

Yes, I shall be able to, but please allow few days (I'm away from chargers until the weekend..) .. and I have seen the error with 1.6.0 once too (but this may have been effect of my actions...) : so I will run with 1.6.0 for few days to confirm first that this one works for me.

BTW: In case of 'bricking' the device is there a way to upload firmware using a cable ? (not sure if this https://www.smartevse.nl/building-and-programming , point 5 is still relevant?)

[dingo35]

Yes although I think it is nearly impossible to brick your device when you stick to commits in the repo, I have been known to disable the webserver in the past, so OTA flashing stopped working :-(

In that case it is very easy to upload a new version, to the right of the top connections there is a USB-C connector, if you edit platformio.ini to your device (e.g. /dev/ttyUSB0) you can run "pio run -t upload" from the command line. VScode must have some icon for that...

[dingo35]

Do not use those instructions for MPlab, it is for SmartEVSE2.

Just install python, do "pip install platformio" , cd to the SmartEVSE directory that has platform.io in it; "pio run" compiles firmware.bin, "pio run -t buildfs" compiles spiffs.bin . You might want to set DBG=1 to enable telnet debugging.

[jaroslawp]

I can confirm that this is indeed the offending change (https://github.com/serkri/SmartEVSE-3/commit/a40885c65b2dc3451b77c495731123ab2d89d0c5) : I reverted it in 1.6.5 and the problem seems to be gone, no more communication errors. Thanks for explanation about recovering the firmware (.. while not sticking to commits in the repo ... :-))

[dingo35]

Offending commit reverted: 21798e1