Connection to BMS lost, aggregate-driver crashes

[regu-tech]

Hello,

i have a problem with crashing with the driver. It runs 3-4 weeks without any problems.

But then it could happen that for a short moment the connection to one of my 4 BMS (JBD) is lost for a short moment. And then the driver crashes and also a reboot does not solve the problem. I then have to restart the driver with the restart script when the Cerbo is up again.

It shows me the following error in the log:

ERROR:root:Occured during step Read battery state, Battery SerialBattery(LLT/JBD). ERROR:root:Read trial nr. 1 Traceback (most recent call last): File "/data/dbus-aggregate-batteries/aggregatebatteries.py", line 842, in _update sum(VoltagesSum_dict.values()) / settings.NR_OF_BATTERIES TypeError: unsupported operand type(s) for +: 'int' and 'NoneType' Terminated

My actual version of the Venus OS is 3.41

best regards Steffen

[onemorename]

Same here, just as i check VRM Portal for new updates things start to go wrong - what you see at the end 15:32 is when i run /data/dbus-aggregate-batteries/restart.sh manually:

INFO:root: CVL: 54.0V, CCL: 280A, DCL: 280A INFO:root: Bat. voltage: 54.0V, Bat. current: 0A, SoC: 98.2%, Balancing state: 0 INFO:root: Min. cell voltage: Pack2_C5: 3.372V, Max. cell voltage: Pack2_C3: 3.379V, difference: 0.007V INFO:dbusmonitor:com.victronenergy.battery.ttyUSB1 disappeared from the dbus. Removing it from our lists INFO:dbusmonitor:com.victronenergy.battery.ttyUSB0 disappeared from the dbus. Removing it from our lists ERROR:root:Fri Sep 6 15:21:11 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 1 ERROR:root:Fri Sep 6 15:21:12 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 2 ERROR:root:Fri Sep 6 15:21:13 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 3 ERROR:root:Fri Sep 6 15:21:14 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 4 ERROR:root:Fri Sep 6 15:21:15 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 5 ERROR:root:Fri Sep 6 15:21:16 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 6 ERROR:root:Fri Sep 6 15:21:17 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 7 ERROR:root:Fri Sep 6 15:21:18 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 8 ERROR:root:Fri Sep 6 15:21:19 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 9 ERROR:root:Fri Sep 6 15:21:20 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 10 ERROR:root:Fri Sep 6 15:21:21 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 11 ERROR:root:Fri Sep 6 15:21:21 2024: DBus read failed. Exiting. INFO:root:Fri Sep 6 15:21:23 2024: Starting AggregateBatteries. INFO:root:registered ourselves on D-Bus as com.victronenergy.battery.aggregate INFO:root:Fri Sep 6 15:21:23 2024: Initial Ah read from file: 536Ah INFO:root:Fri Sep 6 15:21:24 2024: Starting battery monitor. INFO:root:Fri Sep 6 15:21:24 2024: Connected to DBus, and switching over to GLib.MainLoop() INFO:dbusmonitor:===== Search on dbus for services that we will monitor starting... ===== INFO:dbusmonitor:Found: com.victronenergy.settings, scanning and storing items INFO:dbusmonitor: com.victronenergy.settings has device instance 0 INFO:dbusmonitor:Found: com.victronenergy.system, scanning and storing items INFO:root:Fri Sep 6 15:21:25 2024: Searching Settings: Trial Nr. 1 INFO:dbusmonitor: com.victronenergy.system has device instance 0 INFO:dbusmonitor:Found: com.victronenergy.vebus.ttyS4, scanning and storing items INFO:root:Fri Sep 6 15:21:25 2024: com.victronenergy.settings found. INFO:dbusmonitor: com.victronenergy.vebus.ttyS4 has device instance 276 INFO:dbusmonitor:Found: com.victronenergy.battery.aggregate, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.aggregate has device instance 0 INFO:dbusmonitor:===== Search on dbus for services that we will monitor finished ===== INFO:root:Fri Sep 6 15:21:30 2024: Searching batteries: Trial Nr. 1 INFO:root:Fri Sep 6 15:21:30 2024: 0 batteries found. INFO:root:Fri Sep 6 15:21:35 2024: Searching batteries: Trial Nr. 2 INFO:root:Fri Sep 6 15:21:35 2024: 0 batteries found. INFO:root:Fri Sep 6 15:21:40 2024: Searching batteries: Trial Nr. 3 INFO:root:Fri Sep 6 15:21:40 2024: 0 batteries found. INFO:root:Fri Sep 6 15:21:45 2024: Searching batteries: Trial Nr. 4 INFO:root:Fri Sep 6 15:21:45 2024: 0 batteries found. INFO:root:Fri Sep 6 15:21:50 2024: Searching batteries: Trial Nr. 5 INFO:root:Fri Sep 6 15:21:50 2024: 0 batteries found. INFO:root:Fri Sep 6 15:21:55 2024: Searching batteries: Trial Nr. 6 INFO:root:Fri Sep 6 15:21:55 2024: 0 batteries found. INFO:root:Fri Sep 6 15:22:00 2024: Searching batteries: Trial Nr. 7 INFO:root:Fri Sep 6 15:22:00 2024: 0 batteries found. INFO:root:Fri Sep 6 15:22:05 2024: Searching batteries: Trial Nr. 8 INFO:root:Fri Sep 6 15:22:05 2024: 0 batteries found. INFO:root:Fri Sep 6 15:22:10 2024: Searching batteries: Trial Nr. 9 INFO:root:Fri Sep 6 15:22:10 2024: 0 batteries found. INFO:root:Fri Sep 6 15:22:15 2024: Searching batteries: Trial Nr. 10 INFO:root:Fri Sep 6 15:22:15 2024: 0 batteries found. INFO:root:Fri Sep 6 15:22:20 2024: Searching batteries: Trial Nr. 11 INFO:root:Fri Sep 6 15:22:20 2024: 0 batteries found. ERROR:root:Fri Sep 6 15:22:20 2024: Required number of batteries not found. Exiting. INFO:root:Fri Sep 6 15:22:22 2024: Starting AggregateBatteries. INFO:root:registered ourselves on D-Bus as com.victronenergy.battery.aggregate INFO:root:Fri Sep 6 15:22:22 2024: Initial Ah read from file: 536Ah INFO:root:Fri Sep 6 15:22:22 2024: Starting battery monitor. INFO:root:Fri Sep 6 15:22:22 2024: Connected to DBus, and switching over to GLib.MainLoop() INFO:dbusmonitor:===== Search on dbus for services that we will monitor starting... ===== INFO:dbusmonitor:Found: com.victronenergy.settings, scanning and storing items INFO:dbusmonitor: com.victronenergy.settings has device instance 0 INFO:dbusmonitor:Found: com.victronenergy.system, scanning and storing items INFO:dbusmonitor: com.victronenergy.system has device instance 0 INFO:dbusmonitor:Found: com.victronenergy.vebus.ttyS4, scanning and storing items INFO:root:Fri Sep 6 15:22:23 2024: Searching Settings: Trial Nr. 1 INFO:dbusmonitor: com.victronenergy.vebus.ttyS4 has device instance 276 INFO:root:Fri Sep 6 15:22:23 2024: com.victronenergy.settings found. INFO:dbusmonitor:Found: com.victronenergy.battery.aggregate, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.aggregate has device instance 0 INFO:dbusmonitor:===== Search on dbus for services that we will monitor finished ===== INFO:dbusmonitor:Found: com.victronenergy.battery.ttyUSB0, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.ttyUSB0 has device instance 5 INFO:root:Fri Sep 6 15:22:34 2024: Searching batteries: Trial Nr. 1 INFO:root:Fri Sep 6 15:22:34 2024: SerialBattery(Seplos) found, named as: Pack2. INFO:root:Fri Sep 6 15:22:34 2024: 1 batteries found. INFO:dbusmonitor:Found: com.victronenergy.battery.ttyUSB1, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.ttyUSB1 has device instance 1 INFO:root:Fri Sep 6 15:22:39 2024: Searching batteries: Trial Nr. 2 INFO:root:Fri Sep 6 15:22:39 2024: SerialBattery(Seplos) found, named as: Pack2. INFO:root:Fri Sep 6 15:22:39 2024: SerialBattery(Seplos) found, named as: Pack1. INFO:root:Fri Sep 6 15:22:39 2024: 2 batteries found. ERROR:root:Fri Sep 6 15:22:40 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 1 ERROR:root:Fri Sep 6 15:22:41 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read battery state, Battery Pack2. ERROR:root:Read trial nr. 2 ERROR:root:Fri Sep 6 15:22:42 2024: Error: unsupported operand type(s) for +=: 'float' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack1. ERROR:root:Read trial nr. 3 ERROR:root:Fri Sep 6 15:22:43 2024: Error: unsupported operand type(s) for +=: 'float' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack1. ERROR:root:Read trial nr. 4 Traceback (most recent call last): File "/data/dbus-aggregate-batteries/aggregatebatteries.py", line 842, in _update sum(VoltagesSum_dict.values()) / settings.NR_OF_BATTERIES TypeError: unsupported operand type(s) for +: 'float' and 'NoneType' Terminated INFO:root:Fri Sep 6 15:32:17 2024: Starting AggregateBatteries. INFO:root:registered ourselves on D-Bus as com.victronenergy.battery.aggregate INFO:root:Fri Sep 6 15:32:17 2024: Initial Ah read from file: 536Ah INFO:root:Fri Sep 6 15:32:17 2024: Starting battery monitor. INFO:root:Fri Sep 6 15:32:17 2024: Connected to DBus, and switching over to GLib.MainLoop() INFO:dbusmonitor:===== Search on dbus for services that we will monitor starting... ===== INFO:dbusmonitor:Found: com.victronenergy.settings, scanning and storing items INFO:root:Fri Sep 6 15:32:19 2024: Searching Settings: Trial Nr. 1 INFO:dbusmonitor: com.victronenergy.settings has device instance 0 INFO:root:Fri Sep 6 15:32:19 2024: com.victronenergy.settings found. INFO:dbusmonitor:Found: com.victronenergy.system, scanning and storing items INFO:dbusmonitor: com.victronenergy.system has device instance 0 INFO:dbusmonitor:Found: com.victronenergy.vebus.ttyS4, scanning and storing items INFO:dbusmonitor: com.victronenergy.vebus.ttyS4 has device instance 276 INFO:dbusmonitor:Found: com.victronenergy.battery.aggregate, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.aggregate has device instance 0 INFO:dbusmonitor:Found: com.victronenergy.battery.ttyUSB0, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.ttyUSB0 has device instance 5 INFO:dbusmonitor:Found: com.victronenergy.battery.ttyUSB1, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.ttyUSB1 has device instance 1 INFO:dbusmonitor:===== Search on dbus for services that we will monitor finished ===== INFO:root:Fri Sep 6 15:32:24 2024: Searching batteries: Trial Nr. 1 INFO:root:Fri Sep 6 15:32:24 2024: SerialBattery(Seplos) found, named as: Pack2. INFO:root:Fri Sep 6 15:32:24 2024: SerialBattery(Seplos) found, named as: Pack1. INFO:root:Fri Sep 6 15:32:24 2024: 2 batteries found.

[Dr-Gigavolt]

Hello guys, it is correct to crash and restart if a battery is disconnected or send an invalid value. Otherwise would aggregate batteries provide invalid charge parameters which is dangerous to the battery. Try to monitor the failing instance of the dubs serial battery in order to understand which value goes to none.

[regu-tech]

Hello, last night i had again a connection loss to one of my four battery packages but there the restart has worked properly.

[onemorename]

I do agree that it should crash, but why isn‘t it recovering by itself but after a restart only? I mean the data from serialbattery is the same before and after the crash

[Dr-Gigavolt]

If my battery crashes and restarts, the aggregate restarts as well. If there is an error message about incompatible operands 'int' and 'none', some battery instance, even found, doesn't send the value of the given quantity.

[Fred2938]

Hi, I do have exactly the same issue. Just install both dbus-serialbattery and dbus-aggregate-batteries. I do have 2 LFP pack (16cells). Both are visible on the Cerbo, with all details. So what kind of setup we should apply in serialbattery driver to ensure the aggregate-batteries driver works fine ? (ie in config.default.ini of serialbattery) Thank you

[Fred2938]

here is log of serialbattery.ttyUSB0 (other batterie is same in USB1): ![Uploading Capture d’écran 2024-09-09 210202.jpg…]()

[Dr-Gigavolt]

The JPG seems to be broken, please try again. And check with dbus-spy command if all batteries are visible on the dbus and which values do they send.

[Dr-Gigavolt]

The right guy who can help you with the serial-battery setup is @mr-manuel .

[mr-manuel]

If you see errors in the dbus-serialbattery open a issue in the repository: https://github.com/mr-manuel/venus-os_dbus-serialbattery/issues/new/choose

Please read carefully the instructions and don't skip anything.

[Fred2938]

Hi With the corrected link to the missing above jpg:

BTW, my BMS are both JK BMS B2A20S20P. Each battery are composed of 16 cells LFP 304Ah

[Dr-Gigavolt]

Your cerbo screenshots contain no relevant info. Please check the aggregate batteries log file and observe dbus by dbus-spy command. Find which path contains 'None' value. If this is the case, my program must crash and restart.

[Fred2938]

Hi, dbus-spy does not show any error message or line with 'None' value. See attached screenshots. I only share for 1 battery, the 2nd is fully aligned with similar details (well, its name change, ... ;) )

[Fred2938]

Hi,

Here is the /data/log/dbus-aggregate-batteries/current log:

[Fred2938]

Hi, and the Cerbo Gx screenshot

[Dr-Gigavolt]

Hi, now I see the problem. Your paths are not as usual. Your voltages:

My voltages:

The exception is raised in the 842, where the sum of "Voltages/Sum" from the lines 661 and 662 are calculated. But you have the path "Cell/Sum" instead. The value of not existing path "Voltages/Sum" is "None".

@mr-manuel : Have you renamed the paths in some update?

[mr-manuel]

No this is selectable since many years. Here is the commit from 2022.02.24 https://github.com/Louisvdw/dbus-serialbattery/commit/3cfff2d11b38f4b25e6b49621d5b7c09baa39aab

https://github.com/Louisvdw/dbus-serialbattery/blob/fd0fc1a193427df6ca0fadd141aa57826f761819/etc/dbus-serialbattery/config.default.ini#L342-L347

[Fred2938]

Hi, That is the issue and the solution ! As soon I change the BATTERY_CELL_DATA_FORMAT to 1, I get the dbus-aggregate_battery working on Cerbo !

Many thanks for your kind support and time ! Definitely really appreciate :)

Best regards

[onemorename]

But isn't that the default, so if it's not changed manually it should be 1 already? Cells have the correct format on my end, without any customization - and aggregate_battery works perfectly for me, unless i trigger firmware search from the VRM Portal. Serial Battery disconnects then, aggregate batteries has no values anymore and crashes as well. What happens then is, as serial battery reconnects, it has no values for 20-30 seconds until the connection is re-established fully again - and at this time aggregate batteries has given up already. I think key would be if serial battery would not post anything to dbus until it's fully up, or aggregatebatteries handles this situation better.

[onemorename]

Just did look dbus-spy while things happen. The Cell voltages have empty values for around 20-30 seconds. Aggregate batteries is in a dead end afterwards

[onemorename]

[Dr-Gigavolt]

Please read comments in settings.py. You can increase number of trials before the program gives up. But even given up, it should be restarted automatically by Linux.

[onemorename]

Please see my post with the log above:

it's exiting after trial 11 to search for batteries and then restarts. Here it finds the batteries then, however with still empty values. At the read thread it just goes down to retry 4 until it fails and doesn't recover from there anymore (even while the settings.py is still set to 10 read retries per default, it doesn't seem to make it)

INFO:dbusmonitor:Found: com.victronenergy.battery.ttyUSB1, scanning and storing items INFO:dbusmonitor: com.victronenergy.battery.ttyUSB1 has device instance 1 INFO:root:Fri Sep 6 15:22:39 2024: Searching batteries: Trial Nr. 2 INFO:root:Fri Sep 6 15:22:39 2024: SerialBattery(Seplos) found, named as: Pack2. INFO:root:Fri Sep 6 15:22:39 2024: SerialBattery(Seplos) found, named as: Pack1. INFO:root:Fri Sep 6 15:22:39 2024: 2 batteries found. ERROR:root:Fri Sep 6 15:22:40 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack2. ERROR:root:Read trial nr. 1 ERROR:root:Fri Sep 6 15:22:41 2024: Error: unsupported operand type(s) for +=: 'int' and 'NoneType'. ERROR:root:Occured during step Read battery state, Battery Pack2. ERROR:root:Read trial nr. 2 ERROR:root:Fri Sep 6 15:22:42 2024: Error: unsupported operand type(s) for +=: 'float' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack1. ERROR:root:Read trial nr. 3 ERROR:root:Fri Sep 6 15:22:43 2024: Error: unsupported operand type(s) for +=: 'float' and 'NoneType'. ERROR:root:Occured during step Read V, I, P, Battery Pack1. ERROR:root:Read trial nr. 4 Traceback (most recent call last): File "/data/dbus-aggregate-batteries/aggregatebatteries.py", line 842, in _update sum(VoltagesSum_dict.values()) / settings.NR_OF_BATTERIES TypeError: unsupported operand type(s) for +: 'float' and 'NoneType' Terminated

[Fred2938]

Hi, On my side, I were facing 2 situations: the SW issue as reported above and a poor quality RS485-USB + USB dongle solution which create lot of disconnections. I solved the HW link 2 days ago after upgrading my HW with a robust and galvanic isolated interface between batteries and Cerbo.

Then, based on your replies, I made a quick trial with BATTERY_CELL_DATA_FORMAT to 1 (I have set it to 2). and aggregate-battery driver lived again. But, as I made here and there other changes in settings, being a bit too curious, I uninstall / install the 2 drivers to recover the default settings and rely on a liable setup.

So far, about 24h run without disconnection or alarm :)