The driver is non loaded after update.

cokyc commented 1 year ago

Describe the bug

I just update from version v1.0.20230508beta to the last beta, rebooted and the driver is not loaded.

I then tried to start the driver manually with "start-serialbattery.sh" and here is the output:

UTC-2023.05.28-12:44:58` Starting dbus-serialbattery.py on
INFO:SerialBattery:
INFO:SerialBattery:Starting dbus-serialbattery
INFO:SerialBattery:dbus-serialbattery v1.0.0 (20230508)
INFO:SerialBattery:Testing Daly
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:Unexpected err=AttributeError('__enter__'), type(err)=<class 'AttributeError'>
INFO:SerialBattery:Testing Daly
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:Unexpected err=AttributeError('__enter__'), type(err)=<class 'AttributeError'>
INFO:SerialBattery:Testing Daly
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:Unexpected err=AttributeError('__enter__'), type(err)=<class 'AttributeError'>
INFO:SerialBattery:Testing Daly
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:Unexpected err=AttributeError('__enter__'), type(err)=<class 'AttributeError'>
INFO:SerialBattery:Testing Daly
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:Unexpected err=AttributeError('__enter__'), type(err)=<class 'AttributeError'>
INFO:SerialBattery:Testing Daly
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:[Errno 21] could not open port /dev/: [Errno 21] Is a directory: '/dev/'
ERROR:SerialBattery:Unexpected err=AttributeError('__enter__'), type(err)=<class 'AttributeError'>
ERROR:SerialBattery:ERROR >>> No battery connection at /dev/
rm: '/var/lock/serial-starter' is a directory

How to reproduce

Steps to reproduce the behavior:

wget -O /tmp/install.sh https://raw.githubusercontent.com/Louisvdw/dbus-serialbattery/master/etc/dbus-serialbattery/install.sh
bash /tmp/install.sh
choose option 3
choose option master
reboot

Expected behavior

The driver loaded and showed in the console

Driver version

v1.0.20230508beta

Venus OS device type

Raspberry Pi

Venus OS version

v3.00-42

BMS type

Daly Smart BMS

Cell count

16

Connection type

Serial USB adapter to RS485

Config file

config.ini:
[DEFAULT]

; If you want to add custom settings, then check the settings you want to change in "config.default.ini"
; and add them below to persist future driver updates.
; --------- Battery Current limits ---------
MAX_BATTERY_CHARGE_CURRENT    = 140.0
MAX_BATTERY_DISCHARGE_CURRENT = 280.0

; Set steps to reduce battery current
; The current will be changed linear between those steps if LINEAR_LIMITATION_ENABLE is set to True
CELL_VOLTAGES_WHILE_CHARGING = 3.56,  3.48,  3.45,  3.30
MAX_CHARGE_CURRENT_CV_FRACTION =  0,  0.02,   0.05,     1

CELL_VOLTAGES_WHILE_DISCHARGING =   2.80,  2.90,  3.05,  3.10
MAX_DISCHARGE_CURRENT_CV_FRACTION =    0,   0.1,   0.5,     1

; Set Steps to reduce battery current. The current will be changed linear between those steps
TEMPERATURE_LIMITS_WHILE_CHARGING = 0,   2,   5,   7,  10,  15,  20,  35,  40,  55
MAX_CHARGE_CURRENT_T_FRACTION =     0, 0.1, 0.3, 0.5, 0.8,   1,   1,   1, 0.6,   0

TEMPERATURE_LIMITS_WHILE_DISCHARGING = -20,   0,   5,   10,   15,   45,  55
MAX_DISCHARGE_CURRENT_T_FRACTION =       0,  .2,  .3,   .6,    1,    1,   0

; --------- SOC limitation (affecting CCL/DCL) ---------
; Description: Maximal charge / discharge current will be increased / decreased depending on State of Charge,
;              see CC_SOC_LIMIT1 etc.
; Example: The SoC limit will be monitored to control the currents.
; Charge current control management enable (True/False).
CCCM_SOC_ENABLE = False
; Discharge current control management enable (True/False).
DCCM_SOC_ENABLE = False

; --------- Additional settings ---------
; Specify only one BMS type to load else leave empty to try to load all availabe
; LltJbd, Ant, Daly, Daly, Jkbms, Lifepower, Renogy, Renogy, Ecs
BMS_TYPE = Daly

; -- Daly settings
; Battery capacity (amps) if the BMS does not support reading it
BATTERY_CAPACITY = 280
; Invert Battery Current. Default non-inverted. Set to -1 to invert
INVERT_CURRENT_MEASUREMENT = -1

Relevant log output

/var/log/dbus-serialbattery.ttyUSB0/current has no log

root@raspberrypi4:/data/etc/dbus-serialbattery# tail -f /var/log/serial-starter/current
@40000000647349f915464fdc INFO: loading config file /data/conf/serial-starter.d/dbus-serialbattery.conf
@40000000647349fb0d84cecc INFO: Create daemontools service dbus-cgwacs.ttyUSB0
@4000000064734a01170d2174 INFO: Start service dbus-cgwacs.ttyUSB0 once
@4000000064734b6e15800164 *** CCGX booted (0) ***
@4000000064734b6e2fb5c0b4 *** starting serial-starter ***
@4000000064734b6f10f0b544 serstart starting
@4000000064734b6f1133d8c4 INFO: loading config file /etc/venus/serial-starter.conf
@4000000064734b700aebb914 INFO: loading config file /data/conf/serial-starter.d/dbus-serialbattery.conf
@4000000064734b713ac53ffc INFO: Create daemontools service dbus-cgwacs.ttyUSB0
@4000000064734b780c6f578c INFO: Start service dbus-cgwacs.ttyUSB0 once

Any other information that may be helpful

The previous installation has been done from scratch for both venus os and the serial driver and all worked well.

mr-manuel commented 1 year ago

Please install from dev branch. The master is not updated yet and contains a small error.

cokyc commented 1 year ago

Reinstalled from dev branch. Same result.

mr-manuel commented 1 year ago

Please post the output of this commands:

cat /data/rc.local

ls -la /service

ls -l /data/etc/dbus-serialbattery

cokyc commented 1 year ago

here you are.

cat /data/rc.local

#!/bin/bash
bash /data/etc/dbus-serialbattery/reinstall-local.sh

ls -la /service

drwxr-xr-x   30 root     root           700 May 28 17:10 .
drwxr-xr-x   21 root     root          4096 May 14 09:22 ..
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-adc
drwxr-xr-x    4 root     root           120 May 28 17:10 dbus-ble-sensors
lrwxrwxrwx    1 root     root            42 May 28 17:10 dbus-cgwacs.ttyUSB0 -> /var/volatile/services/dbus-cgwacs.ttyUSB0
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-digitalinputs
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-fronius
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-generator-starter
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-modbus-client
drwxr-xr-x    4 root     root           120 May 28 17:10 dbus-pump
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-qwacs
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-shelly
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-systemcalc-py
drwxr-xr-x    4 root     root           120 May 28 17:10 dbus-tempsensor-relay
drwxr-xr-x    4 root     root           100 May 28 17:10 dbus-vebus-to-pvinverter
drwxr-xr-x    4 root     root           100 May 28 17:10 gui
drwxr-xr-x    4 root     root           100 May 28 17:10 hub4control
drwxr-xr-x    4 root     root           100 May 28 17:10 llmnrd
drwxr-xr-x    4 root     root           100 May 28 17:10 localsettings
drwxr-xr-x    4 root     root           100 May 28 17:10 netmon
drwxr-xr-x    4 root     root           100 May 28 17:10 nginx
drwxr-xr-x    4 root     root           120 May 28 17:10 node-red-venus
lrwxrwxrwx    1 root     root            20 May 28 17:10 openssh -> /run/service/openssh
drwxr-xr-x    4 root     root           120 May 28 17:10 ppp
drwxr-xr-x    4 root     root           100 May 28 17:10 serial-starter
drwxr-xr-x    4 root     root           100 May 28 17:10 service-advertiser
drwxr-xr-x    4 root     root           120 May 28 17:10 signalk-server
drwxr-xr-x    4 root     root           100 May 28 17:10 simple-upnpd
lrwxrwxrwx    1 root     root            23 May 28 17:10 ssh-tunnel -> /run/service/ssh-tunnel
drwxr-xr-x    4 root     root           100 May 28 17:10 venus-access
drwxr-xr-x    4 root     root           100 May 28 17:10 venus-html5-logger
drwxr-xr-x    4 root     root           100 May 28 17:10 venus-platform
lrwxrwxrwx    1 root     root            27 May 28 17:10 vesmart-server -> /run/service/vesmart-server
drwxr-xr-x    4 root     root           100 May 28 17:10 vrmlogger
lrwxrwxrwx    1 root     root            25 May 28 17:10 websockify-c -> /run/service/websockify-c

ls -l /data/etc/dbus-serialbattery

-rw-r--r--    1 root     root          1075 May 28 16:59 LICENSE
-rw-r--r--    1 root     root           204 May 28 16:59 README.md
-rwxr-xr-x    1 root     root         36746 May 28 16:59 battery.py
drwxr-xr-x    2 root     root          4096 May 28 16:59 bms
-rw-r--r--    1 root     root         11654 May 28 16:59 config.default.ini
-rw-r--r--    1 root     root          1938 May 13 17:50 config.ini
-rwxr-xr-x    1 root     root          5247 May 28 16:59 dbus-serialbattery.py
-rwxr-xr-x    1 root     root         26814 May 28 16:59 dbushelper.py
-rwxr-xr-x    1 root     root          1354 May 28 16:59 disable.sh
-rwxr-xr-x    1 root     root          3990 May 28 16:59 install-qml.sh
-rwxr-xr-x    1 root     root          4889 May 28 16:59 install.sh
-rwxr-xr-x    1 root     root        137100 May 28 16:59 minimalmodbus.py
drwxr-xr-x    2 root     root          4096 May 28 16:59 qml
-rwxr-xr-x    1 root     root          9146 May 28 16:59 reinstall-local.sh
-rwxr-xr-x    1 root     root           827 May 28 16:59 restart-driver.sh
-rwxr-xr-x    1 root     root          1185 May 28 16:59 restore-gui.sh
drwxr-xr-x    3 root     root          4096 May 28 16:59 service
-rwxr-xr-x    1 root     root           282 May 28 16:59 start-serialbattery.sh
-rwxr-xr-x    1 root     root          1269 May 28 16:59 uninstall.sh
-rwxr-xr-x    1 root     root         20902 May 28 16:59 utils.py

mr-manuel commented 1 year ago

You did already reboot the system, right?

Can you write me on Discord?

cokyc commented 1 year ago

Ok, can you tell me your discord username? My user is cokyc#7625

thx.

Il giorno lun 29 mag 2023 alle ore 12:30 Manuel @.***> ha scritto:

You did already reboot the system, right?

Can you write me on Discord?

— Reply to this email directly, view it on GitHub https://github.com/Louisvdw/dbus-serialbattery/issues/675#issuecomment-1566935392, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOTPXKYIE5I7ERTK3WA2O3TXIR3EXANCNFSM6AAAAAAYR2XZVI . You are receiving this because you authored the thread.Message ID: @.***>

mr-manuel commented 1 year ago

Somehow the serial starter is starting dbus-cgwacs against ttyUSB0 and blocking other drivers from starting.

@4000000064734b6e15800164 *** CCGX booted (0) ***
@4000000064734b6e2fb5c0b4 *** starting serial-starter ***
@4000000064734b6f10f0b544 serstart starting
@4000000064734b6f1133d8c4 INFO: loading config file /etc/venus/serial-starter.conf
@4000000064734b700aebb914 INFO: loading config file /data/conf/serial-starter.d/dbus-serialbattery.conf
@4000000064734b713ac53ffc INFO: Create daemontools service dbus-cgwacs.ttyUSB0
@4000000064734b780c6f578c INFO: Start service dbus-cgwacs.ttyUSB0 once

@Louisvdw maybe you know how to solve this?

Louisvdw commented 1 year ago

At the moment there is not a permanent way that we can solve this. The built in drivers is always tested before the serialbattery driver and if any of them get a responce that think is correct that driver will attached to the device.

There is a ways to get around this. VenusOS's serial-starter remembers what driver was last successful for each port, so it will rescan that driver first, and if that fail it will go through it normal list. You can trick your port to start with the serialbattery driver. If something change and it rescan it will however override your trick.

@cokyc you can try:

In SSH you can run echo "sbattery" > /data/var/lib/serial-starter/ttyUSB# where # is your USB port number. So if I want for force it for port 1 it would be echo "sbattery" > /data/var/lib/serial-starter/ttyUSB1

After this reboot your GX or just plug in your battery connection again.

cokyc commented 1 year ago

Thanks Louis. Unfortunately it doesn't work. Only if I disconnect and then reconnect the USB cable does the driver start.

But if I restart the rpi, the driver doesn't start.

here is the serial-starter log if it can help

2023-05-31 21:35:44.090356500 INFO: Start service dbus-serialbattery.ttyUSB0 once
2023-05-31 21:57:16.474098500  CCGX booted (0) 
2023-05-31 21:57:17.002169500  starting serial-starter 
2023-05-31 21:57:17.467468500 serstart starting
2023-05-31 21:57:17.475356500 INFO: loading config file /etc/venus/serial-starter.conf
2023-05-31 21:57:18.246204500 INFO: loading config file /data/conf/serial-starter.d/dbus-serialbattery.conf
2023-05-31 21:57:20.193299500 INFO: Create daemontools service dbus-serialbattery.ttyUSB0
2023-05-31 21:57:46.775345500 INFO: Start service dbus-serialbattery.ttyUSB0 once
2023-05-31 21:57:49.955948500 INFO: Create daemontools service dbus-cgwacs.ttyUSB0
2023-05-31 21:57:56.009487500 INFO: Start service dbus-cgwacs.ttyUSB0 once

transistorgit commented 1 year ago

@Louisvdw there is another and permanent solution, but it is not so easy for the customer.

In my last company, we built measurement systems for vessel/offshore craft performance monitoring. As we redesigned for cost sensitive customers, we changed from industrial interfaces to USB, so we had to cope with a lot USB Devices, especially multiple identical serial converters for NMEA, Modbus etc.

We solved it by making udev rules for each physical USB port and set up a schematic and labeling. Each device had it's fixed Port then, like in the good old times before USB.

Of course, here we have to support an unknown zoo of host devices. So for easy deployment, you have to implement a piece by piece setup procedure: Unplug everything, then plug in 1 device at a time device and let the user select the correct device type. Store device type together with USB path. The user then should lable the port appropriately.

Not plug and play, but however my opinion is that these types of devices (batteries) are potential harmfull and should not be build by noobs.

brgds

Louisvdw commented 1 year ago

If there is a work around then perhaps we can try it for those that need it. Most users should not require it. @transistorgit do you want to try and assist cokyc to see if he can implement it on his side and then we can write up the procedure?

mr-manuel commented 1 year ago

See https://github.com/Louisvdw/dbus-serialbattery/discussions/481#discussioncomment-5276084, https://github.com/Louisvdw/dbus-serialbattery/discussions/481#discussioncomment-5480101 and https://github.com/Louisvdw/dbus-serialbattery/issues/636#issuecomment-1547327584.

Is this like @transistorgit would do it?

transistorgit commented 1 year ago

no, that's "the usual" way, that is not working here. Problem is, that you can distinguish the serial converters only by type (most (not all) manufacturers are too stingy to give it unique IDs). As you often have the same converter types, you can't build up a decent system that survives replugging, changing broken converters etc.

what I suggest is to map the USB port device path to fixed device names. We did that on a commercial project and it worked well.

On my venus raspi 3b+:

root@venus:~# udevadm info -q path /dev/ttyUSB1
/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.2/1-1.2.2:1.0/ttyUSB1/tty/ttyUSB1

The middle part with the numbers and dashed is the path over the system's internal and external USB hubs. You can set an udev rule like "if a device is plugged into the physical port "..../1-1/1-1.2/1-1.2.2/1-1.2.2", give it the device name "/dev/ttyUSB_upper_left" or "/dev/Gridmeter_Port". And label your ports accordingly with a sticker.

So no matter which type of converter is used, it will always get the same device name when plugged into the same port.

I will have to go into it to generate a detailed writeup, though. I did it quite a while ago. The additional challenge here is, that we don't have a nice fixed hardware but a lot different device types that the people use as venus platform. So we have to develop some teach-in setup procedure: plug in one device at a time, probe that, store the USB path, configure udev rule and then do the next... don't know if the users would like that.

what do you think, would it be worth the effort? how many people need this?

mr-manuel commented 1 year ago

I think that would help the users a lot and solve also:

VRM ID change
MQTT battery ID change
Node-RED ID change
Make Custom Name persistant, since the ID does not change

cokyc commented 1 year ago

I know what transistorgit is suggesting. I use this technique on an RPI as bridge for an ET340 meter and an SDM230 (simulated) using 2 pieces of usb to rs485 adapter of the same type. Both adapters have the same identification, so the usual way doesn't work. I have seen the usb path that each adapter (ls /dev/serial/by-path) have and then I have passed the dev path (es. /dev/serial/by-path/platform-fd500000.pcie-pci-0000:01:00.0-usb-0:1.1:1.0-port0) instead of the usual /dev/ttyUSBx in the command line. And yes the position of the usb port has to be fixed.

Are we sure that the udev rules take control of the custom usb first so when serial-starter starts the other victron drivers (in the standard way) drop the busy port and assign the next usb number?

However, I saw that this happens when the rpi has only one usb adapter plugged in (the one for the BMS). If I connect also the MK3 cable to the inverter all the drivers are loaded correctly.

mr-manuel commented 1 year ago

I added some infos from Louisvdw in the documentation.

mr-manuel commented 1 year ago

The problem was solved following the new documentation and "resetting" the saved values of the serial starter by executing:

rm /data/var/lib/serial-starter/*

and then reboot.

transistorgit commented 1 year ago

regarding udev rules @mr-manuel and I discussed some possible solutions but found all too complex, too inflexible or too difficult for the user. So for now, we propose that the drivers should improved so that the recognise the device correctly and don't attach to unknown devices. mr-manuel also implemented an additional second-level check in the dbus-serialbattery.py file.

We can't address the problem, that cgwacs grabs a port beforehand, that would only be feasible with udev rules or changes in victron code, though

Louisvdw / dbus-serialbattery