Hilbrand
commented
3 years ago Did you try assigning the serial ports a name as described in the serial configuration documentation page. Because after reboot serial ports are kind of randomly assigned.
Open t2000 opened 3 years ago
Hilbrand
commented
3 years ago Did you try assigning the serial ports a name as described in the serial configuration documentation page. Because after reboot serial ports are kind of randomly assigned.
t2000
commented
3 years ago so with all aliases and all device names, also in one attempt just the device names and once just the aliases. In the end I used: -Dgnu.io.rxtx.SerialPorts=/dev/heatpump:/dev/finedust:/dev/energyhome:/dev/energyheat:/dev/ttyUSB0:/dev/ttyUSB1:/dev/ttyUSB2:/dev/ttyUSB3
So, yes I also used these names as configuration values in my Things of course. It is a matter of luck whether the port even EXISTS inside of the SerialPortManager of openHAB, even though the port exists in the OS all the time and the device is never disconnected! The ports just vanish. Sometimes after some connects/disconnects but also randomly over time :(
I decided to switch to ser2net with RFC2217 and forward the ports to the network and use the RFC2217 implementation in openHAB. However, I found several issues there as well :( I will post some more bugs soon.
splatch
commented
3 years ago In order to solve issue in reliable way I think we need another thought on how these work. So far Linux did not come with persistent port identifiers for Linux and all of serial connections we have rely on port identifiers.
My though on it is trying to add a logic within SerialPortManager to identify actual USB device. Each and every USB device has few coordinates:
Based on that we could assign serial connection to specific device and not port identifier. With this logic reconnecting of USB dongles wouldn't cause any troubles any more.
I know it sounds simple but it isn't. It requires further investigation. It doesn't solve issue of multiple devices having the same coordinates (and we know this happens) but it shifts the issue into area where it can be controlled by OH and not mixture of udev rules and pure randomnesses caused by async startup of operating system kernel.
I know we have serial ports coming from other places than USB (ie. onboard ports ttyS0 / ttySC0 from hats) but actual problem in this issue are virtual comm ports coming over USB.
t2000
commented
3 years ago So far Linux did not come with persistent port identifiers for Linux and all of serial connections we have rely on port identifiers. Right, which is why users like me use UDEV rules that create symlinks to named devices based on the IDs you mentioned. So this way this should stay stable already.
Based on that we could assign serial connection to specific device and not port identifier. With this logic reconnecting of USB dongles wouldn't cause any troubles any more.
Not sure if this is the issue if you have UDEV rules in place already.
What I observe is that CommPortIdentifier.getPortIdentifiers() simply forgets whatever was there already: Startup openHAB with some USB devices (in my case 4) and if you are lucky you have 4 named devices and CommPortIdentifier.getPortIdentifiers() returns all of them, so you can use them. However, depending on the binding, it tries to show the available ports to the user in an error message and thus calls CommPortIdentifier.getPortIdentifiers() by itself. This alone MIGHT make the system "forget" the port.
If you are lucky and the port still exists after the bindings are running: Disable a thing, which should close the port while shutting down the ThingHandler. afterwards chances are VERY low that you are able to enable the thing again, because the CommPortIdentifier.getPortIdentifiers() will not contain your port anymore until you restart the whole runtime.
openhab-bot
commented
3 years ago This issue has been mentioned on openHAB Community. There might be relevant details there:
https://community.openhab.org/t/modbus-stops-without-any-errors/104050/73
dschall
commented
3 years ago I am having similar issues with OH3.0.2 when using both a Z-Wave and a Zigbee USB stick. Only one of the two can successfully open the serial port, and the other does no longer find the other port. I am not even using docker or tty symlinks, just plain old /dev/ttyUSB0 and /dev/ttyACM0, and still the ports randomly disappear.
Is there an update/fix for this issue yet?
lolodomo
commented
3 years ago I don't think I already encountered such issue with a RPI. Maybe an issue in nrjavaserial only with few Linux OS?
t2000
commented
3 years ago I am 99.99999999% sure that the issue is here: https://github.com/NeuronRobotics/nrjavaserial/blob/5d48a20bac48826f4e3741b6b3b2db3e2b298bc4/src/main/java/gnu/io/CommPortIdentifier.java#L334
Deep down in the stack it calls https://github.com/NeuronRobotics/nrjavaserial/blob/5d48a20bac48826f4e3741b6b3b2db3e2b298bc4/src/main/java/gnu/io/RXTXCommDriver.java#L437
which is a native call that depends on the current state of the OS and all the locking etc. Sometimes the port is just "busy" and thus it will "disappear" from the nrjavaserial perspective.
There is also no way to work around this, because https://github.com/NeuronRobotics/nrjavaserial/blob/5d48a20bac48826f4e3741b6b3b2db3e2b298bc4/src/main/java/gnu/io/CommPortIdentifier.java#L313 is ALWAYS called, even if you try to access a specific port and you don't want to scan for other ports.
One option for openHAB would be to implement a new serial transport bundle using https://fazecast.github.io/jSerialComm/ and see if that works better.
Meanwhile I decouple the serial ports from openHAB by passing them through ser2net which works somehow more reliable.
dschall
commented
3 years ago Meanwhile I decouple the serial ports from openHAB by passing them through ser2net which works somehow more reliable.
@t2000, can you elaborate on that, please?
I've seen people use rfc2217, but that doesn't seem to work with OH3 (specifically the zwave and zigbee bindings) any more..
dschall
commented
3 years ago Is there an update on this, or workaround?
splatch
commented
3 years ago @dschall I believe there is a way to emulate udev rules without involving udev itself. I began working on some prototype SerialPortProvider which does that.
openhab-bot
commented
3 years ago This issue has been mentioned on openHAB Community. There might be relevant details there:
t2000
commented
3 years ago I am afraid to disappoint you... The workaround mentioned here https://community.openhab.org/t/co7io-persistent-serial-port-identifier-provider-for-openhab/124223 will NOT work.
The issue is within the native part of the serial port library that is used by openHAB, i.e. in the native part of nrjavaserial.
Hence, I recommend to give this a try: https://fazecast.github.io/jSerialComm/ see my previous comment https://github.com/openhab/openhab-core/issues/2119#issuecomment-841815402
You can define as much udev rules/hashes/what not on the OS layer as you want, but it won't fix the issue. You will still get the error that the port is missing in openHAB, although on the OS side it clearly exists and also doesn't change its name.
splatch
commented
3 years ago I am afraid to disappoint you... The workaround mentioned here https://community.openhab.org/t/co7io-persistent-serial-port-identifier-provider-for-openhab/124223 will NOT work.
It will definitely not solve native quirks of rxtx which are beyond the reach. Mentioned solution offers a way to identify USB devices and rely on their identity rather assigned serial port name. For situations where port is emulated anyway (ie. docker) or not bound to debian/fedora installations which nrjavaserial attempts to handle I made a serial port provider based on a plain java library called https://github.com/nyholku/purejavacomm. It has no native parts. Native access for Linux/Windows/FreeBSD is provided through JNA.
I implemented a serial provider compatible with openHAB 3.x which works for me in places where rxtx can't. Please download it and test if you wish. You will need openhab-runtime-jna installed. This serial provider ships purejavacomm 1.0.3 aligned with JNA 5.4.0 which is used in OH 3.x.
openhab-bot
commented
3 years ago This issue has been mentioned on openHAB Community. There might be relevant details there:
https://community.openhab.org/t/enocean-usb-300-gateway-config-error/126352/10
elordude
commented
3 years ago Is there a solution for this yet? I have 4 serial ports and I need to restart OpenHab multiple times with small modification to /etc/default/openhab until all ports are recognized. I try different combinations of the line below, changing the order of the ports until everything is recognized, it is not an ideal situation.
-Dgnu.io.rxtx.SerialPorts=/dev/ttyZWAVE:/dev/ttyZIGB:/dev/ttyCADDX:/dev/ttyINSTEON" -Dgnu.io.rxtx.SerialPorts=/dev/ttyUSB0:/dev/ttyZWAVE:/dev/ttyUSB3:/dev/ttyZIGB:/dev/ttyUSB2:/dev/ttyCADDX:/dev/ttyUSB1:/dev/ttyINSTEON
What is interesting is I never had a problem with this while running on a Protectli Vault FW4B Intel® J3160, I recently moved my OH3 to a Intel NUC NUC6i7KYK and all my serial problem started with that hardware change. Is there a specific chipset or bios setting that might be causing this?
t2000
commented
2 years ago I implemented a serial provider compatible with openHAB 3.x which works for me in places where rxtx can't. Please download it and test if you wish. You will need openhab-runtime-jna installed. This serial provider ships purejavacomm 1.0.3 aligned with JNA 5.4.0 which is used in OH 3.x.
@splatch Since the link s broken: Do you have a link to a branch that contains your implementation based on purejavacomm?
How about creating an official merge request? I would like to give it a try to get rid of my ser2net and RFC2217 hack to shrink my setup and configuration/maintenance effort.
splatch
commented
2 years ago I implemented a serial provider compatible with openHAB 3.x which works for me in places where rxtx can't. Please download it and test if you wish. You will need openhab-runtime-jna installed. This serial provider ships purejavacomm 1.0.3 aligned with JNA 5.4.0 which is used in OH 3.x.
@splatch Since the link s broken: Do you have a link to a branch that contains your implementation based on purejavacomm?
How about creating an official merge request? I would like to give it a try to get rid of my ser2net and RFC2217 hack to shrink my setup and configuration/maintenance effort.
I believe link still works. In case if you experience troubles you can go directly most recent sources (4Q2021) org.connectorio.addons.io.transport.serial.purejavacomm - the most recent change I did was migration to purejavacomm 1.0.5 binary published by @cdjackson under opensmarthouse coordinates.
The purejavacomm has no support for RFC2217, If you wish to create your own PR feel free to do such, code to provide is fairly basic as PJC and rxtx apis are twins.
t2000
commented
2 years ago @splatch Thanks for the new link to your branch. I see some custom package names in there from your project. Is there any specific reason that you only made it for your project?
Are you planning to make this an official addon for openHAB and get it merged? May others take your code and contribute it to openHAB, if you don't want to take care of opening a PR by yourself?
splatch
commented
2 years ago @splatch Thanks for the new link to your branch. I see some custom package names in there from your project. Is there any specific reason that you only made it for your project?
I made this for own purposes (I run & manage multiple assemblies together with an embedded build), which lead me to troubles with existing serial port provider based on nrjavaserial. Easiest solution to me was dropping it in favor of purejavacomm which made me free of a) native dependency and b) unreliable locking logic, which require c) specific linux flavor. I have the same trouble with arch linux which I use for development purposes.
Are you planning to make this an official addon for openHAB and get it merged? May others take your code and contribute it to openHAB, if you don't want to take care of opening a PR by yourself?
I don't have a plan to contribute it into OH, that's reason why I put these into separate repository. I have no issues If someone takes code I published and do PRs with it (for example #2617). The PJC provider is already part of opensmarthouse distro since a bit which is EPLv2. The osh 0.10 binaries are compatible with OH 3.1+ so you can utilize it through maven central without any troubles.
J-N-K
commented
2 years ago Sounds reasonable. Could provide a PR for that, @t2000?
I am now trying to migrate from openHAB 2 to openHAB 3 since almost one week, but my serial devices prevent me from doing that.
During my investigations I found that the bug(s) that I am facing do not neccesarily have to do with OH3 but might have existed in OH2 as well.
My setup:
2 smartmeter things 1 novafinedust thing 1 (custom binding for stiebelheatpump currently under testing here: https://github.com/t2000/openhab2-addons/tree/stiebelheatpumpOH3TecalorVentilation)
My devices have aliases in my Arch Linux system: /dev/heatpump, /dev/finedust, /dev/energyheat, and /dev/energyhome which are symbolic links to the corresponding /dev/ttyUSBX devices, which have group r+w permissions for the openhab user. The openhab user is also allowed to create its lock files in
/run/lock, starting withLCK... (I though that these lock files were removed at some point, because newer linux distributions do not support it anymore, but well its still there).In OH2 I only had the novafinedust and the stiebelheatpump thing running in a docker container where I forwared them via:
and inside docker I used the
/dev/ttyUSB0and/dev/ttyUSB1in my things, because I knew which one is which, thanks to the docker mapping.I always had trouble that I needed multiple starts of openHAB or retries of disabling/enabling my things in order for them to close and reopen the serial port until they came up and running.
So far I blamed docker for this and was looking forward to install OH3 without docker to get rid of this problem. However, the problem still persists and my 2 new devices make it even worse!
Its a matter of luck which
Thinghandleris able to open a serial port and use it.At first I configured my things with the alias names instead of
ttyUSBX, buter later I used the device names directly, but it didn't make much of a difference as it seems.I started debugging this a little bit and found the environment setting
-Dgnu.io.rxtx.SerialPortsand I filled this in various attemps with all names, that I had, so with all aliases and all device names, also in one attempt just the device names and once just the aliases. In the end I used:-Dgnu.io.rxtx.SerialPorts=/dev/heatpump:/dev/finedust:/dev/energyhome:/dev/energyheat:/dev/ttyUSB0:/dev/ttyUSB1:/dev/ttyUSB2:/dev/ttyUSB3.I stumbled across the term
rxtxand thought that it was replaced bynrjavaserial, but the bundle list shows them both:Looking into the source I see that
RxTxSerialPortpoints tognu.io, which I thought isrxtx. However my Eclipse navigated me into thenrjavaserialbundle which now contains thisgnu.iopackage.https://github.com/openhab/openhab-core/blob/987af36bcf08d0904897605e2b93a59133c42aa9/bundles/org.openhab.core.io.transport.serial.rxtx/src/main/java/org/openhab/core/io/transport/serial/rxtx/RxTxSerialPort.java#L27
In the source code there is also the
org.openhab.core.io.transport.serial.javacommbundle which usesjavax.comm.SerialPortwhich isnrjavaserial, right?:https://github.com/openhab/openhab-core/blob/987af36bcf08d0904897605e2b93a59133c42aa9/bundles/org.openhab.core.io.transport.serial.javacomm/src/main/java/org/openhab/core/io/transport/serial/internal/SerialPortImpl.java#L38
But this javacomm is not installed on my openHAB instance, I dont know why (maybe disabled in some pipeline or karaf feature by default.
So I think I am now debugging the
rxtximplementation and library which is in thenrjavaserialbundle? Why?It seems as the
CommPortIdentifier.getPortIdentifiers()scan does not find all of the USB ports, or it even sometimes "forgets" some of them over time here:https://github.com/openhab/openhab-core/blob/987af36bcf08d0904897605e2b93a59133c42aa9/bundles/org.openhab.core.io.transport.serial.rxtx/src/main/java/org/openhab/core/io/transport/serial/internal/SerialPortUtil.java#L79
An "automatic" scan might miss some stuff (especially aliases, which is why I also tried to use the device names directly), so I had hope in
https://github.com/openhab/openhab-core/blob/987af36bcf08d0904897605e2b93a59133c42aa9/bundles/org.openhab.core.io.transport.serial.rxtx/src/main/java/org/openhab/core/io/transport/serial/internal/SerialPortUtil.java#L92-L106
But all it does is calling
CommPortIdentifier.getPortIdentifiers()again, which is the automatic scan from before, why? See:https://github.com/openhab/openhab-core/blob/987af36bcf08d0904897605e2b93a59133c42aa9/bundles/org.openhab.core.io.transport.serial.rxtx/src/main/java/org/openhab/core/io/transport/serial/internal/RxTxPortProvider.java#L63-L65
What I did afterwards is to adjust the method
getPortIdentifiersUsingProperty()to:But I get exceptions like: (from
serial.internal.SerialConfigOptionProvider):And from the binding currently under development (stiebelheathump):
So at some point the ports go missing and I have absolutely no clue where :-(
Maybe someone can shed a light on the usage of rxtx vs nrjavaserial and which is now used and how in openHAB, because to me this looks a bit confusing at the moment, especially why its still creating lock files although there have been discussions in the past that this should not be the case anymore with the latest nrjavaserial, but is openHAB using this?
I am also aware of that the java serial libraries are not the best and I know that they contain bugs, probably in their native code. So the bug might also be there. Its a bit frustrating to spend so many days without finding the root cause, which is why I am opening this bug as a last resort.
Because plan B is to use another raspberry and convert the serial events into soemthing openHAB can use, which I am trying to avoid as I am trying to get a away from self build custom solutions.
PS: I have seen https://github.com/openhab/openhab-core/issues/1842 but it might only be related because my issue also occurs on a fresh startup of the server/openHAB instance.