Serial port not found at startup

[5iver]

@cdjackson, I'm creating this here, but it could likely be caused by a core issue. @maggu2810, in case this is a core issue, you may want to be aware of this.

Issue 1 Since #1152, which moved zwave to openhab-core-io-transport-serial, the zwave binding does not find the configured serial port after an OH restart. After manually restarting the binding, the port is found.

Issue 2 Likely related to issue 1, but possibly different. After the zwave binding finds the serial port, the dropdown for serial ports in all bindings (I've only tested using zwave and zigbee), which used to display all available ports...

... only displays the port used by zwave...

The other binding(s) continue to function, but what I have found is that the zigbee binding, which does not use openhab-core-io-transport-serial, will then lose connection to its serial port after 24 hours.

[cdjackson]

I think we should move this to the core. The problem that is reported on the forum has stated that it is not limited to ZWave.

1. ZWave is using the ESH serial proxy. From what I read from peoples reports, it fails to find the serial port during first initialisation - if the binding is stopped and restarted, it finds the port - to be more precise, I think the error is that there is no provider for this type of port. I wonder if there is anything to ensure that the different providers (ie nrjavaserial and rfc2217) are started before the binding starts? My guess is not?

2. The serial port lists are not in any way linked to the binding - these are added somewhere else in the system. The trigger for adding them is setting the parameter context to serial-port and then something in one of the UI bundles provides the options.

I would suggest we move this to wherever the ESH serial provider is now located, as I think the error (at least 1) is located there.

[cdjackson]

The following logs are reported when this issue occurs -:

2019-05-02 19:45:43.586 [INFO ] [ing.zwave.handler.ZWaveSerialHandler] - Connecting to serial port '/dev/ttyACM0'

2019-05-02 19:45:43.603 [WARN ] [erial.internal.SerialPortManagerImpl] - No SerialPortProvider found for: /dev/ttyACM0

I think this shows that the provider is not yet registered. The reason a restart fixes it is probably simply that it gives time for the provider to register.

I'm not familiar with the serial provider so don't know how it works. All I can offer is to remove the serial provider, but as I was asked to migrate this over, and as other bindings are also using this, if it is a feature the system wants to provide, then I think it should be fixed there.

[maggu2810]

@cdjackson Can you show me the code you used previously (before your migration to the ESH serial manager) to get all available serial ports?

The current approach does not work reliable but IIRC this is caused by nrjavaserial itself.

I added a console extension for some serial commands, so I can look at the current known serial identifiers easily.

g! smarthome:serial port-creators
class org.eclipse.smarthome.io.transport.serial.internal.RxTxPortProvider, accepted protocols: [ProtocolType [pathType=LOCAL, scheme=rxtx]], port identifiers: []
class org.eclipse.smarthome.io.transport.serial.rxtx.rfc2217.internal.RFC2217PortProvider, accepted protocols: [ProtocolType [pathType=NET, scheme=rfc2217]], port identifiers: []

The port-creators show me all registered port creators and its know port identifiers (the port creators are queried about its identifiers. Boths return an empty list. For the RFC2217PortProvider the method is not implemented at all (there is a "TODO" comment). The RxTxPortProvider calls CommPortIdentifier.getPortIdentifiers() and as we can see, it returns "nothing".

g! smarthome:serial identifier-name /dev/ttyUSB0
[name: /dev/ttyUSB0, current owner: null]

Now I request a specific port identifier by its name. The RxTxPortProvider calls CommPortIdentifier.getPortIdentifier(port.getPath()). As we can see, it returns an identifier.

g! smarthome:serial port-creators
class org.eclipse.smarthome.io.transport.serial.internal.RxTxPortProvider, accepted protocols: [ProtocolType [pathType=LOCAL, scheme=rxtx]], port identifiers: [[name: /dev/ttyUSB0, current owner: null]]
class org.eclipse.smarthome.io.transport.serial.rxtx.rfc2217.internal.RFC2217PortProvider, accepted protocols: [ProtocolType [pathType=NET, scheme=rfc2217]], port identifiers: []

Now the CommPortIdentifier is known and will be shown in the list.

-- edit --

I realized that RxTxPortProvider adds the path of the requested port (by name) to the serial port property on Linux system. So, a port that is not added to the property on startup script will be added on its first call...

SerialPortUtil.appendSerialPortProperty(port.getPath());

-- edit --

So, the behaviour I identified (and did not remember) is the expected one. If I set the property "gnu.io.rxtx.SerialPorts" to "/dev/ttyUSB0" I don't need to call the specific identifier request anymore.

[cdjackson]

The previous version can be seen in the changes in the PR -:

https://github.com/openhab/org.openhab.binding.zwave/pull/1152/files

Doesn't that still seem like a system start issue then?

[maggu2810]

If you would like to give it a try yourself, here the relevant changes to use the serial port commands: https://github.com/maggu2810/openhab-core/compare/decb8c6f7176358c1cba82a027ed6eb92842873b..maggu2810:e95467b7b1e7b1d694a676b3cd3f1b23acd790de?expand=1

[5iver]

I updated to S1585 tonight, as there were some interesting changes in core. I also updated the zigbee and zwave bindings as they have now been built with BND, using a manual install. However, neither binding showed as installed (the zigbee libraries did though). I then removed the jars from addons and added them to addons.cfg. Both bindings started properly and continued to start after restarting OH a few times. So, this issue looks to be resolved(!)... at least partially. The serial port configuration for both controllers are behaving the same as before, and only the zwave port is in the list. It will be interesting to see if zigbee can stay online, or if it will drop after a day as was happening before.

1) The startup issue looks to be resolved, but maybe zigbee will fail. Should it be updated to use openhab-core-io-transport-serial? 2) Users will need to be informed that these bindings can no longer be manually installed. This means that testing the zigbee libraries in development will be much more difficult. I vaguely remember that this was discussed as a change that would occur when moving to BND. Is this accurate, and if so, why? 3) The serial port dropdown is not being populated properly and it is still not certain if this is core or binding.

[cdjackson]

The serial port population is nothing to do with the binding - I think I mention thinsin my earlier response. The binding just sets the parameter context in the xml file and another bundle fills this in.

Sent from my iPhone

On 5 May 2019, at 03:14, Scott Rushworth notifications@github.com wrote:

I updated to S1585 tonight, as there were some interesting changes in core. I also updated the zigbee and zwave bindings as they have now been built with BND, using a manual install. However, neither binding showed as installed (the zigbee libraries did though). I then removed the jars from addons and added them to addons.cfg. Both bindings started properly and continued to start after restarting OH a few times. So, this issue looks to be resolved(!)... at least partially. The serial port configuration for both controllers are behaving the same as before, and only the zwave port is in the list. It will be interesting to see if zigbee can stay online, or if it will drop after a day as was happening before.

The startup issue looks to be resolved, but maybe zigbee will fail. Should it be moved to openhab-core-io-transport-serial? Users will need to be informed that these bindings can no longer be manually installed. This means that testing the zigbee libraries in development will be much more difficult. I vaguely remember that this was discussed as a change that would occur when moving to BND. Is this accurate, and if so, why? The serial port dropdown is not being populated properly, and still not certain if this is core or binding. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[cdjackson]

It’s a shame if we now have a system that doesn’t allow easy manual install - that will make ZigBee much more problematic :(

Sent from my iPhone

On 5 May 2019, at 03:14, Scott Rushworth notifications@github.com wrote:

I updated to S1585 tonight, as there were some interesting changes in core. I also updated the zigbee and zwave bindings as they have now been built with BND, using a manual install. However, neither binding showed as installed (the zigbee libraries did though). I then removed the jars from addons and added them to addons.cfg. Both bindings started properly and continued to start after restarting OH a few times. So, this issue looks to be resolved(!)... at least partially. The serial port configuration for both controllers are behaving the same as before, and only the zwave port is in the list. It will be interesting to see if zigbee can stay online, or if it will drop after a day as was happening before.

The startup issue looks to be resolved, but maybe zigbee will fail. Should it be moved to openhab-core-io-transport-serial? Users will need to be informed that these bindings can no longer be manually installed. This means that testing the zigbee libraries in development will be much more difficult. I vaguely remember that this was discussed as a change that would occur when moving to BND. Is this accurate, and if so, why? The serial port dropdown is not being populated properly, and still not certain if this is core or binding. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[5iver]

I'm not sure what @maggu2810's analysis is revealing, but I'll close this and open an issue in core for the population of the serial port dropdown. That serial interface for the console would be handy! I don't think this is just a UI issue, as I have found that the zigbee binding is randomly dropping the serial port and the only way I have found to get it back online is to restart OH. However, I was able to uninstall the libraries (and bridge?) and use the 1.1.11 jars. So, we should still be able to test, but my manual install script should no longer be used.

[cdjackson]

When I referred to the problem being a UI issue - I was only taking about the listing of the ports. If there are issues opening ports, dropping ports etc, then this won’t be UI related.