[feat] Add socat/remote zwave device option

vlycop commented 2 years ago

Is your feature request related to a problem? Please describe. I am slowly moving to docker to make it easier to manage for my relative. The docker instance in on a virtual environment where i cannot pass usb device directly Because of this, i use ssl socat server on a raspberry A who's only job is to provide device to VM Because of https://github.com/docker/for-linux/issues/77, i can't use the "device" option to mount my zwave device, and have to use the "volume" as a work around. This have the huge drawback that if the connection is restarted between the raspberry and the VM (because the vm migrated, because timed out, because the router cleaned up state) the device in the zwavejs container will "freeze" and report I/O error. This isn't seen by the /health endpoint, so i cannot restart the container, but to be fair i would prefer not to anyway.

With the actual capability, a lost in network connection break the docker service, and that breakage isn't reported in the /health endpoint so Zwave will be unavailable until a user realize and restart it manually.

Describe the solution you'd like I would like to be able to specify an IP and a port of a socat server instead of a /dev/XXX device, maybe with a cert and a CA to keep SSL. That way the connection can be restarted when detected closed from inside the docker images, not forcing a full network restart and not forcing user action Describe alternatives you've considered

a fix docker side don't seam to be there anytime soon.
Trying to work with the /health didn't help, but maybe you could add a regular check that the usb device still reply in the /health/zwave ? it's not ideal because it will force a network restart but at least i can automate it from inside docket.
You can also provide any other way to connect to remote usb device. i know there is a "in kernel" way to do so but i haven't played with it at all (https://www.kernel.org/doc/readme/tools-usb-usbip-README)
Last is not using docker, and will surely be the way for me given to surely low priority of this.

Additional context i've look into any issue already closed to make sure you didn't already excluded that usecase, but didn't find anything. I don't know if i am the only one with that use-case, but Home assistant droping OpenZwave (Good !) mean i have to make this project work now and i didn't find an alternative to socat.

robertsLando commented 2 years ago

@vlycop Did you consider using ser2net ?

vlycop commented 2 years ago

@vlycop Did you consider using ser2net ?

No, because at risk of saying something complety wrong, it is the same ? as far as i know, ser2net is basically a socat server, without much more, and my issue is on the client side, exposed to the docker host and docker container. Actually having this feature implemented would allow for a ser2net server on one side, and zwavejs ui directly on the other side without having to setup a client on the docket host

But you mentioning it again made me look at it and found all of those resolved issue https://github.com/zwave-js/zwavejs2mqtt/issues/1246 https://github.com/zwave-js/zwavejs2mqtt/issues/2086 https://github.com/zwave-js/zwavejs2mqtt/issues/1800

so i've search for ser2net or tcp:// in https://zwave-js.github.io/zwavejs2mqtt/ to see if i didn't miss a way to put a tcp url in the device path already, didn't found anything

Serial port: The serial port where your controller is connected

Did i miss something obvious ?

robertsLando commented 2 years ago

@vlycop in the serlport input you can manually write the tcp url in the format tcp://yoururl:5000

vlycop commented 2 years ago

Oh, Cool ! i didn't found that in the doc and the last issue did hint about it.

I'll try to see if this work with a socat server, but it should. The only "missing" bit then, and i will rename that feat if you do think it's interesting, would be to add a cert and CA possibility for that.

for exemple my socat look like this right now

client : socat openssl-connect:ap-rpi3bp-002.domain.com:4435,cert=/home/socat/cert/ap-univm-021.pem,cafile=/home/socat/cert/ap-rpi3bp-002.crt,verify=0 pty,link=/dev/lio_zwave,user=socat,group=dialout,mode=660,nonblock,raw,ignoreof
server: socat openssl-listen:4435,keepalive,reuseaddr,cert=/srv/homeassistant/socat/ap-rpi3bp-002.pem,cafile=/srv/homeassistant/socat/ap-univm-021.crt,verify=0 /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_92a36bbd7784e911bd509cd6217343c2-if00-port0,raw,echo=0

I believe having a s2 secured zwave network end up silly if all communication go through unencrypted in the network.

robertsLando commented 2 years ago

I think this is something that would require changes also on zwave-js level

cc @AlCalzone

vlycop commented 2 years ago

To keep thing safe for now, i'm double bouncing with ease rpi --- SSL Socat ---> VM --- Clear local Socat ---> Container and it work :) Only a single issue that may be easy to fix but i can't find the doc :(

2022-04-04T17:07:16.814Z DRIVER   starting driver...
2022-04-04T17:07:16.825Z DRIVER   opening serial port tcp://172.17.0.1:5896
#### clicking save after 3mn
2022-04-04T17:10:52.182Z DRIVER   destroying driver instance...
2022-04-04T17:10:52.184Z DRIVER   driver instance destroyed
2022-04-04T17:10:52.197Z DRIVER

The driver don't seam to timeout and retry if the serial port is closed when it start. Clicking the save button again will trigger a driver restart, and if the port is open, connect without issue This happens during restart of socat or if the container start before the socat service.

AlCalzone commented 2 years ago

Actually, I have no clue what's necessary to connect to a secure socat instance. The network connection is just a simple TCP socket. FWIW, the documentation is over in the driver repo: https://zwave-js.github.io/node-zwave-js/#/usage/tcp-connection

Regarding the automatic reconnection, there seems to be something broken: https://github.com/zwave-js/node-zwave-js/issues/4174 I hope to investigate this soon.

vlycop commented 2 years ago

Thank you for the doc, I'll read it tomorrow. I will also try to look around in the code. I can hardly navigate it, but I already found where the connection is made and will dig a bit more tomorrow.

No promises tho.

AlCalzone commented 2 years ago

If this is about the secure connection, packages/serial/src is where the serial/socket code resides. ZWaveSerialPortBase is the common base class for all serial connections, ZWaveSerialPort is the specific implementation for (local) serial ports, ZWaveSocket for TCP/IPC sockets.

As for the reconnection:

The socket implementation should throw/emit an error when the connection goes down
The driver either catches this in the "error" event handler in the start() method (if already connected), or in the openSerialport method (if not yet connected) and self-destructs

vlycop commented 2 years ago

If this is about the secure connection, packages/serial/src is where the serial/socket code resides. ZWaveSerialPortBase is the common base class for all serial connections, ZWaveSerialPort is the specific implementation for (local) serial ports, ZWaveSocket for TCP/IPC sockets.

Would it be possible to use https://nodejs.org/docs/latest-v17.x/api/tls.html instead of https://nodejs.org/docs/latest-v17.x/api/net.html when cert are provided ? Having never done any JS, https://riptutorial.com/node-js/example/19326/tls-socket--server-and-client doesn't sound so different.

vlycop commented 2 years ago

As for the reconnection:

i am surprised to not see a connectListener here https://github.com/zwave-js/node-zwave-js/blob/966e51fc81eb7a1686f9ac94971a06bab5ed26f1/packages/serial/src/ZWaveSocket.ts#L17 in the connect section, but i don't know much about JS syntax i only see a serial.on("close", ()) which if i'm not wrong won't catch connection error because it need to listen for error ?

If there is a problem connecting, instead of a 'connect' event, an 'error' event will be emitted with the error passed to the 'error' listener. The last parameter connectListener, if supplied, will be added as a listener for the 'connect' event once.

AlCalzone commented 2 years ago

i am surprised to not see a connectListener here

Its the callback to connect: https://github.com/zwave-js/node-zwave-js/blob/966e51fc81eb7a1686f9ac94971a06bab5ed26f1/packages/serial/src/ZWaveSocket.ts#L31-L34

The missing error listener might be a problem, at the very least it's handled differently for the serial implementation: https://github.com/zwave-js/node-zwave-js/blob/966e51fc81eb7a1686f9ac94971a06bab5ed26f1/packages/serial/src/ZWaveSerialPort.ts#L33-L71

robertsLando commented 2 years ago

@AlCalzone Can I move this to zwavejs?

AlCalzone commented 2 years ago

@AlCalzone Can I move this to zwavejs?

giphy-downsized

vlycop commented 2 years ago

i'll close this and iopen another one if/when new config field are needed for ssl.

vlycop commented 2 years ago

@robertsLando out of curiosity, what does the restart on failed ? Zwave-js or the front end ? Because when trying the source of zwave-js with it's debug script, it does stop both in when it disconnect (but don't restart like in the front end) and if the tcp socket isn't available (while it hang for ever in the front end)

If this is a managed in the front end, then it might be more an issue for here than for AICalzone ?

Edit: I think i've seen the issue in zwavejs, the error is never emitted and i've read that you use that error to know when to restart.

robertsLando commented 2 years ago

Edit: I think i've seen the issue in zwavejs, the error is never emitted and i've read that you use that error to know when to restart

Exactly

zwave-js / zwave-js-ui

[feat] Add socat/remote zwave device option #2345