search

tiagocoutinho / modbus-proxy

Connect multiple clients to modbus devices
Other
71 stars 13 forks source link

Please make a Docker of this #1

Closed magpern closed 3 years ago

magpern commented 3 years ago

I have a solaredge PV inverter that only allows one connection at a time I have tried to make a docker, but my knowledge is not enough.

My use case is to have modbus tcp running in a docker with configurable ip and port. I have two applications that needs to read the same ip and port

tiagocoutinho commented 3 years ago

Hi,

Thanks for the suggestion.

I just committed a Dockerfile and some doc in the README: https://github.com/tiagocoutinho/modbus-proxy#docker

Let me know if its good so I can close the issue

magpern commented 3 years ago

Wow! This was really fast! It looks like the Docker works. I see no reason for using python:3.8 and then python:3.8-alpine. You could use alpine only, that way one does not have to download a 880MB image, but only a 44MB one.

I tried the Dockerfile and it works. But it is only possible for one computer to connect. If two tries, the pipe breaks, which pretty much makes this application useless.

I have a SolarEdge PV inverter. It exposed port 1502 (configurable), but I set the parameters for modbus-proxy to connect to 192.168.1.8:1502 and bind to 0:1502 In docker I redirect 1502 to 502

I also have a Victron Battery system running something called Venus OS. It can connect to SolarEdge, but only on port 502, hence the redirect of port.

When I run Venus OS and points it to 192.168.1.224:502 (where the docker runs) it works perfectly. I see updates in Venus OS and it says it is connected.

Then I add HomeAssistant (HA) to the mix. I use a Solaredge plugin (which works), configures it to same IP (192.168.1.224:502). Now nothing works. No update in HA and Venus OS stops reporting. Removing HA plugin, Venus OS starts reporting again (I did reboot the docker container, maybe it would recover, configured it to Restart: Always in the docker-compose file)

Here is the stack trace: 2021-01-23T00:09:26.833341000Z return await reader.read(4096), 2021-01-23T00:09:26.833430000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.833520000Z raise self._exception, 2021-01-23T00:09:26.833610000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.833698000Z yield await func(*args, kwargs), 2021-01-23T00:09:26.833785000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.833872000Z return await reader.read(4096), 2021-01-23T00:09:26.833959000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.834048000Z raise self._exception, 2021-01-23T00:09:26.834135000Z File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 910, in write, 2021-01-23T00:09:26.834222000Z n = self._sock.send(data), 2021-01-23T00:09:26.834308000Z BrokenPipeError: [Errno 32] Broken pipe, 2021-01-23T00:09:26.834398000Z Task exception was never retrieved, 2021-01-23T00:09:26.834485000Z future: <Task finished name='Task-863' coro=<run..cb() done, defined at ./modbus_proxy.py:22> exception=BrokenPipeError(32, 'Broken pipe')>, 2021-01-23T00:09:26.834572000Z Traceback (most recent call last):, 2021-01-23T00:09:26.834663000Z File "./modbus_proxy.py", line 23, in cb, 2021-01-23T00:09:26.834752000Z await (, 2021-01-23T00:09:26.834841000Z File "/root/.local/lib/python3.8/site-packages/aiostream/core.py", line 29, in wait_stream, 2021-01-23T00:09:26.834930000Z async for item in streamer:, 2021-01-23T00:09:26.835021000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/combine.py", line 72, in smap, 2021-01-23T00:09:26.835108000Z async for item in streamer:, 2021-01-23T00:09:26.835195000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine, 2021-01-23T00:09:26.835282000Z result = task.result(), 2021-01-23T00:09:26.835371000Z File "./modbus_proxy.py", line 23, in cb, 2021-01-23T00:09:26.835458000Z await (, 2021-01-23T00:09:26.835545000Z File "/root/.local/lib/python3.8/site-packages/aiostream/core.py", line 29, in wait_stream, 2021-01-23T00:09:26.835634000Z async for item in streamer:, 2021-01-23T00:09:26.835722000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/combine.py", line 72, in smap, 2021-01-23T00:09:26.835809000Z async for item in streamer:, 2021-01-23T00:09:26.835896000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine, 2021-01-23T00:09:26.835982000Z result = task.result(), 2021-01-23T00:09:26.836072000Z File "./modbus_proxy.py", line 23, in cb, 2021-01-23T00:09:26.836160000Z await (, 2021-01-23T00:09:26.836247000Z File "/root/.local/lib/python3.8/site-packages/aiostream/core.py", line 29, in wait_stream, 2021-01-23T00:09:26.836336000Z async for item in streamer:, 2021-01-23T00:09:26.836423000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/combine.py", line 72, in smap, 2021-01-23T00:09:26.836510000Z async for item in streamer:, 2021-01-23T00:09:26.836597000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine, 2021-01-23T00:09:26.836683000Z result = task.result(), 2021-01-23T00:09:26.836779000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.836868000Z yield await func(*args, *kwargs), 2021-01-23T00:09:26.836958000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.837046000Z return await reader.read(4096), 2021-01-23T00:09:26.837133000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.837220000Z raise self._exception, 2021-01-23T00:09:26.837309000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.837396000Z yield await func(args, kwargs), 2021-01-23T00:09:26.837485000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.837574000Z return await reader.read(4096), 2021-01-23T00:09:26.837662000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.837749000Z raise self._exception, 2021-01-23T00:09:26.837835000Z File "./modbus_proxy.py", line 23, in cb, 2021-01-23T00:09:26.837922000Z await (, 2021-01-23T00:09:26.838008000Z File "/root/.local/lib/python3.8/site-packages/aiostream/core.py", line 29, in wait_stream, 2021-01-23T00:09:26.838097000Z async for item in streamer:, 2021-01-23T00:09:26.838187000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/combine.py", line 72, in smap, 2021-01-23T00:09:26.838275000Z async for item in streamer:, 2021-01-23T00:09:26.838361000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine, 2021-01-23T00:09:26.838448000Z result = task.result(), 2021-01-23T00:09:26.838537000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.838630000Z yield await func(*args, kwargs), 2021-01-23T00:09:26.838719000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.838808000Z return await reader.read(4096), 2021-01-23T00:09:26.838898000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.838989000Z raise self._exception, 2021-01-23T00:09:26.839080000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.839169000Z yield await func(*args, *kwargs), 2021-01-23T00:09:26.839260000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.839350000Z return await reader.read(4096), 2021-01-23T00:09:26.839438000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.839526000Z raise self._exception, 2021-01-23T00:09:26.839617000Z File "./modbus_proxy.py", line 23, in cb, 2021-01-23T00:09:26.839706000Z await (, 2021-01-23T00:09:26.839794000Z File "/root/.local/lib/python3.8/site-packages/aiostream/core.py", line 29, in wait_stream, 2021-01-23T00:09:26.839884000Z async for item in streamer:, 2021-01-23T00:09:26.839973000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/combine.py", line 72, in smap, 2021-01-23T00:09:26.840061000Z async for item in streamer:, 2021-01-23T00:09:26.840149000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine, 2021-01-23T00:09:26.840238000Z result = task.result(), 2021-01-23T00:09:26.840329000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.840418000Z yield await func(args, kwargs), 2021-01-23T00:09:26.840509000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.840597000Z return await reader.read(4096), 2021-01-23T00:09:26.840685000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.840783000Z raise self._exception, 2021-01-23T00:09:26.840873000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.840962000Z yield await func(*args, *kwargs), 2021-01-23T00:09:26.841052000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.841146000Z return await reader.read(4096), 2021-01-23T00:09:26.841236000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.841325000Z raise self._exception, 2021-01-23T00:09:26.841413000Z File "/root/.local/lib/python3.8/site-packages/aiostream/stream/create.py", line 75, in call, 2021-01-23T00:09:26.841501000Z yield await func(args, **kwargs), 2021-01-23T00:09:26.841589000Z File "./modbus_proxy.py", line 16, in process, 2021-01-23T00:09:26.841676000Z return await reader.read(4096), 2021-01-23T00:09:26.841769000Z File "/usr/local/lib/python3.8/asyncio/streams.py", line 665, in read, 2021-01-23T00:09:26.841858000Z raise self._exception, 2021-01-23T00:09:26.841946000Z File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 910, in write, 2021-01-23T00:09:26.842034000Z n = self._sock.send(data), 2021-01-23T00:09:26.842123000Z BrokenPipeError: [Errno 32] Broken pipe,

tiagocoutinho commented 3 years ago

It looks like the Docker works. I see no reason for using python:3.8 and then python:3.8-alpine. You could use alpine only, that way one does not have to download a 880MB image, but only a 44MB one.

Great recommendation. Thanks! I used a two phase build in docker because I thought alpine did not come with pip. Turns out I was wrong. It should be fixed now in master.

Concerning your error: in principle the server should be able to handle concurrent connections which is the main purpose of the project as you correctly pointed out. From the log it seems the server is not handling disconnections properly.

Have you tried the same without docker? Does it work?

Anyway, I have updated the master branch with a robust version of the server. It should be able to handle abrupt disconnections from both client and the modbus device sides. I also made a new release (0.2.0) on pypi in case you are interested. I have made some stress tests in this version and it seems to handle well multi client.

Would you have time to test? Thanks in advance

magpern commented 3 years ago

This was a tough nut to crack.

I installed the new version. It works fine against Venus OS. I see all updates on the fly. I then configure HomeAssistent to read from the same modbus-proxy. Boom.. nothing works.. broken pipe.

I stopped mdbus-prxoxy. Configured HomeAssistent to read the SolarEdge Ip address directly. It worked. I see updated. I change HA to point to the docker server on port 502. And start docker modbus proxy again. I excpet Venus OS and HomeAssistent to see the data.... now no one sees the data...

192.168.1.87 is Victron (Venus OS)

Tracefile of only Venus OS connected. (this looks like it is working. Veus OS is reporting 0 Watts at the moment which is correct, and does not report "Not Connected") 2021-01-24 00:29:13,227: INFO:modbus-proxy:Starting..., 2021-01-24 00:29:13,232: INFO:modbus-proxy:Ready!, 2021-01-24 00:30:09,921: INFO:modbus-proxy.Client(192.168.1.87:56968):new connection, 2021-01-24 00:30:09,921: INFO:modbus-proxy.modbus(192.168.1.8:1502):connecting to modbus at ('192.168.1.8', 1502)..., 2021-01-24 00:30:09,923: INFO:modbus-proxy.modbus(192.168.1.8:1502):connected!, 2021-01-24 00:30:10,126: INFO:modbus-proxy.Client(192.168.1.87:56970):new connection, 2021-01-24 00:30:10,127: INFO:modbus-proxy.Client(192.168.1.87:56968):ConnectionClosedError('Client disconnected'), 2021-01-24 00:31:15,129: INFO:modbus-proxy.Client(192.168.1.87:56976):new connection, 2021-01-24 00:31:15,335: INFO:modbus-proxy.Client(192.168.1.87:56976):ConnectionClosedError('Client disconnected'), 2021-01-24 00:32:20,344: INFO:modbus-proxy.Client(192.168.1.87:56982):new connection, 2021-01-24 00:32:20,500: INFO:modbus-proxy.Client(192.168.1.87:56982):ConnectionClosedError('Client disconnected'), 2021-01-24 00:33:25,507: INFO:modbus-proxy.Client(192.168.1.87:56988):new connection, 2021-01-24 00:33:25,675: INFO:modbus-proxy.Client(192.168.1.87:56988):ConnectionClosedError('Client disconnected'), 2021-01-24 00:34:30,679: INFO:modbus-proxy.Client(192.168.1.87:56994):new connection, 2021-01-24 00:34:30,869: INFO:modbus-proxy.Client(192.168.1.87:56994):ConnectionClosedError('Client disconnected'),

Tracefile of Venus OS and HomeAssistent connected (HomeAssistant is 172.19.0.1) 2021-01-24 00:12:03,561: INFO:modbus-proxy.Client(172.19.0.1:52240):new connection, 2021-01-24 00:12:03,561: INFO:modbus-proxy.modbus(192.168.1.8:1502):connecting to modbus at ('192.168.1.8', 1502)..., 2021-01-24 00:12:03,563: INFO:modbus-proxy.modbus(192.168.1.8:1502):connected!, 2021-01-24 00:12:10,832: INFO:modbus-proxy.Client(192.168.1.87:49902):new connection, 2021-01-24 00:12:33,596: INFO:modbus-proxy.Client(172.19.0.1:52258):new connection,

I removed HA and only have Venus OS (192.168.1.87) 2021-01-24 00:43:09,715: INFO:modbus-proxy.Client(172.19.0.1:56804):new connection, 2021-01-24 00:43:09,715: ERROR:modbus-proxy.Client(172.19.0.1:56804):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:43:24,757: INFO:modbus-proxy.Client(172.19.0.1:56876):new connection, 2021-01-24 00:43:24,757: ERROR:modbus-proxy.Client(172.19.0.1:56876):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:43:24,761: INFO:modbus-proxy.Client(172.19.0.1:56880):new connection, 2021-01-24 00:43:24,761: ERROR:modbus-proxy.Client(172.19.0.1:56880):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:43:26,356: INFO:modbus-proxy.Client(192.168.1.87:57042):new connection, 2021-01-24 00:43:26,356: ERROR:modbus-proxy.Client(192.168.1.87:57042):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:43:27,768: INFO:modbus-proxy.Client(172.19.0.1:56884):new connection, 2021-01-24 00:43:27,768: ERROR:modbus-proxy.Client(172.19.0.1:56884):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:43:27,772: INFO:modbus-proxy.Client(172.19.0.1:56888):new connection, 2021-01-24 00:43:27,773: ERROR:modbus-proxy.Client(172.19.0.1:56888):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:13:03,756: INFO:modbus-proxy.Client(172.19.0.1:52276):new connection, 2021-01-24 00:13:30,867: INFO:modbus-proxy.Client(192.168.1.87:49918):new connection, 2021-01-24 00:13:33,842: INFO:modbus-proxy.Client(172.19.0.1:52294):new connection, 2021-01-24 00:14:03,582: INFO:modbus-proxy.Client(172.19.0.1:52240):ConnectionClosedError('Modbus disconnected'), 2021-01-24 00:14:03,583: INFO:modbus-proxy.Client(192.168.1.87:49902):ConnectionClosedError('Modbus disconnected'), 2021-01-24 00:14:03,583: INFO:modbus-proxy.Client(172.19.0.1:52258):ConnectionClosedError('Modbus disconnected'), 2021-01-24 00:14:03,584: ERROR:modbus-proxy.Client(172.19.0.1:52276):ConnectionResetError('Connection lost'), 2021-01-24 00:14:03,585: ERROR:modbus-proxy.Client(192.168.1.87:49918):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:14:03,585: ERROR:modbus-proxy.Client(172.19.0.1:52294):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:14:03,614: INFO:modbus-proxy.Client(172.19.0.1:52314):new connection, 2021-01-24 00:14:03,614: ERROR:modbus-proxy.Client(172.19.0.1:52314):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:14:06,599: INFO:modbus-proxy.Client(172.19.0.1:52338):new connection, 2021-01-24 00:14:06,599: ERROR:modbus-proxy.Client(172.19.0.1:52338):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:14:06,998: INFO:modbus-proxy.Client(172.19.0.1:52342):new connection, 2021-01-24 00:14:06,998: ERROR:modbus-proxy.Client(172.19.0.1:52342):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:14:09,607: INFO:modbus-proxy.Client(172.19.0.1:52356):new connection, 2021-01-24 00:14:09,607: ERROR:modbus-proxy.Client(172.19.0.1:52356):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:14:10,003: INFO:modbus-proxy.Client(172.19.0.1:52364):new connection, 2021-01-24 00:14:10,003: ERROR:modbus-proxy.Client(172.19.0.1:52364):BrokenPipeError(32, 'Broken pipe'),

Restarted Venus OS (192.168.1.87) and no HA 2021-01-24 00:46:41,427: INFO:modbus-proxy.Client(192.168.1.87:57060):new connection, 2021-01-24 00:46:41,428: ERROR:modbus-proxy.Client(192.168.1.87:57060):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:47:46,443: INFO:modbus-proxy.Client(192.168.1.87:57066):new connection, 2021-01-24 00:47:46,443: ERROR:modbus-proxy.Client(192.168.1.87:57066):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:48:51,458: INFO:modbus-proxy.Client(192.168.1.87:57072):new connection, 2021-01-24 00:48:51,458: ERROR:modbus-proxy.Client(192.168.1.87:57072):BrokenPipeError(32, 'Broken pipe'), 2021-01-24 00:49:22,775: INFO:modbus-proxy:Starting..., 2021-01-24 00:49:22,780: INFO:modbus-proxy:Ready!, 2021-01-24 00:49:56,482: INFO:modbus-proxy.Client(192.168.1.87:57078):new connection, 2021-01-24 00:49:56,482: INFO:modbus-proxy.modbus(192.168.1.8:1502):connecting to modbus at ('192.168.1.8', 1502)..., 2021-01-24 00:49:56,484: INFO:modbus-proxy.modbus(192.168.1.8:1502):connected!, 2021-01-24 00:49:56,712: INFO:modbus-proxy.Client(192.168.1.87:57080):new connection, 2021-01-24 00:49:56,713: INFO:modbus-proxy.Client(192.168.1.87:57078):ConnectionClosedError('Client disconnected'),

This is the HA plugin https://github.com/erikarenhill/solaredge-modbus-hass

magpern commented 3 years ago

To be fair, the docker works, so case is resolved. Closing it and continuing the connection problem in a new issue https://github.com/tiagocoutinho/modbus-proxy/issues/2