Getting modbus4mqtt off the ground with or without docker

[kurt-klingbeil]

I am looking for a way to move data from a PLC via modbus via mqtt to a online dashboard/portal. I followed the bouncing ball... When I ran the pip3 install --user modbus4mqtt I got a screenfull of stuff about externally managed environment,

It suggested (sudo) apt install python3-modbus4mqtt which, after giving my request some consideration, respectfully declined with a "unable to locate package" error

So I thought I'll venture into the mysterious Domain of Docker

Yesterday when I ran: docker pull tjhowse/modbus4mqtt:latest i was told "permission denied" Today when I naively and foolishly decided to try again... it successfully pulled about a dozen "things" and appeared to have zero Cows... When I then ran: docker run modbus4mqtt --help I was told Unable to find image 'modbus4mqtt:latest' locally pull access denied repository does not exist or may require 'docker login' I did docker run --help and got a screenfull of stuff which is incomprehensible at this point

There is a link to https://pypi.org/project/modbus4mqtt/Dockerfile - which 404's out ditto for https://pypi.org/project/modbus4mqtt/modbus4mqtt/Sungrow_SH5k_20.yaml

So I am equally out of brilliant and dumb ideas.

I get the sense that docker is a little microcosm into which stuff can be installed without fouling the larger pi-nest, but naively assumed that the docker pull would magically "pull it together'

The overall conceptual question I have is: Is there a way to manually interact as a man-in-the-middle ? i.e. issue some manual commands which test whether the modbus TCP connection is up and running and coughing up data - kinda like using ModbusPoll ? and conversely, to manually simulate the generation of "manual-modbus" data which would then be mqtt-shovelled up into the dashboard thereby segmenting a three-node commissioning into two two-nodes commissionings ?

Of course, any clues, hints, over-the-top spoonfeeding for pulling all the bits of string together would be most welcome ;-)

best regards

[tjhowse]

Hm, you may have found an error in the documentation! Try docker run tjhowse/modbus4mqtt:latest --help and see what you get.

The python environment problem you have experienced, the "externally managed environment" thing, is a symptom of how the python environment of the operating system would prefer not to be polluted with packages from pypi. One possible way out of your mire without using docker is to install modbus4mqtt into a "venv", or python virtual environment:

# This creates a virtual environment inside the ".venv" hidden directory
python3 -m venv .venv
# This "activates" the venv, such that python commands use your venv for the duration of your shell session
. .venv/bin/activate
# This installs modbus4mqtt into your venv
python3 -m pip install modbus4mqtt
# This runs modbus4mqtt from inside the venv.
python3 -m modbus4mqtt --help

The links on pypi.org unfortunately don't work, as they're relative paths to files in the repo on github: https://github.com/tjhowse/modbus4mqtt/blob/master/Dockerfile and https://github.com/tjhowse/modbus4mqtt/blob/master/modbus4mqtt/Sungrow_SH5k_20.yaml.

As for your commissioning: You may have success using command line utilities like mosquitto_sub and mosquitto_pub, from the mosquitto-clients debian package, to poke and prod at your MQTT broker. You can monitor for publications and publish your own test messages to the broker to simulate the action of modbus4mqtt to leave the hardware out of the loop until you're ready.

Hope that helps, tjhowse.

[kurt-klingbeil]

On Thu, Jan 11, 2024 at 9:32 PM Travis Howse @.***> wrote:

Hm, you may have found an error in the documentation! Try docker run tjhowse/modbus4mqtt:latest --help and see what you get.The python environment problem you have experienced, the "externally managed environment" thing, is a symptom of how the python environment of the opera ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌

https://ca.report.cybergraph.mimecast.com/alert-details/?dep=bHr51GI6bwrEBjJWlstMOA%3D%3D2%2B2fRMb2vkOc9xE0EXyZh1ZScqXu1Bhf46iL52xso1lxjRIVPRgNVtsd7EyvE3xz0XVmF%2BlkQP1vvU2V8%2FtV%2BIh73eNfuv3IhuP0ETFgp%2FXPVaLJU%2BhavgnsZGq1iRx%2BAB6OdSTeQhfrufWdhrqja200aEqcCCyspQPL3V3Avel4pQ8hFYpZZqA6jB08c0zhN1Ba7%2BAbRAhccXEfwXO%2FGXTsweyHlVknX0usS9Tr7Y0zEsfStIhhMJqvAUcEvNcBAJP5ZlqaEK8kLZv%2Bs3L8YI2fDp9z1nM%2B8alskgLzYAITiP%2FZVvVlo1GD0EmCLaBZwbJARWnrfwinWVmGRImxvBJLiZwkUdGqWrW2%2FWgYUhT6d8IIc78jWmGDifsYxMk2yFwPGDPFE03D4VNA7XU7GlWxNDGUNyly0DO4WhyPVvwPuJhV1pOOtIH7Xem2vpDx2xglxN4777gMI00xgubYcGkyeu4en63c9WbcL5ThMT6czp5Ku%2B1wwPB3UrjGXuzdcjRedjWf23envZKajYYKwxIA4y8W%2Fscy7shNv0pmUD2e35RsU7g%2Fv3N%2FYTbVJ5TC7JlnBPpXnMfPQ3JLjcv26RKXIIAODNx0HTOixTaw%2Bxl0aCaniO%2BdBQIBLQtMN8beYr%2BzehlmsNYAajyFlUh%2F7tf53dTOosIfvxN9%2BFpT6LGH%2F4%2BAJbFtgfZHjfGDWccKaHaV7YxnJpNNRPEOmG88stojyaiJSR%2FwyHpKubfaDf0HprDD0QmpjKo5Gh1PJm1owmRNTiLJZ9pQ6UpwqmVs6izIzXJbHktXuLO0PYux5CZZ1eX9umdA7rhvH3LD0qGzTbxi5Wcia5aIkMVUQSItGB5la4XuM1%2Bu0C%2Bi7xtFOrRf%2Fk9MXglpuGS2rfz2QKmSUgTsH0Wizu8OUV07jiE5wCwDyj0zzNkFhUhtWqQV9Ww%3D

Hm, you may have found an error in the documentation! Try docker run tjhowse/modbus4mqtt:latest --help and see what you get.

docker: invalid reference format. See 'docker run --help'.

The python environment problem you have experienced, the "externally managed environment" thing, is a symptom of how the python environment of the operating system would prefer not to be polluted with packages from pypi. One possible way out of your mire without using docker is to do install modbus4mqtt into a "venv", or python virtual environment:

I am not specifically averse to running Docker - just unfamiliar at this point I think I figured out how to use the https://github.com/tjhowse/modbus4mqtt/blob/master/Dockerfile to build a Docker container.. I assume that the YAML file has to be pre-prepared and is then incorporated into the docker image ? are there any definitive status tags/variables such that when modbus4mqtt finds the TCP device down it reports that ? or the device is up but not responding to polls ? or some polls are responding but others are "illegal address" ? I'll deep dive the dox once I get something with a pulse ;-)

Oh... leaping ahead to the Crystal-Palace-of-the-Unattainable ;-) does the "reverse" flow work ? i.e. while telemetry data flows from modbus TCP to MQTT to deashboard, can values be pushed from dashboard through mqtt into modbus TCP registers ? I have farted around with this kind of stuff and RPC with Teltonika and other gateways and so far have not made that fly...

This creates a virtual environment inside the ".venv" hidden directory

python3 -m venv .venv

This "activates" the venv, such that python commands use your venv for the duration of your shell session

. .venv/bin/activate

This installs modbus4mqtt into your venv

python3 -m pip install modbus4mqtt

so far so good... everything went fine

This runs modbus4mqtt from inside the venv.

python3 -m modbus4mqtt --help

/home/pi/.venv/bin/python3: No module named modbus4mqtt.main; 'mosbus4mqtt is a package and connot be directly executed

The links on pypi.org unfortunately don't work, as they're relative paths to files in the repo on github: https://github.com/tjhowse/modbus4mqtt/blob/master/Dockerfile https://github.com/tjhowse/modbus4mqtt/blob/master/Dockerfile and https://github.com/tjhowse/modbus4mqtt/blob/master/modbus4mqtt/Sungrow_SH5k_20.yaml https://github.com/tjhowse/modbus4mqtt/blob/master/modbus4mqtt/Sungrow_SH5k_20.yaml .

As for your commissioning: You may have success using command line utilities like mosquitto_sub and mosquitto_pub, from the mosquitto-clients debian package, to poke and prod at your MQTT broker. You can monitor for publications and publish your own test messages to the broker to simulate the action of modbus4mqtt to leave the hardware out of the loop until you're ready.

Hope that helps, tjhowse.

— Reply to this email directly, view it on GitHub https://github.com/tjhowse/modbus4mqtt/issues/51#issuecomment-1888416534, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGID46PNYZBSEIONUOLXXFDYOC4CTAVCNFSM6AAAAABBXQKDACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBYGQYTMNJTGQ . You are receiving this because you authored the thread.Message ID: @.***>

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.