search

spacemanspiff2007 / sml2mqtt

Sml to MQTT Bridge
GNU General Public License v3.0
25 stars 8 forks source link

Docker support #8

Closed git-developer closed 1 year ago

git-developer commented 2 years ago

I prefer using Docker over installing a package and its dependencies directly to a system. Thus, I wrote a Dockerfile that allows using sml2mqtt without installing it.

Maybe it's useful for someone else, so I'd like to notify you about it. The project is currently hosted on GitHub under git-developer/sml2mqtt. Docker images are available on Docker hub under ckware/sml2mqtt, currently:

These images are based on latest stable Alpine linux.

For each version, there's another tag prefixed with legacy-:

These images are based on Alpine 3.12. They are targeted at hosts with outdated versions of Docker (< 19.03.9) or libseccomp2 (< 2.4.2), e.g. Debian/Raspbian Buster. See time64 requirements for details.

There's an update check running every morning that automatically creates a new image when new commits have been pushed to the main branch of sml2mqtt.

Thanks for maintaining sml2mqtt!

spacemanspiff2007 commented 1 year ago

@git-developer I've put lots of time and effort into a internal rework and also added a dockerfile. Would you be willing to try

git-developer commented 1 year ago

No problem. I'm currently busy, but I'll post my feedback here when I find the time to make a review.

git-developer commented 1 year ago

Your Dockerfile looks good. I didn't have the time to look at all the changes of the branch in detail, but I saw that you're already publishing to Docker Hub supporting 4 architectures. That's great. I tried your image spacemanspiff2007/sml2mqtt:rework and it worked without problems. I was able to use it as drop-in replacement for my Docker image using the same configuration file.

git-developer commented 1 year ago

Your dockerfile:

FROM python:3.10-alpine

VOLUME /sml2mqtt

RUN apk add --no-cache python3 py3-wheel py3-pip gcc musl-dev python3-dev git && \
    pip install --no-cache-dir --no-clean sml2mqtt && \
    apk del py3-wheel py3-pip gcc musl-dev python3-dev git && \
    rm -fr /tmp/* \
    cd  /sml2mqtt

CMD [ "sml2mqtt", "--config", "/sml2mqtt/config.yml"]

Summary:

FROM alpine
VOLUME /sml2mqtt
ARG SOURCE=sml2mqtt
RUN apk add --no-cache python3 py3-wheel py3-pip gcc musl-dev python3-dev git && \
    pip install --no-cache-dir "${SOURCE}" && \
    pip install --no-cache-dir pyclean && pyclean /usr && pip uninstall -y pyclean setuptools wheel pip && \
    apk del py3-wheel py3-pip gcc musl-dev python3-dev git
CMD [ "sml2mqtt", "--config", "/sml2mqtt/config.yml" ]
git-developer commented 1 year ago

Feedback which is unrelated to the Dockerfile and may be discussed in different issues if you like:

  1. The documentation is not available on https://sml2mqtt.readthedocs.io
  2. sml2mqtt seems to work fine, but I get a lot of CRC errors. This is not related to your rework, I have the same behavior using 1.0.1 or 1.2.2. Example (only relevant lines, extracted from a larger log): 
    sml2mqtt  | [2023-03-03 15:58:27,529] [sml                    ] INFO     | Starting V1.2.2
sml2mqtt  | [2023-03-03 15:58:30,070] [sml.ttyUSB0            ] ERROR    | Crc error: 11849 != 53275
sml2mqtt  | [2023-03-03 15:58:33,611] [sml.ttyUSB0            ] ERROR    | Crc error: 49535 != 50322
sml2mqtt  | [2023-03-03 15:58:39,480] [sml.ttyUSB0            ] ERROR    | Crc error: 673 != 45339
sml2mqtt  | [2023-03-03 15:58:43,420] [sml.ttyUSB0            ] ERROR    | Crc error: 61188 != 14046
sml2mqtt  | [2023-03-03 15:58:48,874] [sml.ttyUSB0            ] ERROR    | Crc error: 22400 != 6939
sml2mqtt  | [2023-03-03 15:58:52,814] [sml.ttyUSB0            ] ERROR    | Crc error: 29658 != 5513
sml2mqtt  | [2023-03-03 15:58:58,689] [sml.ttyUSB0            ] ERROR    | Crc error: 47387 != 27265
sml2mqtt  | [2023-03-03 15:59:01,308] [sml.ttyUSB0            ] ERROR    | Crc error: 9082 != 8987
sml2mqtt  | [2023-03-03 15:59:08,332] [sml.ttyUSB0            ] ERROR    | Crc error: 923 != 7128
sml2mqtt  | [2023-03-03 15:59:10,404] [sml.ttyUSB0            ] ERROR    | Crc error: 8313 != 6939
sml2mqtt  | [2023-03-03 15:59:30,243] [sml.ttyUSB0            ] ERROR    | Crc error: 28554 != 65375
sml2mqtt  | [2023-03-03 15:59:42,872] [sml.ttyUSB0            ] ERROR    | Crc error: 8540 != 28321
sml2mqtt  | [2023-03-03 15:59:45,264] [sml.ttyUSB0            ] ERROR    | Crc error: 37557 != 6939
sml2mqtt  | [2023-03-03 15:59:50,251] [sml.ttyUSB0            ] ERROR    | Crc error: 46102 != 47651
sml2mqtt  | [2023-03-03 15:59:57,538] [sml.ttyUSB0            ] ERROR    | Crc error: 5629 != 6939
sml2mqtt  | [2023-03-03 16:00:01,428] [sml.ttyUSB0            ] ERROR    | Crc error: 45537 != 50127
sml2mqtt  | [2023-03-03 16:00:11,778] [sml.ttyUSB0            ] ERROR    | Crc error: 30536 != 62935
sml2mqtt  | [2023-03-03 16:00:16,270] [sml.ttyUSB0            ] ERROR    | Crc error: 11133 != 47070
sml2mqtt  | [2023-03-03 16:00:19,709] [sml.ttyUSB0            ] ERROR    | Crc error: 59920 != 30158
sml2mqtt  | [2023-03-03 16:00:22,985] [sml.ttyUSB0            ] ERROR    | Crc error: 8160 != 6300
sml2mqtt  | [2023-03-03 16:00:29,549] [sml.ttyUSB0            ] ERROR    | Crc error: 23141 != 7816
sml2mqtt  | [2023-03-03 16:00:36,138] [sml.ttyUSB0            ] ERROR    | Crc error: 29900 != 16508
sml2mqtt  | [2023-03-03 16:00:45,383] [sml.ttyUSB0            ] ERROR    | Crc error: 37362 != 16278
sml2mqtt  | [2023-03-03 16:00:58,543] [sml.ttyUSB0            ] ERROR    | Crc error: 59363 != 33911
sml2mqtt  | [2023-03-03 16:01:05,894] [sml.ttyUSB0            ] ERROR    | Crc error: 34191 != 31949
sml2mqtt  | [2023-03-03 16:01:50,394] [sml.ttyUSB0            ] ERROR    | Crc error: 36195 != 14423
sml2mqtt  | [2023-03-03 16:02:10,465] [sml.ttyUSB0            ] ERROR    | Crc error: 30757 != 31828
sml2mqtt  | [2023-03-03 16:02:16,715] [sml.ttyUSB0            ] ERROR    | Crc error: 3225 != 22940
sml2mqtt  | [2023-03-03 16:02:30,511] [sml.ttyUSB0            ] ERROR    | Crc error: 30765 != 43366
sml2mqtt  | [2023-03-03 16:02:42,341] [sml.ttyUSB0            ] ERROR    | Crc error: 54965 != 16784
spacemanspiff2007 commented 1 year ago

Thank you for your feedback! I did some work on the dockerfile and install the source directly from the git sources. Image size is now at around 30mb. What do you think? Does it still work for you as expected?

  1. The documentation is not available on https://sml2mqtt.readthedocs.io

Preview is here https://sml2mqtt.readthedocs.io/en/rework/

2. sml2mqtt seems to work fine, but I get a lot of CRC errors.

Is your reader properly positioned and what kind of hardware are you using? These errors seem to happen at arbitrary intervals so I am unsure if this is a systematic issue.

git-developer commented 1 year ago

The image works as expected. Building from sources is OK if you don't need custom sources, so you can omit the build-arg.

The image is not 30 MB, it's 91.2 MB (x64) / 81.4 MB (armv7). Changing the base image from python:3.10-alpine to alpine reduces the size down to 49.2 MB (x86) / 34.2 MB (armv7). You can check the size with the command docker image ls | grep sml2mqtt on the build machine.

I found out that the CRC errors occur when the sml device is not used exclusively by sml2mqtt. I have another application connected to the SML device. When I stop this application, the CRC errors are gone. The application might send commands to the device, I'm not sure about it. I will open a new issue if this behavior is a problem for me.

spacemanspiff2007 commented 1 year ago

so you can omit the build-arg.

What do you mean by that?

When I stop this application, the CRC errors are gone

Interesting. Out of curiosity what application is it/what does it do? Does it also read from the serial port?

Changing the base image from python:3.10-alpine to alpine reduces the size down to 49.2 MB (x86) / 34.2 MB (armv7).

I looked at the compressed size on dockerhup and confused it with the image size. grafik

Currently it's unclear how I can pin the python version to python 3.10 with the alpine image. Since it's not clear from the build image which python version is available I'm afraid that it might break and it'll be hard for me to debug. Since I'm not a docker expert I think I'd stick with the provided python3.10 alpine image. But thank you for your support and tips! They are highly appreciated!

git-developer commented 1 year ago

What do you mean by that?

The line ARG SOURCE=sml2mqtt that I suggested. You don't need it when you build from source only.

what application is it/what does it do? Does it also read from the serial port?

The application is solarview-d0, part of a commercial application for monitoring a photovoltaic plant. It uses the same serial port, e.g. /dev/ttyUSB0. It's probably not a good idea to let 2 applications use the same serial port at the same time.

Currently it's unclear how I can pin the python version to python 3.10 with the alpine image.

Ah OK, now I understand why you prefer the python image.

Once your rework and Docker image are in a stable state, I will disable my docker image project because it's not required anymore. Thanks for developing sml2mqtt further!

spacemanspiff2007 commented 1 year ago

I've just published 2.0 on dockerhub

git-developer commented 1 year ago

I disabled my docker image project because it's not required anymore. Thanks!

spacemanspiff2007 commented 1 year ago

Thank you very much for your help and quick feeback!