Thought about conversion to docker?

[psyciknz]

I still use this, have for a couple of years, old version though from when I did the multiple devices code.

I converted it to a docker container a while back. If you’re interested in the docker file

[eschava]

I don't mind converting it to docker and even have a pull request for this But that request moves all recorded commands into the data folder and I don't want to change existing layout of files Does your container change the placement of existing files?

[psyciknz]

I’ll have a look tomorrow, I don’t think so.

[psyciknz]

Docker file here:

FROM __BASEIMAGE_ARCH__/python:2.7.15-jessie

__CROSS_COPY qemu/qemu-__QEMU_ARCH__-static /usr/bin/

# based on https://github.com/pfichtner/docker-mqttwarn

# install python libraries (TODO: any others?)
RUN pip install paho-mqtt broadlink

# build /opt/mqttwarn
RUN mkdir -p /opt/broadlink-mqtt
WORKDIR /opt/broadlink-mqtt
RUN mkdir -p /var/log/broadlink

# add user mqttwarn to image
RUN groupadd -r broadlink && useradd -r -g broadlink broadlink
RUN chown -R broadlink /opt/broadlink-mqtt
RUN chown -R broadlink /var/log/broadlink

# process run as mqttwarn user
USER broadlink

# conf file from host
VOLUME ["/opt/broadlink-mqtt/conf"]

# commands dir
VOLUME ["/opt/broadlink-mqtt/commands/"]

# set conf path
ENV BROADLINKMQTTCONF="/opt/broadlink-mqtt/conf/mqtt.conf"

# finally, copy the current code (ideally we'd copy only what we need, but it
#  is not clear what that is, yet)
COPY . /opt/broadlink-mqtt

# run process
CMD python mqtt.py

It's a weird dockerfile, as I had a custom build process for arm64, arm32 and amd64. I think you can do it now with travis ci building the containers.

I don't think I moved the code, but possibly the conf file location. Yours:

dirname = os.path.dirname(os.path.abspath(__file__)) + '/'
CONFIG = os.getenv('BROADLINKMQTTCONFIG', dirname + 'mqtt.conf')

Mine:

 location = os.path.expanduser(location)
 if os.path.exists(location) and os.path.isfile(location + '/conf/mqtt.conf'):
        print "found config at " + location
        dirname = location

[mebrew17]

Sorry if I am butting in on anyone but I a noobie and am trying to get BG1 mqtt working. Can someone please point me in the direction of a forum or something to show me the way.

thank you

[digginsa]

Has anyone tried BG1 with mqtt all the current guides no longer work?

[psyciknz]

@eschava hello, back again!!

I've redone the process now, and using volume maps to the mqtt.conf file directly so it doesn't have to move in the projest file..

I do have some code alterations around a log file, but it might not be important, I think I can lose them. I have the container (amd64 only) on docker hub: https://hub.docker.com/u/psyciknz

[eschava]

Hi!

Do I need only the Dockerfile file from your branch?

[psyciknz]

Yeah probably. I did make some code changes relating to logging in the mqtt.py. But suspect they're not needed.

Also my dockerfile was for python 3 (some print adjustments).

I've got a compose file that can show the volume mounting for the conf file and commands which we can add to the readme if you want.

[eschava]

added. Thanks for your efforts!

[psyciknz]

So the next step, if we confirm that's working is getting dockerhub to automatically build it.

I might kill my branch, and pull from master (is that where the added docker file is?). The I can test the dockerfile and show you the docker hub config for build

[tlc76]

Does your image support Raspberry Pi 4 (with Raspberry OS 32bit) architecture? This is the error I get when I pull the image:

pi@pi4:~ $ docker pull psyciknz/broadlink-mqtt
Using default tag: latest
Error response from daemon: manifest for psyciknz/broadlink-mqtt:latest not found: manifest unknown: manifest unknown

Thanks

[psyciknz]

Possibly not at the moment as it is built with docker hub. And while I used to have a travis integration for another project that did build manifests - a travis update broke the emulation need to build on a pi.

Unfortunately you might need to pull the source and build the image on a pi.

[tlc76]

I've had some unsuccessful attempts to compile @eschava's broadlink-mqtt as a docker image. It always fails when running build_rust:

  writing manifest file 'src/cryptography.egg-info/SOURCES.txt'
  copying src/cryptography/py.typed -> build/lib.linux-armv7l-3.7/cryptography
  running build_ext
  generating cffi module 'build/temp.linux-armv7l-3.7/_padding.c'
  creating build/temp.linux-armv7l-3.7
  generating cffi module 'build/temp.linux-armv7l-3.7/_openssl.c'
  running build_rust

      =============================DEBUG ASSISTANCE=============================
      If you are seeing a compilation error please try the following steps to
      successfully install cryptography:
      1) Upgrade to the latest pip and try again. This will fix errors for most
         users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
      2) Read https://cryptography.io/en/latest/installation.html for specific
         instructions for your platform.
      3) Check our frequently asked questions for more information:
         https://cryptography.io/en/latest/faq.html
      4) Ensure you have a recent Rust toolchain installed:
         https://cryptography.io/en/latest/installation.html#rust
      5) If you are experiencing issues with Rust for *this release only* you may
         set the environment variable `CRYPTOGRAPHY_DONT_BUILD_RUST=1`.
      =============================DEBUG ASSISTANCE=============================

  error: can't find Rust compiler

  If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.

  To update pip, run:

      pip install --upgrade pip

  and then retry package installation.

  If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.

  This package requires Rust >=1.41.0.
  ----------------------------------------
  ERROR: Failed building wheel for cryptography
  Building wheel for paho-mqtt (setup.py): started
  Building wheel for paho-mqtt (setup.py): finished with status 'done'
  Created wheel for paho-mqtt: filename=paho_mqtt-1.5.1-py3-none-any.whl size=61546 sha256=f1e9d46f989f73d16d25843d4a7e8d59cda909530995abf0cebe86986a71ec0a
  Stored in directory: /root/.cache/pip/wheels/c9/be/2a/883db47312c70ef7ffcaff281f3294e3075f62da075474d4bb
Successfully built broadlink paho-mqtt
Failed to build cryptography
ERROR: Could not build wheels for cryptography which use PEP 517 and cannot be installed directly
The command '/bin/sh -c pip install -r /opt/broadlink-mqtt/requirements.txt' returned a non-zero code: 1

Full build output here:

broadlink-mqtt_docker_compile_error.txt

I managed to get over this by adding the following 2 lines in Dockerfile

ARG CRYPTOGRAPHY_DONT_BUILD_RUST=1
RUN apt-get install libffi-dev

(I am not sure if the libffi-dev libraries are really necessary, but the env. variable CRYPTOGRAPHY_DONT_BUILD_RUST=1 is necessary for sure to avoid the attempt to compile rust). The image has compiled successfully.

The container obviously does not start without the configuration files:

pi@pi4:~ $ docker logs broadlink-mqtt-tlc-20210409
Cannot load configuration from file /opt/broadlink-mqtt/mqtt.conf: [Errno 2] No such file or directory: '/opt/broadlink-mqtt/mqtt.conf'

Now I am in the process of mapping the external configuration files, but they are in the same place as the binaries, so probably the mounts must be done at file level and not at directory/volume level. (each of the config files: mqtt.conf, logging.conf, custom.conf should be mounted separately in the container). How about mqtt.log?

Yeah probably. I did make some code changes relating to logging in the mqtt.py. But suspect they're not needed.

Also my dockerfile was for python 3 (some print adjustments).

I've got a compose file that can show the volume mounting for the conf file and commands which we can add to the readme if you want.

@psyciknz : Can you please share your docker-composer.yml file that you've been mentioning above?

thanks a lot

[psyciknz]

Ok. I might be able to look at adding that of @eschava can (or you if you want to make a pull request.

Not sure I'll be able to format this (GitHub on the phone). But here's my compose file.

version: '2.2'

services:
  broadlink-mqtt2:
    container_name: broadlink-mqtt2
    hostname: broadlink2.iot.andc.nz
    environment:
      - ARCH=amd64
      - BASEPATH=/hdd/docker-data
    image: psyciknz/broadlink-mqtt:${ARCH}-0.0.5
    restart: always
    dns:
      - 192.168.10.1
    dns_search: iot.andc.nz
    networks:
      - iot.andc.nz
    volumes:
      - ${BASEPATH}/ha-iot/broadlink/conf/mqtt.conf:/opt/broadlink-mqtt/mqtt.conf
      - ${BASEPATH}/ha-iot/broadlink/commands:/opt/broadlink-mqtt/commands
      - ${BASEPATH}/ha-iot/broadlink/macros:/opt/broadlink-mqtt/macros
      - "/etc/timezone:/etc/timezone:ro"
      - "/etc/localtime:/etc/localtime:ro"

Hope it helps. I'm away for the next week but can look into this once back, as I have another python service I used to multi arch build via travis but it stopped working. Maybe they've fixed the problem.

[tlc76]

Thank you! I adapted your file to my environment. When I start the container, I get the following error:

pi@pi4:~ $ docker logs broadlink-mqtt-tlc-20210409
Configuration parameter 'logdir' should be specified

I wasn't able to find this parameter in the configuration template files nor in the broadlink-mqtt documentation... I've tried adding the line logdir = /opt/broadlink in file custom.conf but I get the same error message.

Any ideas on how to solve this?

thanks!

[psyciknz]

I Have that in the MQTT.conf

I’m not sure how custom.conf relates to MQTT.conf. And depending how you’re doing your compose file if it’s being picked up.

[tlc76]

I mapped the custom.conf file in docker-compose.yml:

version: '3.7'

services:
  broadlink-mqtt-tlc:
    image: "broadlink-mqtt-tlc-20210410"
    container_name: "broadlink-mqtt-tlc-20210409"
    environment:
      - TZ=Europe/Bucharest
    restart: always
    network_mode: host
    volumes:
      - /opt/broadlink-mqtt/conf/mqtt.conf:/opt/broadlink-mqtt/mqtt.conf
      - /opt/broadlink-mqtt/conf/custom.conf:/opt/broadlink-mqtt/custom.conf
      - /opt/broadlink-mqtt/conf/broadlink.log:/opt/broadlink-mqtt/broadlink.log
      - /opt/broadlink-mqtt/commands:/opt/broadlink-mqtt/commands
      - /opt/broadlink-mqtt/macros:/opt/broadlink-mqtt/macros
      - "/etc/timezone:/etc/timezone:ro"
      - "/etc/localtime:/etc/localtime:ro"

I did some more investigation and it seems that the custom.conf file is ignored. Back when I was using broadlink-mqtt directly on my RPi, I used custom.conf to overwrite settings in mqtt.conf file. This is by design.

This explains why adding the logdir parameter in custom.conf didn't work.

I did all configuration directly in mqtt.conf and the container is starting correctly.

However I am still not able to execute commands stored in commands directory via mqtt:

2021-04-11 01:56:25,111 DEBUG Received MQTT message broadlink/tv/philips_old/power on
2021-04-11 01:56:25,113 WARNING Unrecognized MQTT message

Still have to do some more troubleshooting. I got the same error also when running the broadlink-mqtt instance on the physical machine (which is now disabled).

Has the way of running commands changed in broadlink-mqtt? I am running commands like this, using an mqtt client application:

• mqtt topic: \<path to recorded infrared command file>
• mqtt message payload : \<empty message>

Example:

• mqtt topic: broadlink/tv/philips_old/power the file power exists in /opt/broadlink-mqtt/commands/tv/philips_old/ and is readable from within the container.

Thank you!