Docker image doesn't work with Linak desk

[ipatalas]

Describe the bug When trying to start a docker container with only Linak desk configured I'm getting error and the container fails to start.

To Reproduce Steps to reproduce the behavior:

Config file:

mqtt:
  host: mosquitto
  port: 1883
  topic_prefix: bluetooth
  client_id: bt-mqtt-gateway
  availability_topic: lwt_topic

manager:
  sensor_config:
    topic: homeassistant
    retain: true
  topic_subscription:
    update_all:
      topic: homeassistant/status
      payload: online
  workers:
    linakdesk:
      args:
        mac: VALID_MAC_ADDRESS_OF_MY_DESK
        topic_prefix: linak_desk
      update_interval: 300

Then run the following command: docker run --name bt2mqtt -v $PWD/config:/config zewelor/bt-mqtt-gateway

Expected behavior The container should start successfully.

Debug gateway logs

fido@home: ~/docker/bt2mqtt
$ docker run --name bt2mqtt -v $PWD/config:/config zewelor/bt-mqtt-gateway
Start in normal mode
18:40:02 Starting
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
  ERROR: Error [Errno 2] No such file or directory: 'git': 'git' while executing command git clone -q https://github.com/zewelor/linak_bt_desk.git /tmp/pip-install-tee1xae6/linakdpgbt
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?
Traceback (most recent call last):
  File "./gateway.py", line 67, in <module>
    manager.register_workers(global_topic_prefix).start(mqtt)
  File "/application/workers_manager.py", line 91, in register_workers
    command_timeout, global_topic_prefix, **worker_config["args"]
  File "/application/workers/base.py", line 7, in __init__
    self._setup()
  File "/application/workers/linakdesk.py", line 20, in _setup
    from linak_dpg_bt import LinakDesk
ModuleNotFoundError: No module named 'linak_dpg_bt'

Server (please complete the following information):

• OS: Linux
• Distro: Debian 9 (Stretch)
• Version: 30 December 2018

Additional context

It looks like it's trying to download the library to handle Linak desk but git is not available.

[ipatalas]

Did some digging and found out why the docker image is failing. Dockerfile first install git and then uninstalls it: https://github.com/zewelor/bt-mqtt-gateway/blob/90273d09a89339593c6a519629e949ae27ecf766/Dockerfile#L23-L28

I've cloned the repo, changed the line not to remove git and ran again. Almost there.

My desk still cannot be found. I'm setting it in pair mode - I guess that's correct? Or do I have to pair it with the machine first? I was able to run the gateway in debug mode:

2020-01-25 19:52:25,955 INFO bt-mqtt-gw gateway.py:61:<module> - Starting
2020-01-25 19:52:25,955 DEBUG bt-mqtt-gw.mqtt mqtt.py:30:__init__ - Setting LWT to: bluetooth/lwt_topic
2020-01-25 19:52:26,307 DEBUG bt-mqtt-gw.workers_manager workers_manager.py:106:register_workers - Added linakdesk worker with 30 seconds interval and a 35 seconds timeout
2020-01-25 19:52:26,355 DEBUG bt-mqtt-gw.mqtt mqtt.py:116:callbacks_subscription - Subscribing to: bluetooth/linak_desk/update_interval
2020-01-25 19:52:26,355 DEBUG bt-mqtt-gw.mqtt mqtt.py:116:callbacks_subscription - Subscribing to: bluetooth/homeassistant/status
2020-01-25 19:52:26,356 DEBUG bt-mqtt-gw.workers_manager workers_manager.py:174:update_all - Updating all workers
2020-01-25 19:52:26,362 ERROR linak_dpg_bt.connection connection.py:50:__enter__ - Second connection try to EC:F7:90:55:37:81 failed: Failed to connect to peripheral EC:F7:90:55:37:81, addr type: random
2020-01-25 19:52:26,363 ERROR bt-mqtt-gw.workers.linakdesk logger.py:50:log_exception - Error during update of linak desk 'linakdesk' (EC:F7:90:55:37:81): BTLEDisconnectError
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/linak_dpg_bt/connection.py", line 44, in __enter__
    self._conn.connect(self._mac, addrType='random')
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 445, in connect
    self._connect(addr, addrType, iface)
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 439, in _connect
    raise BTLEDisconnectError("Failed to connect to peripheral %s, addr type: %s" % (addr, addrType), rsp)
bluepy.btle.BTLEDisconnectError: Failed to connect to peripheral EC:F7:90:55:37:81, addr type: random

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/application/workers/linakdesk.py", line 43, in _get_height
    self.desk.read_dpg_data()
  File "/usr/lib/python3.6/site-packages/linak_dpg_bt/linak_device.py", line 74, in read_dpg_data
    with self._conn as conn:
  File "/usr/lib/python3.6/site-packages/linak_dpg_bt/connection.py", line 48, in __enter__
    self._conn.connect(self._mac, addrType='random')
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 445, in connect
    self._connect(addr, addrType, iface)
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 439, in _connect
    raise BTLEDisconnectError("Failed to connect to peripheral %s, addr type: %s" % (addr, addrType), rsp)
bluepy.btle.BTLEDisconnectError: Failed to connect to peripheral EC:F7:90:55:37:81, addr type: random
2020-01-25 19:52:26,364 ERROR bt-mqtt-gw logger.py:50:log_exception - Timeout while executing worker command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/linak_dpg_bt/connection.py", line 44, in __enter__
    self._conn.connect(self._mac, addrType='random')
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 445, in connect
    self._connect(addr, addrType, iface)
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 439, in _connect
    raise BTLEDisconnectError("Failed to connect to peripheral %s, addr type: %s" % (addr, addrType), rsp)
bluepy.btle.BTLEDisconnectError: Failed to connect to peripheral EC:F7:90:55:37:81, addr type: random

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/application/workers/linakdesk.py", line 43, in _get_height
    self.desk.read_dpg_data()
  File "/usr/lib/python3.6/site-packages/linak_dpg_bt/linak_device.py", line 74, in read_dpg_data
    with self._conn as conn:
  File "/usr/lib/python3.6/site-packages/linak_dpg_bt/connection.py", line 48, in __enter__
    self._conn.connect(self._mac, addrType='random')
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 445, in connect
    self._connect(addr, addrType, iface)
  File "/usr/lib/python3.6/site-packages/bluepy/btle.py", line 439, in _connect
    raise BTLEDisconnectError("Failed to connect to peripheral %s, addr type: %s" % (addr, addrType), rsp)
bluepy.btle.BTLEDisconnectError: Failed to connect to peripheral EC:F7:90:55:37:81, addr type: random

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./gateway.py", line 73, in <module>
    mqtt.publish(_WORKERS_QUEUE.get(timeout=10).execute())
  File "/application/workers_manager.py", line 58, in execute
    messages = self._callback(*self._args)
  File "/application/workers/linakdesk.py", line 27, in status_update
    topic=self.format_topic("height/cm"), payload=self._get_height()
  File "/application/workers/linakdesk.py", line 54, in _get_height
    raise DeviceTimeoutError
exceptions.DeviceTimeoutError

Can't find anything useful in the stacktrace. Any clue what I might be doing wrong?

[zewelor]

Try to pair with desk: https://wiki.archlinux.org/index.php/Bluetooth#Pairing

[ipatalas]

That's a good point... I did that, also shared devices between host and container but still the gateway does not see the device even though the container does:

The left: bluetoothctl ran on the container easily seeing paired desk The right: gateway logs

That's closer but still not there yet. Any other thoughts?

[zewelor]

Maybe phone app also connected ? No range ? Maybe it would be good to test without container, if it can connect. Maybe pairing needs something saved on host, and when inside container, its unpaired. ( I don't remember if I had to pair. For more than a year I didn't touch it )

[ipatalas]

Range is not an issue for sure, devices are 50 cm of each other. Anyway, host machine was able to connect to it so I guess it must be something related to docker.

I was trying to do the same without docker but no luck as well. I'm not very familiar with Python and when running ./gateway.py in virtualenv I'm getting this:

(bt2mqtt) fido@home: ~/bt-mqtt-gateway
$ sudo ./gateway.py
Traceback (most recent call last):
  File "./gateway.py", line 11, in <module>
    import logger
  File "/home/fido/bt-mqtt-gateway/logger.py", line 3, in <module>
    import yaml
ImportError: No module named 'yaml'

There is no manual to install any packages but still there are some missing. However I've got this package installed:

$ pip show pyyaml
Name: PyYAML
Version: 5.3
Summary: YAML parser and emitter for Python
Home-page: https://github.com/yaml/pyyaml
Author: Kirill Simonov
Author-email: xi@resolvent.net
License: MIT
Location: /home/fido/bt2mqtt/lib/python3.5/site-packages
Requires:

[zewelor]

https://github.com/zewelor/bt-mqtt-gateway#virtualenv

[ipatalas]

That's exactly what I have been using to install everything.

[mountainsandcode]

@ipatalas : If that doesn't work, try sudo-installing the pip requirements

[zewelor]

I've made few fixes in docker setup. It should install all now.

[ipatalas]

Oh, that's great. I'll give it a shot tomorrow. Thanks!

[ipatalas]

All right, it's better because it installs correctly but still the same problem persists.

The desk is paired and connected to the host (left side) but container cannot access it:

When I sh into the container, run bluetoothctl and get device info, it shows it's connected (same output as from the host - left part of the screenshot) so looks like the container itself sees the device but the gateway for some reason can not.

Do you know any Python sample app that uses the same lib for BT so that I can try something different than the gateway and try to troubleshoot?

What bothers me is that the exception from the gateway says BTLEDisconnectError. That's a strong suggestion it tries connecting via BTLE but my desk is using regular BT rather than BTLE. hcitool lescan fails on my machine (I/O error) even though it's an Intel NUC which is supporting BT 5.0 according to the specs so that shouldn't be a problem. Anyway it's connected with the desk so it shouldn't be a problem.

[Masch67]

Hello,

I've installed the venv bt to mqtt gateway and it works for a Xiaomi LYWSD03MMC sensor. I tried it out with 3 sensors but had the same problems as yvon-indel (cf. issue #169). Now I'm trying to use the docker installation on my raspberry pi zero w. The image is downloaded, container is created starts for a few seconds and stops with exit code 139.

I'm using the same config.yaml as in venv.

I find nothing in container logs, so has anyone an idea ?

[darkkatarsis]

All right, it's better because it installs correctly but still the same problem persists.

The desk is paired and connected to the host (left side) but container cannot access it:

When I sh into the container, run bluetoothctl and get device info, it shows it's connected (same output as from the host - left part of the screenshot) so looks like the container itself sees the device but the gateway for some reason can not.

Do you know any Python sample app that uses the same lib for BT so that I can try something different than the gateway and try to troubleshoot?

What bothers me is that the exception from the gateway says BTLEDisconnectError. That's a strong suggestion it tries connecting via BTLE but my desk is using regular BT rather than BTLE. hcitool lescan fails on my machine (I/O error) even though it's an Intel NUC which is supporting BT 5.0 according to the specs so that shouldn't be a problem. Anyway it's connected with the desk so it shouldn't be a problem.

Same here in none docker version. Raspberry Zero W. bluetoothctl connected but have BTLEDisconnectError

[mountainsandcode]

Are you using an IKEA desk by any chance? I realized that I somehow cannot get it to subscribe to changes (perhaps not supported by the controller) - see https://github.com/zewelor/bt-mqtt-gateway/issues/144#issuecomment-587889537

[darkkatarsis]

I ordered a desk directly from Linak with the dpg1c module connection:

When I run through the gateway, it works amazingly:

But when im running systemctl bt-mqtt-gateway:

[darkkatarsis]

It began to work after putting the environment from scratch. Is it possible to control the desk height via mqtt or only read the status?

[twsl]

has anyone managed to connect to the ikea idasen desk?

[maxime1992]

@twsl I have not with this lib and decided to take a different path, which eventually worked out perfectly.

I've created a project to dockerize an MQTT wrapper which uses a python script connecting to the desk in bluetooth. It works great for me and I've been able to fully integrate that solution into my Home Assistant setup. Here's the project if you're interested https://github.com/maxime1992/linak-2-mqtt

Note: I've mentionned Home Assistant but it could be used with anything really, it just exposes some MQTT endpoints

[ipatalas]

@twsl I have not with this lib and decided to take a different path, which eventually worked out perfectly.

I've created a project to dockerize an MQTT wrapper which uses a python script connecting to the desk in bluetooth. It works great for me and I've been able to fully integrate that solution into my Home Assistant setup. Here's the project if you're interested https://github.com/maxime1992/linak-2-mqtt

Thanks! I'll give it a shot when I have some time over the weekend. I have already gave up with this bt-mqtt-gateway as this already took too much time and still not working. Maybe this one will. MQTT is even better for me as it's more universal, great idea @maxime1992 .