Failed to start BSEC-Conduit Daemon

[azrael783]

I tried to setup your code but stuck at the end with to above mentioned message after using: sudo systemctl start bsec-conduit.service; journalctl -f -u bsec-conduit.service

I am trying to summarize my setup: I have HA installed on a Pi 3+ using the "Manual installation on a Raspberry Pi" from the HA docs page. Due to this installation my homeassistant user doesn't have a password, so he can't run sudo commands (I am fairly new to the Pi / Linux stuff). Therefore I did running the intall.py as user pi. I run a MQTT broker on my QNAP NAS, no need for user and passoword for that.
I did not recognize that I need the python-systemd & paho.mqtt so I went thru the installation getting the error message from above. After that I installed the pyhton-sytemd & paho.mqtt using the pip3 command (apt-get didn't work) getting the same error message again. Let me know if I should try using the internal HA MQTT broker, or if you need some more information. Would be great if we could get it running on my Pi :-) I really apreciate your work! Thanks again.

[timothybrown]

Alright, installing as the pi user should be fine. What OS are you running on the Pi? If it's Raspbian or Ubuntu, then apt-get should have worked. However, if you installed the modules directly with pip3 it should still be working. So, let's see if we can find out what's going on...

It looks like it's not finding the paho.mqtt module. As the pi user, run pip3 list and see if paho.mqtt and python-systemd are showing up. (In fact, paste the entire output here for me if you can.)

[azrael783]

I used a special Raspbian image from Dresden Elektronik (I'm using their ZigBee stick). See: https://www.dresden-elektronik.de/funktechnik/solutions/wireless-light-control/raspbee-gw-sd-card-image/?L=1. But it is based on Raspbian. Unfortunately my SD card broke again and I have to set it up again. This time I'm using a HDD instead of the SD card. As soon as I am back running I will give it a try again.

[timothybrown]

Okay, no worries. One of the solutions to your problem is that I'm rewriting the install script a bit. I think I'm also going to change the instructions a so that we install into a Python virtual environment, that way you can use PIP to install the correct modules separately from your system-installed modules.

To facilitate this I'm packaging the underlying bseclib Python library up into a real module that will be available on pip as well.

Basically the instructions will be something like this (you'd replace HAUSER with the username you made for HA, usually hass or homeassistant):

sudo mkdir /opt/bsec
sudo git clone https://github.com/timothybrown/BSEC-Conduit.git /opt/bsec
sudo python3 -m venv /opt/bsec
sudo chown -R HAUSER:HAUSER /opt/bsec
cd /opt/bsec
source bin/activate
sudo -u HAUSER pip3 install systemd-python paho.mqtt bseclib
sudo ./install.py

(If you're wondering, the sudo -u HAUSER command lets you run a command as the HA user, even though you've got it setup as a system user without a login. I thought I'd point that out, as it's a handy command for you to know when dealing with HA.)

Basically, we've only added one or two commands compared to the existing instructions, and it will allow me to simplify the install script (which will have the sole job of downloading the BSEC source code and setting up the Systemd service).

I should have my repo updated by tonight with the new code. I'll update this thread when it's there. :)

I'll update

[azrael783]

Hey, I had again problems with my Pi and the HDD / SD cards. I could manage to get the ZigBee gateway back up running - which is the most important thing, cause we are already using ZigBee lights which won't work without the gateway. Due to the problems I had I decided to split my system. Means the Pi is only running the gateway and HA is now running on my NAS in a VM. I hope this will prevent to brake again my ZigBee gateway. Nevertheless it should be also possible to have the BME 680 set up in a the virtual venv, or? I just use the regular Pi user or I can create a special user only for this ... Sorry for all this confusion, but my wife kills me if the lights won't work ... Btw: Since I have now a fresh new Pi I will give it a try again, to see if the original version will work. I forked your repo so you can go ahead with your development (just in cause it still won't work :-) )

[timothybrown]

Yes, you can still set it up in a venv if you want (I prefer this approach as it isolates things) but it’s up to you. No need to add a dedicated user, running as pi will work just fine. (As long as the user is in the I2C group it’ll work: You can find out by running groups pi.)

Oh, yeah, speaking of which, make sure you run sudo raspi-config and enable the I2C interface as well. I need to add a check for that into the installer script too.

Before you run the installer script, make sure you run sudo apt-get update so that the apt package list is up to date. I think there’s an issue with how the installer is trying to use apt to install the Python dependencies.

Thanks for your help testing this!

By the way, the updated version I’m about to post should be used if possible, there’s two critical fixes in the bsec-conduit code itself.

[azrael783]

Ok, will give it try at the weekend I think. Thank's again for your great work!

[azrael783]

Hey, a little later, but I tried it again. This time I stuck at the installation. Here is the log:

I followed your commands and everything went well until this point. I tried it a second time switching into the venv using the command:

sudo source bin/activate

but I got the same result.

[timothybrown]

Okay, what happens when you switch into the venv (source bin/activate) and run pip install paho-mqtt?

I appreciate you helping to debug this. I have a feeling this is something simple and we’re close to tracking it down.

[timothybrown]

And I see one problem with the install script already. It looks like I should be trying to catch ImportErrors in addition to ModuleNotFoundErrors.

Okay, I fixed that error in the install script. Try re-downloading it and running it again. :)

[timothybrown]

@azrarel783 Okay, so I just uploaded an even better of the install script.

Now, if you're running inside a venv we will automatically attempt to install the required modules with PIP. We also check to make sure I2C is enabled on the Pi and, if not, enable it.

Try running this script from your venv and let me know how it goes. :)

[azrael783]

Thank's again for your great work! I will test it this evening (to bad that we are sitting in two different time zones). ;-)

[azrael783]

Ok, so installation went throug now. But it still doesn't work. I think also that we are getting closer. I now stumple on the last command getting this message:

Doesn't matter if I'm using it from user "pi" or inside "venv".

[timothybrown]

Ah, looks like the systemd service file is missing line breaks some places. I might have broken that in my last update. I shall investigate and get back to you shortly. :)

[timothybrown]

Okay, I'm implementing a fix for the installer now, in the mean time, I'm attaching a new systemd service file for you. You can install it like this: sudo systemctl edit --full bsec-conduit.service Delete the current contents of the file (you can press CTRL-K to delete entire lines at a time) and then paste this in:

[Unit]
Description=BSEC-Conduit Daemon
StartLimitBurst=5
StartLimitIntervalSec=30

[Service]
Type=notify
User=pi
WorkingDirectory=/opt/bsec
ExecStart=/opt/bsec/bin/python /opt/bsec/bsec-conduit
WatchdogSec=30s
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Now, press CTRL-X then Y then <Enter> to save it. Now try starting it again.

[azrael783]

I tried and failed with the initial error (no module paho found) ...

[timothybrown]

Okay, that’s very strange... If installation completed before, the module has to be there.

What time zone are you in? I think the quickest way to get this solved would be to get on some form of live chat so I can walk you through some stuff in real time. I can use whatever is handy for you, (Skype, iMessage, SMS, Discord or whatever you use). Just let me know your time zone and a good time to get in touch. :)

[m4dm4xi]

I´d like to jump onto the train.

I have the exact same problem.

Same symptoms as shown in the inital post screenshot.

I have applied the patch you posted a few hours ago.

I too have not used the venv but installed as user pi.

///////////////////////////////////// pi@hassbian:~ $ cat /etc/os-release PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)" NAME="Raspbian GNU/Linux" VERSION_ID="9" VERSION="9 (stretch)" /////////////////////////////////////

Newest Installation files from your git.

i2cdetect -y 1< results in i2c device 77.

[timothybrown]

@m4dm4xi Okay, so you're not running in a venv, correct?

In that case, the installer script doesn't install the required dependancies. If you're using the system Python 3 from Raspbian, you'll need to run these two commands:

sudo apt-get install python3-systemd python3-pip sudo pip3 install paho-mqtt

(Unfortunately, paho-mqtt isn't available as an APT package in Stretch; though it'll be there in Raspbian 10 (Buster); so, we use pip to install it.)

Also, you'll need to change a line in the above patched systemd service file. Change the ExecStart= line to read: ExecStart=/usr/bin/python3 /opt/bsec/bsec-conduit You can do this by running: sudo systemctl edit --full bsec-conduit.service

Then try to run the program again and let me know.

Alternatively, you can download the newest version of my install.py script and run it inside a venv.

cd /opt/bsec
git pull
python3 -m venv .
source bin/activate
sudo ./install.py

That will download the Python modules for you and fix the systemd script.

[timothybrown]

I've updated the install script to fix the issue with the systemd service file. It now writes it correctly. Also, I found a problem where the install script may overwrite your /boot/config.txt file under certain circumstances. (That's what I get for writing code at 4AM... I mistakenly had it opening the file for write, instead of append.) If you already had I2C enabled (dtparam=i2c_arm=on) then this wouldn't affect you.

I've fixed that bug, and also added code to make a copy of the original file as /boot/config.bsec before making any changes. My apologies if this affected anyone! Here is the original config.txt file as it ships with Raspbian, in case anyone needs to restore it. (Make sure to leave the dtparam=i2c_arm=on line.)

[m4dm4xi]

@m4dm4xi Okay, so you're not running in a venv, correct?

In that case, the installer script doesn't install the required dependancies. If you're using the system Python 3 from Raspbian, you'll need to run these two commands:

sudo apt-get install python3-systemd python3-pip sudo pip3 install paho-mqtt

(Unfortunately, paho-mqtt isn't available as an APT package in Stretch; though it'll be there in Raspbian 10 (Buster); so, we use pip to install it.)

Also, you'll need to change a line in the above patched systemd service file. Change the ExecStart= line to read: ExecStart=/usr/bin/python3 /opt/bsec/bsec-conduit You can do this by running: sudo systemctl edit --full bsec-conduit.service

Then try to run the program again and let me know.

Alternatively, you can download the newest version of my install.py script and run it inside a venv.

cd /opt/bsec
git pull
python3 -m venv .
source bin/activate
sudo ./install.py

That will download the Python modules for you and fix the systemd script.

I now get this Error:

[timothybrown]

Huh. That's strange. It looks like the wrong systemd Python module is installed. (There are several different ones, confusingly.)

If you used apt-get install python3-systemd or pip3 install systemd-python it should be working.

Let's try this... Paste me the output of pip3 list. You should see something like this:

(bsec) timb@testipi ~/bsec> pip3 list
Package        Version
-------------- -------
paho-mqtt      1.4.0
systemd-python 234

[m4dm4xi]

pi@hassbian:~ $ pip3 list Traceback (most recent call last): File "/usr/bin/pip3", line 9, in from pip import main ImportError: cannot import name 'main'

and

pi@hassbian:~ $ sudo pip3 list DEPRECATION: The default format will switch to columns in the future. You can us e --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.con f under the [list] section) to disable this warning. cryptography (1.7.1) idna (2.2)
keyring (10.1)
keyrings.alt (1.3) paho-mqtt (1.4.0)
pip (9.0.1) pyasn1 (0.1.9) pycrypto (2.6.1) pygobject (3.22.0) python-apt (1.1.0b5) python-systemd (0.0.9) pyxdg (0.25) SecretStorage (2.3.1) setuptools (33.1.1) six (1.10.0) systemd-python (233) unattended-upgrades (0.1) wheel (0.29.0)

but furthermore:

pi@hassbian:~/.local/bin $ ./pip3 list
Package             Version
------------------- ---------
aiohttp             3.3.2
asn1crypto          0.24.0
astral              1.6.1
async-timeout       3.0.0
attrs               18.1.0
certifi             2018.8.13
cffi                1.11.5
chardet             3.0.4
cryptography        2.3.1
homeassistant       0.76.0
idna                2.7
idna-ssl            1.1.0
Jinja2              2.10
keyring             10.1
keyrings.alt        1.3
MarkupSafe          1.0
multidict           4.3.1
paho-mqtt           1.4.0
pip                 18.0
pyasn1              0.1.9
pycparser           2.18
pycrypto            2.6.1
pygobject           3.22.0
PyJWT               1.6.4
python-apt          1.1.0b5
python-systemd      0.0.9
pytz                2018.5
pyxdg               0.25
PyYAML              3.13
requests            2.19.1
SecretStorage       2.3.1
setuptools          33.1.1
six                 1.11.0
systemd-python      233
unattended-upgrades 0.1
urllib3             1.23
voluptuous          0.11.5
wheel               0.29.0
yarl                1.2.6
You are using pip version 18.0, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

[azrael783]

@timothybrown: Sorry wasn't at home over the weekend. I am living in Germany (Timezone=Europe/Berlin) ;-) I will give it a try again this week. I think best time for a live chat will be at the weekend due to the bigger time difference.

[azrael783]

Ok, seems to be that my BME 680 is broken. Now I am getting:

(bsec) pi@phoscon:/opt/bsec $ sudo systemctl start bsec-conduit.service; journalctl -f -u bsec-conduit.service -- Logs begin at Thu 2016-11-03 17:16:42 UTC. -- Nov 27 22:17:38 phoscon BSEC-Conduit[23245]: BSEC-Conduit v0.3.4 Nov 27 22:17:38 phoscon BSEC-Conduit[23245]: Connected to MQTT Broker. Nov 27 22:17:39 phoscon BSEC-Conduit[23245]: Found existing BSEC-Library executable, skipping build. Nov 27 22:17:39 phoscon BSEC-Conduit[23245]: Using existing BSEC-Library configuration [generic_33v_3s_28d]. Nov 27 22:17:39 phoscon BSEC-Conduit[23245]: Found existing BSEC-Library state file, skipping creation. Nov 27 22:17:40 phoscon python3[23245]: user_i2c_write: Remote I/O error Nov 27 22:17:40 phoscon BSEC-Conduit[23245]: BSEC-Library started. Nov 27 22:17:40 phoscon systemd[1]: Started BSEC-Conduit Daemon. Nov 27 22:17:40 phoscon BSEC-Conduit[23245]: BSEC-Library ran out of data to yield! Nov 27 22:17:40 phoscon BSEC-Conduit[23245]: BSEC-Library encountered an unhandled exception. Terminating.

After the systemctl command. And i2cdetect -y 1 is giving this back:

pi@phoscon:~ $ i2cdetect -y 1 0 1 2 3 4 5 6 7 8 9 a b c d e f 00: -- -- -- -- -- -- -- -- -- -- -- -- -- 10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 70: -- -- -- -- -- -- -- --

So I think my BME is broken :-(

[azrael783]

Finally it is working! Almost, I see the entities in Home Assistant, but the values are missing. I will close this issue, because I think it is more a HA topic. I can see the values in MQTT, so I think I should be able to make them also available in HA. Thank you so much for your work! I really appreciate it!