Undocumented requirement for Python 3.6.x due to use of Fstrings

[mlava]

Describe the Bug

Hi. Sorry for this noob issue, but I'm struggling to simply get past step 1!

I've installed using the Linux instructions and updated config files etc. If I try to run the python file wyzesense2mqtt.py I get the following error:

File "wyzesense2mqtt.py", line 30
    print(f"File error: {str(error)}")
                                    ^
SyntaxError: invalid syntax

I vim into the file and see that is related to IO errors. I therefore wondered if it isn't finding my dongle.

lsusb shows my usb_dongle as:

Bus 001 Device 005: ID 1a86:e024 QinHeng Electronics`

I therefore changed the python file as shown:

   if (CONFIG['usb_dongle'].lower() == "auto"):
        device_list = subprocess.check_output(
                ["ls", "-la", "/sys/class/qinheng"]
        ).decode("utf-8").lower()
        for line in device_list.split("\n"):
            if (("e024" in line) and ("1a86" in line)):
                for device_name in line.split(" "):
                    if ("QinHeng" in device_name):
                        CONFIG['usb_dongle'] = "/dev/%s" % device_name
                        break

It appears my dongle has the same ID strings as the config (e024 and 1a86) but this still throws the same error.

I'm not sure what else to try!

Desktop

RPiW0 Raspbian GNU/Linux 8 (jessie)

Any advice would be gratefully received!

[raetha]

Hi @mlava. Based on the error, it looks like it is having trouble parsing one of the YAML config files. If you look just above at line 17 of the script it gives the names that are expected for the config files, along with that it's looking for them in a "config" subfolder. The sensors.yaml file doesn't actually need to exist, but logging.yaml and config.yaml do need to exist. I'm not sure why it's giving a syntax error though. Could you be running this under Python2 by any chance? I didn't test things there, so it might be the syntax doesn't work for the older version of Python.

Anyway, start by checking that the config files are in place and named correctly.

Let me know, would like to help get you through this quickly.

[mlava]

@raetha Thanks for the fast reply. A list of files in /config shows: config.yaml logging.yaml sensors.yaml.sample

vim config/config.yaml:

mqtt_host: 192.168.1.148
mqtt_port: 1883
mqtt_username:
mqtt_password:
mqtt_client_id: wyzesense2mqtt
mqtt_clean_session: false
mqtt_keepalive: 60
mqtt_qos: 2
mqtt_retain: true
self_topic_root: /wyzesense2mqtt
hass_topic_root: /homeassistant
usb_dongle: auto

vim config/logging.yaml:

version: 1
formatters:
  simple:
    format: '%(message)s'
  verbose:
    datefmt: '%Y-%m-%d %H:%M:%S'
    format: '%(asctime)s %(levelname)-8s %(name)-15s %(message)s'
handlers:
  console:
    class: logging.StreamHandler
    formatter: simple
    level: DEBUG
  file:
    backupCount: 7
    class: logging.handlers.TimedRotatingFileHandler
    encoding: utf-8
    filename: logs/wyzesense2mqtt.log
    formatter: verbose
    level: INFO
    when: midnight
root:
  handlers:
    - file
    - console
  level: DEBUG

I have both python2 and 3 installed on the Pi.

I ran the .py file with python3 wyzesense2mqtt.py

Cheers :)

[raetha]

I think I might have found something, looking at my config. Can you changing the username/password lines in config.yaml to this. Note on the two single quotes implying an empty string. I think it might be throwing the YAML off, but just a guess.

mqtt_username: ''
mqtt_password: ''

The other quick thought I have is that on most linux systems the devices can only be accessed using root or sudo. So unless you've made changes intentionally, give a test with root/sudo as well and see if anything different there.

If neither of those is it, I'm going to add some more debugging code for you to the devel branch, so we can figure out which file it's trying to parse that is giving the error. It makes sense to have in there anyway, I'd just never hit an issue here myself. Not sure when I'll get that in, but hopefully later today.

[mlava]

Hi @raetha Sorry for the delay, I've been asleep. I added the single quote pairs to user and pass, and I tried both with and without sudo. Sadly, I get the same error, no change.

I think I figured it out. I'm using Python 3.4.2 and that version doesn't appear to support print(f syntax. See https://stackoverflow.com/questions/42126794/python-3-returns-invalid-syntax-when-trying-to-perform-string-interpolation

I took all the 'f' out of the logging and console prints in the py script and the syntax errors are gone. Now I have a new error with mqtt but I'll keep chipping away.

I got this error when running the python script:

>python3 wyzesense2mqtt.py

Logging initialized...
Reading configuration...
Traceback (most recent call last):
  File "wyzesense2mqtt.py", line 390, in <module>
    init_mqtt_client()
  File "wyzesense2mqtt.py", line 82, in init_mqtt_client
    MQTT_CLIENT.reconnect_delay_set(min_delay=1, max_delay=120)
AttributeError: 'Client' object has no attribute 'reconnect_delay_set'

I commented this line as shown:

#MQTT_CLIENT.reconnect_delay_set(min_delay=1, max_delay=120)

The script then seems to run, but I get the following output which appears to show an issue with initialising the dongle:

> sudo python3 wyzesense2mqtt.py

Logging initialized...
Reading configuration...
Connecting to MQTT host {CONFIG['mqtt_host']}
ls: cannot access /sys/class/qinheng/: No such file or directory
ls: cannot access /sys/class/qinheng/: No such file or directory
ls: cannot access /sys/class/qinheng/: No such file or directory

ls: cannot access /sys/class/qinheng/: No such file or directory just repeats over and over, with the exponential increase in duration

I changed the usb_dongle section in the py file to:

def init_wyzesense_dongle():
    global WYZESENSE_DONGLE, CONFIG
    if (CONFIG['usb_dongle'].lower() == "auto"):
        device_list = subprocess.check_output(
                ["ls", "-la", "/sys/class/qinheng/"]
        ).decode("utf-8").lower()
        for line in device_list.split("\n"):
            if (("e024" in line) and ("1a86" in line)):
                for device_name in line.split(" "):
                    if ("QinHeng" in device_name):
                        CONFIG['usb_dongle'] = "/dev/%s" % device_name
                        break

as my lsusb shows:

Bus 001 Device 003: ID 1a86:e024 QinHeng Electronics

When I tried as shown:

def init_wyzesense_dongle():
    global WYZESENSE_DONGLE, CONFIG
    if (CONFIG['usb_dongle'].lower() == "auto"):
        device_list = subprocess.check_output(
                ["ls", "-la", "/sys/class/hidraw/"]
        ).decode("utf-8").lower()
        for line in device_list.split("\n"):
            if (("e024" in line) and ("1a86" in line)):
                for device_name in line.split(" "):
                    if ("QinHeng" in device_name):
                        CONFIG['usb_dongle'] = "/dev/%s" % device_name
                        break

I get the following output in console:

> sudo python3 wyzesense2mqtt.py

Logging initialized...
Reading configuration...
Connecting to MQTT host {CONFIG['mqtt_host']}
Connecting to dongle {CONFIG['usb_dongle']}
No device found on path {CONFIG['usb_dongle']}
Reading sensors configuration...
sensor_mac: {sensor_mac}
Publishing discovery topics for {sensor_mac}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
sensor_mac: {sensor_mac}
Publishing discovery topics for {sensor_mac}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
sensor_mac: {sensor_mac}
Publishing discovery topics for {sensor_mac}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
sensor_mac: {sensor_mac}
Publishing discovery topics for {sensor_mac}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
  {entity_topic}
  {json.dumps(entity_payload)}
Traceback (most recent call last):
  File "wyzesense2mqtt.py", line 396, in <module>
    init_sensors()
  File "wyzesense2mqtt.py", line 136, in init_sensors
    result = WYZESENSE_DONGLE.List()
NameError: name 'WYZESENSE_DONGLE' is not defined

I'm running out of ideas, other than to update my python3... I hope this is useful?

[raetha]

@mlava that is a lot of helpful information, so thanks. I'm thinking that some of the required library versions might be missing or out of date, along with Python itself.

I can say the entire script is configured to use F strings, so that is why the publishing debug output isn't showing the actual values, but instead the strings. At this point I'm not prepared to try to modify the script to handle Python older than 3.6 where that support was added.

Are you using Python for other things that require the older version? If so, then I'd recommend trying to use this project from a Docker container on your RPI, which will make sure you get all the right versions of things and don't have to mess with your main operating system.

If you can safely upgrade Python though, I'd get to 3.6.x and then rerun the command to install/upgrade dependencies. That should make sure you have everything you need.

[raetha]

Oh, and I forgot to confirm that Linux sees the device class for the bridge as hidraw, regardless of manufacturer, so putting it back definitely solved that issue.

[mlava]

No problem, thanks for troubleshooting with me. I Can't remember so will have to check what I have running on that Rpi and whether it relies on python3.4.2 If I can upgrade python I'll do that, otherwise I'll try docker. Thanks again!

[mlava]

@raetha Hey Elias. Sorry to pester. I can't seem to get docker-compose running on my RPi0W so trying run directly from terminal. I've think this is what it should look like... any last pointers?

docker run -d -v '/docker/wyzesense2mqtt/config:/wyzesense2mqtt/config' -v '/docker/wyzesense2mqtt/logs:/wyzesense2mqtt/logs' --network bridge --name wyzesense2mqtt --restart always --device '/dev/hidraw0:/dev/hidraw0' raetha/wyzesense2mqtt

Obviously, after creating those directories and adding the config.yaml and logging.yaml files.

I get a restart loop with exit code 139... I don't suppose you know if anyone else has this running in docker on a RPi0W? I had trouble with a different script before... https://github.com/yllibed/Zigbee2MqttAssistant/issues/174#issuecomment-581063942

Finally, I tried to run it in docker on my Synology NAS but there is no sys/class/hidraw file so it won't work... :(

[raetha]

I'm using an RPi4, so not sure on the 0W, but I can't think of any reason it wouldn't work. I'm running it on Raspbian Buster (aka Raspbian OS), using docker-compose, but I do seem to recall I had to do something special to get that working. I don't think the default docker packages built into Raspbian were up-to-date enough to work right with modern stuff. Pretty sure the command I used was: "curl -sSL https://get.docker.com | sh" There are more detailed instructions at: https://linuxhint.com/install_docker_on_raspbian_os/

Give that a shot and see if that works. I've found over time that a lot of the Linux distros aren't maintaining the docker packages well, so if you just apt-installed docker-compose, it is possibly broken, same with the main docker engine.

Let me know, and this isn't a bother at all, just want to help get you going. :)

[mlava]

@raetha Hey again, sorry for the delay. Been working on getting python3.8.4 working on my Pi0W and it's taken some time. I finally have a working python > 3.6.x and I've finally got your py script running :) I can run the service.sh file directly but had issues running it as a service...

This is the terminal output:

> sudo systemctl status wyzesense2mqtt
● wyzesense2mqtt.service - WyzeSense to MQTT Gateway
   Loaded: loaded (/etc/systemd/system/wyzesense2mqtt.service; disabled)
   Active: failed (Result: start-limit) since Fri 2020-07-17 11:06:26 UTC; 816ms ago
     Docs: https://www.github.com/raetha/wyzesense2mqtt
  Process: 10918 ExecStart=/wyzesense2mqtt/service.sh (code=exited, status=203/EXEC)
 Main PID: 10918 (code=exited, status=203/EXEC)

Jul 17 11:06:26 raspberrypi systemd[1]: wyzesense2mqtt.service holdoff time over, scheduling restart.
Jul 17 11:06:26 raspberrypi systemd[1]: Stopping WyzeSense to MQTT Gateway...
Jul 17 11:06:26 raspberrypi systemd[1]: Starting WyzeSense to MQTT Gateway...
Jul 17 11:06:26 raspberrypi systemd[1]: wyzesense2mqtt.service start request repeated too quickly, refusing to start.
Jul 17 11:06:26 raspberrypi systemd[1]: Failed to start WyzeSense to MQTT Gateway.
Jul 17 11:06:26 raspberrypi systemd[1]: Unit wyzesense2mqtt.service entered failed state.

I had to systemctl enable the service, as well as change the address to the /wyzesense2mqtt/service.sh file and then chmod as it still didn't run with default permissions. But now it's running! Thanks again for all your help! :D

[raetha]

@mlava Did you happen to note down the exact commands you ran for the systemctl and permissions fix? I'd love to get that added to the documentation at least, but since I'm running in Docker haven't actually taken the time to test those commands and just "borrowed" the same ones used by the project I first found. So I'm not surprised that they don't work perfectly, but would like to save others headaches going forward. :)

Otherwise I'm really glad you got things going, and sorry that it took so long. But it's helpful to get some real world feedback on this!

[mlava]

Hey @raetha

In addition to what's in the readme, I did the following between steps 7 & 8 with additions in asterixes:

**cd /wyzesense2mqtt**
**chmod 755 wyzesense2mqtt.py**

vim wyzesense2mqtt.service # Only modify if not using default application path
sudo cp wyzesense2mqtt.service /etc/systemd/system/
**sudo systemctl enable**
sudo systemctl daemon-reload
sudo systemctl start wyzesense2mqtt
sudo systemctl status wyzesense2mqtt

Thanks again, my Wyze sensors are sending mqtt messages and I've got them showing in my openhab. All good!

BTW - I'm not sure if the issue I had with the folder structures in the wyzesense2mqtt.service file is because I accidentally installed in the wrong place and then moved the folder to where I wanted it. I had to change the directories as below:

WorkingDirectory=/home/pi/wyzesense2mqtt
ExecStart=home/pi/wyzesense2mqtt/service.sh