not compatible with python 3.11 ?

[Robert-Riedl]

I'm trying this on Rasbian bookworm wiht python 3.11 and getting weird errors, i.e.

#~ python3 /opt/waveplus_bridge/waveplus_bridge.py --config /opt/waveplus_bridge/waveplus_bridge.yaml 293014466
Traceback (most recent call last):
  File "/opt/waveplus_bridge/waveplus_bridge.py", line 804, in <module>
    main()
  File "/opt/waveplus_bridge/waveplus_bridge.py", line 586, in main
    config = ReadConfiguration()
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/waveplus_bridge/waveplus_bridge.py", line 83, in __init__
    self.read_yaml_config_file(config['config']).items():
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'items'

even though the basic function works, but it seems to ignore the --period setting ? 2-3 seconds instead of 10 ?

~# python3 /opt/waveplus_bridge/waveplus_bridge.py --period 10 2930144663 
2024-01-26 16:39:02 -      __main__[  DEBUG] - Available logger:
2024-01-26 16:39:02 -      __main__[  DEBUG] -    <Logger libs.logdb (WARNING)>
2024-01-26 16:39:02 -      __main__[  DEBUG] -    <Logger libs (WARNING)>
2024-01-26 16:39:02 -      __main__[  DEBUG] -    <Logger libs.threadedsendmail (WARNING)>
2024-01-26 16:39:02 -      __main__[  DEBUG] -    <Logger libs.performancecheck (WARNING)>
2024-01-26 16:39:02 -      __main__[  DEBUG] -    <Logger libs.waveplus (WARNING)>
2024-01-26 16:39:02 -      __main__[  DEBUG] -    <Logger libs.threadedmqttpublisher (WARNING)>
2024-01-26 16:39:02 -      __main__[  DEBUG] -    <Logger __main__ (DEBUG)>
2024-01-26 16:39:02 -      __main__[   INFO] - Setup WavePlus device access
2024-01-26 16:39:02 -      __main__[   INFO] -   Done
2024-01-26 16:39:02 -      __main__[   INFO] - Start main loop. Press ctrl+C to exit program!
2024-01-26 16:39:02 -      __main__[  DEBUG] - Reading sensor data for device 2930144663
2024-01-26 16:39:02 - libs.waveplus[WARNING] - Failed to communicate with device                                2930144663 (attempt 1 of 4): Device disconnected
2024-01-26 16:39:08 -      __main__[  DEBUG] -   -> {'humidity': 43.0, 'radon_st': 35, 'radon_lt': 66, 'temperature': 22.23, 'pressure': 997.32, 'co2': 1185.0, 'voc': 46.0, 'illuminance': 5, 'battery': 100}
2024-01-26 16:39:12 -      __main__[  DEBUG] - Reading sensor data for device 2930144663
2024-01-26 16:39:14 -      __main__[  DEBUG] -   -> {'humidity': 43.0, 'radon_st': 35, 'radon_lt': 66, 'temperature': 22.23, 'pressure': 997.32, 'co2': 1185.0, 'voc': 46.0, 'illuminance': 5, 'battery': 100}
2024-01-26 16:39:22 -      __main__[  DEBUG] - Reading sensor data for device 2930144663
2024-01-26 16:39:24 -      __main__[  DEBUG] -   -> {'humidity': 43.0, 'radon_st': 35, 'radon_lt': 66, 'temperature': 22.23, 'pressure': 997.32, 'co2': 1185.0, 'voc': 46.0, 'illuminance': 5, 'battery': 100}
2024-01-26 16:39:32 -      __main__[  DEBUG] - Reading sensor data for device 2930144663
2024-01-26 16:39:35 -      __main__[  DEBUG] -   -> {'humidity': 43.0, 'radon_st': 35, 'radon_lt': 66, 'temperature': 22.23, 'pressure': 997.32, 'co2': 1185.0, 'voc': 46.0, 'illuminance': 5, 'battery': 100}
2024-01-26 16:39:42 -      __main__[  DEBUG] - Reading sensor data for device 2930144663
2024-01-26 16:39:45 -      __main__[  DEBUG] -   -> {'humidity': 43.0, 'radon_st': 35, 'radon_lt': 66, 'temperature': 22.23, 'pressure': 997.32, 'co2': 1185.0, 'voc': 46.0, 'illuminance': 5, 'battery': 100}
^C2024-01-26 16:39:48 -      __main__[WARNING] - Interrupt/termination request detected
2024-01-26 16:39:48 -      __main__[   INFO] - WavePlus_bridge ended

The only thing I can think of, that makes a difference - would be the python version ?

[Robert-Riedl]

forget the period question.. polling lower than 5 minutes doesn't really make sense anyways.

but i've installes python 3.7.3 now and it works ootb - so it really seems to be a python 3.11 issue ?

[Drolla]

Thanks for submitting this bug report. The WavePlus bridge has indeed only tested with Python 3.7. I will have a look into this issue once I find some time...

[Drolla]

I have just installed Python 3.11.8. I cannot see a problem when the YAML configuration file is loaded. Could you maybe share your YAML configuration file? If it contains personal information or passwords, just replace them with Xes. Thanks!

[Drolla]

While I did my previous trials with Python 3.11.8 on a Windows laptop to check the loading of the YAML configuration file, I installed now on my Raspberry Pi Zero W where I run the WavePlus bridge Python 3.11.8 and 3.12.2 in addition to the system Python version 3.7.3. Pyenv has been used to install simultaneously various Python versions. The required modules have been installed with 'pip3' for each Python version.

Here are my findings when I ran the WavePlus bridge on the 2 newer Python versions:

Backward incompatibility of the MQTT 2.0 client

This issue is not related to the Python version, but due to the fact that paho-mqtt version 2.0 has been released in the meantime, and that this new version is not backward compatible with paho-mqtt 1.0.

WavePlus log error:

2024-03-01 15:16:16 -      __main__[   INFO] - Setup MQTT publishing
2024-03-01 15:16:16 -      __main__[CRITICAL] -   Unable to setup MQTT publishing: Client.__init__() missing 1 required positional argument: 'callback_api_version'

WavePlus error on stdout:

Exception ignored in: <function Client.__del__ at 0xb5786a28>
Traceback (most recent call last):
  File "/home/pi/.pyenv/versions/3.12.2/lib/python3.12/site-packages/paho/mqtt/client.py", line 874, in __del__
    self._reset_sockets()
  File "/home/pi/.pyenv/versions/3.12.2/lib/python3.12/site-packages/paho/mqtt/client.py", line 1133, in _reset_sockets
    self._sock_close()
  File "/home/pi/.pyenv/versions/3.12.2/lib/python3.12/site-packages/paho/mqtt/client.py", line 1119, in _sock_close
    if not self._sock:
           ^^^^^^^^^^
AttributeError: 'Client' object has no attribute '_sock'
Exception ignored in: <function MqttPublisher.__del__ at 0xb578d7f8>
Traceback (most recent call last):
  File "/home/pi/./waveplus_bridge/waveplus_bridge.py", line 518, in __del__
    self.stop()
  File "/home/pi/./waveplus_bridge/waveplus_bridge.py", line 504, in stop
    if self.mqtt_publisher is None:
       ^^^^^^^^^^^^^^^^^^^
AttributeError: 'MqttPublisher' object has no attribute 'mqtt_publisher'
Exception ignored in: <function ThreadedMqttPublisher.__del__ at 0xb5786528>
Traceback (most recent call last):
  File "/home/pi/WavePlus_Bridge-add_battery_level.20230702/libs/threadedmqttpublisher.py", line 166, in __del__
    self.stop()
  File "/home/pi/WavePlus_Bridge-add_battery_level.20230702/libs/threadedmqttpublisher.py", line 156, in stop
    if self.mqttc is None:
       ^^^^^^^^^^
AttributeError: 'ThreadedMqttPublisher' object has no attribute 'mqttc'

A similar error message is obtained when running the demo application of the threadedmqttpublisher.py module (python threadedmqttpublisher.py --host test.mosquitto.org --port 1883 --topic --debug_level 3 "{status: ready}")

A fix is in preparation to correct this issue.

Incompatibility with previous PyYAML revision:

A previous version of the PyYAML module that was installed with Python 3.11.8 and 3.12.2 does not provide the "FullLoader", which leads to the following error when the demo application of the threadedmqttpublisher.py module is executed (see previous paragraph):

__main__ - INFO - Parse arguments provided in Yaml format
Traceback (most recent call last):
  File "threadedmqttpublisher.py", line 285, in <module>
    Loader=yaml.FullLoader)
AttributeError: module 'yaml' has no attribute 'FullLoader'

A fix is in preparation to correct this issue.

BluePy module not working with Python 3.11.8

Running the WavePlus bridge, the following error has been generated by the 'bluepy' module:

Failed to communicate with device                                2930014021 (attempt 1 of 4): Helper exited
  Stack trace:
Traceback (most recent call last):
  File "/home/pi/waveplus_bridge/libs/waveplus.py", line 381, in get
    self.connect()
  File "/home/pi/waveplus_bridge/libs/waveplus.py", line 339, in connect
    self.discover()
  File "/home/pi/waveplus_bridge/libs/waveplus.py", line 309, in discover
    devices = scanner.scan(0.1)  # 0.1 seconds scan period
              ^^^^^^^^^^^^^^^^^
  File "/home/pi/.pyenv/versions/3.11.8/lib/python3.11/site-packages/bluepy/btle.py", line 852, in scan
    self.start(passive=passive)
  File "/home/pi/.pyenv/versions/3.11.8/lib/python3.11/site-packages/bluepy/btle.py", line 790, in start
    self._mgmtCmd("le on")
  File "/home/pi/.pyenv/versions/3.11.8/lib/python3.11/site-packages/bluepy/btle.py", line 309, in _mgmtCmd
    rsp = self._waitResp('mgmt')
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.pyenv/versions/3.11.8/lib/python3.11/site-packages/bluepy/btle.py", line 339, in _waitResp
    raise BTLEInternalError("Helper exited")
bluepy.btle.BTLEInternalError: Helper exited

It turned out that the helper executable 'bluepy-helper' could not be executed by the bluepy module. Running this executable from the Python 3.11.8 installation, the following error is obtained:

~/.pyenv/versions/3.11.8/lib/python3.11/site-packages/bluepy/bluepy-helper
bluepy-helper: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.34' not found
bluepy-helper: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.33' not found

Running the same helper program from the Python 3.12.2 installation, no error is obtained:

~/.pyenv/versions/3.12.2/lib/python3.12/site-packages/bluepy/bluepy-helper
# bluepy-helper.c version 1.3.0 built at 17:34:11 on Feb 28 2024

By copying the 'bluepy-helper' executable from the Python 3.11.8 installation directory into the Python 3.12.2 installation directory, the WavePlus bridge worked well. I have not investigated why the 'bluepy-helper' executable of the 3.11.8 installation did not work correctly.

With all that said, I cannot confirm that the YAML configuration reader does not work correctly with Python 3.11. Maybe you can provide me an anonymized version of the your configuration file for that I can make some trials.

[Robert-Riedl]

I'll try to reproduce this myself and get you the config.