no data to mqtt

[j3sk0]

Hi

i edit the mqtt setting in the config and i see the data.csv but I miss the data in the mqtt fpr my iobroker. Other data from other scripts arrive at mqtt.

remeha.py -d /dev/ttyUSB0 --output data.csv remeha_convert.py --input-file data.csv > converted.csv

Error on plotlytest.py

`root@ioB:/opt/iobroker# remeha.py -d /dev/ttyUSB0 --output data.csv
^CTraceback (most recent call last):
  File "/usr/local/bin/remeha.py", line 109, in <module>
    log(args.device, args.output, args.mqtt_freq)
  File "/usr/local/bin/remeha.py", line 97, in log
    time.sleep(1)
KeyboardInterrupt

root@ioB:/opt/iobroker# remeha_convert.py --input-file <data csv> > converted.csv
bash: syntax error near unexpected token `>'
root@ioB:/opt/iobroker# remeha_convert.py --input-file data.csv converted.csv
usage: remeha_convert.py [-h] [-i INPUT_FILE]
remeha_convert.py: error: unrecognized arguments: converted.csv
root@ioB:/opt/iobroker# remeha_convert.py --input-file data.csv > converted.csv
root@ioB:/opt/iobroker# --help
bash: --help: command not found
root@ioB:/opt/iobroker# remeha.py --help
usage: remeha.py [-h] [-d DEVICE] [-o OUTPUT] [-m MQTT_FREQ]

Log data from Remeha boiler

optional arguments:
  -h, --help            show this help message and exit
  -d DEVICE, --device DEVICE
                        serial device the boiler is connected to. i.e. /dev/ttyUSB0 [Default: /dev/ttyS0]
  -o OUTPUT, --output OUTPUT
                        file to log the data to [Default: data.csv]
  -m MQTT_FREQ, --mqtt_freq MQTT_FREQ
                        frequency for publishing MQTT data in seconds [Default: 15]
root@ioB:/opt/iobroker# plotytest.py --input-file data.csv
bash: plotytest.py: command not found
root@ioB:/opt/iobroker# plotlytest.py --input-file data.csv
/usr/local/bin/plotlytest.py:7: UserWarning: 
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/usr/local/bin/plotlytest.py:8: UserWarning: 
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
usage: plotlytest.py [-h] [--verbose] [--debug] [-m MAX_READ] [-p MAX_POINTS] [-i INPUT]
plotlytest.py: error: unrecognized arguments: --input-file data.csv
root@ioB:/opt/iobroker# plotlytest.py -i data.csv
/usr/local/bin/plotlytest.py:7: UserWarning: 
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/usr/local/bin/plotlytest.py:8: UserWarning: 
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
Traceback (most recent call last):
  File "/usr/local/bin/plotlytest.py", line 220, in <module>
    data_description += [[str(translator.translate(map_entry[2], map_entry[3])), str(map_entry[4]), map_entry[5]]]
IndexError: list index out of range
root@ioB:/opt/iobroker# plotlytest.py --input-file data.csv > plotly.csv
/usr/local/bin/plotlytest.py:7: UserWarning: 
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/usr/local/bin/plotlytest.py:8: UserWarning: 
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
usage: plotlytest.py [-h] [--verbose] [--debug] [-m MAX_READ] [-p MAX_POINTS]
                     [-i INPUT]
plotlytest.py: error: unrecognized arguments: --input-file data.csv
root@ioB:/opt/iobroker# plotlytest.py -i data.csv > plotly.csv
/usr/local/bin/plotlytest.py:7: UserWarning: 
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/usr/local/bin/plotlytest.py:8: UserWarning: 
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
Traceback (most recent call last):
  File "/usr/local/bin/plotlytest.py", line 220, in <module>
    data_description += [[str(translator.translate(map_entry[2], map_entry[3])), str(map_entry[4]), map_entry[5]]]
IndexError: list index out of range
root@ioB:/opt/iobroker# plotlytest.py -h
/usr/local/bin/plotlytest.py:7: UserWarning: 
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/usr/local/bin/plotlytest.py:8: UserWarning: 
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
usage: plotlytest.py [-h] [--verbose] [--debug] [-m MAX_READ] [-p MAX_POINTS] [-i INPUT]

Process some integers.

optional arguments:
  -h, --help            show this help message and exit
  --verbose             increase verbosity
  --debug               run flask server in debug mode
  -m MAX_READ, --max-read MAX_READ
                        maximum to read in kb [Default: 1024]
  -p MAX_POINTS, --max-points MAX_POINTS
                        maximum data points to hold [Default: 50]
  -i INPUT, --input INPUT
                        file read the data from [Default: data.csv]
root@ioB:/opt/iobroker# plotlytest.py --i data.csv 
/usr/local/bin/plotlytest.py:7: UserWarning: 
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/usr/local/bin/plotlytest.py:8: UserWarning: 
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
Traceback (most recent call last):
  File "/usr/local/bin/plotlytest.py", line 220, in <module>
    data_description += [[str(translator.translate(map_entry[2], map_entry[3])), str(map_entry[4]), map_entry[5]]]
IndexError: list index out of range
root@ioB:/opt/iobroker# mqtt_logger.py -h
bash: mqtt_logger.py: command not found
root@ioB:/opt/iobroker# mqtt_logger.py --h
bash: mqtt_logger.py: command not found
root@ioB:/opt/iobroker# ^C
root@ioB:/opt/iobroker# remeha_convert.py --input-file <data csv> > converted.csv`

[ciddi89]

Hey,

I guess I have the same problems and can't get it to work. I've installed it with pip install remeha-tz on a RaspberryPi. But where will the files be located? I can't find them on the Pi. And if I try it to start with remeha.py -d /dev/ttyUSB0 --output test.csv I get this:

remeha.py -d /dev/ttyUSB0 --output test.csv
Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.9/site-packages/serial/serialposix.py", line 322, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
FileNotFoundError: [Errno 2] No such file or directory: '/dev/ttyUSB0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/bin/remeha.py", line 109, in <module>
    log(args.device, args.output, args.mqtt_freq)
  File "/home/pi/.local/bin/remeha.py", line 57, in log
    ser = serial.Serial(source_serial,
  File "/home/pi/.local/lib/python3.9/site-packages/serial/serialutil.py", line 244, in __init__
    self.open()
  File "/home/pi/.local/lib/python3.9/site-packages/serial/serialposix.py", line 325, in open
    raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'

In hope you can help me.

Thanks

[TheRikke]

Sorry for the delay, somehow missed the notification.

@j3sk0 Which version did you use? The released one, or directly from github? I really need to update the released one. Regarding the mqtt issue. Could you show your config? Also seeing what has been logged to the data.csv file could help understand the issue. Regarding the remeha_convert issue. The "\<data csv>" is meant as a placeholder for the file name of the csv, so the command would be remeha_convert.py --input-file data.csv > converted.csv

@ciddi89 It looks like your system does not have /dev/ttyUSB0. Did you connect your boiler through a USB serial adapter? If you directly connect it to a serial port on your computer, the port often is called /dev/ttyS0. Also dmesg | grep tty might help find available serial devices on your system.

[j3sk0]

Hi @TheRikke

Many thanks for your answer. I think I took the version from github. Here is my config file

the project runs in a docker container i forwarded the path /volume1/docker/ioB_userscripts /opt/userscripts /volume1/docker/Phyton/config /usr/local/lib/python3.9/site-packages/config

here the code´s

remeha_TheRikke.zip

[TheRikke]

Hi @j3sk0

I did not see any obvious issues, but there are some points we should check:

1. user name and password need to be quoted like other strings: ex. "user_name": "secret"
2. does your broker use SSL/TLS? If yes, I added a new parameter to the config for enabling SSL/TLS support. You need to update to the lates version.
3. You use port 1884. The default port is 1883 for unencrypted connections and 8883 for encrypted connections
4. I have added some logging to detect MQTT connection errors. If you use the latest version the error message might give a hint about the reason.
5. The csv looks different than mine, which could be just a different configuration of your boiler or a different type. What is the type of your boiler?

[ciddi89]

Hello @TheRikke, thanks for your respond. Today I had time to test it again and now I get data in the data.csv and the converted data looks good and matched to the values of the boiler. (I had used the wrong serial port) But now I got the message when I use the command to start the script:

remeha.py -d /dev/ttyS0 --output data.csv

Traceback (most recent call last):
  File "/home/pi/.local/bin/remeha.py", line 109, in <module>
    log(args.device, args.output, args.mqtt_freq)
  File "/home/pi/.local/bin/remeha.py", line 81, in log
    log_mqtt.log(frame, runtime_seconds)
  File "/home/pi/.local/lib/python3.9/site-packages/mqtt_logger.py", line 48, in log
    print("Temp: %d" % self.frame_decoder.decode(unpacked_data, 'outside_temp'))
AttributeError: 'NoneType' object has no attribute 'decode'

and I get also no connection to mqtt. Maybe the remeha.conf is in the wrong folder? I put it in /etc/remeha/remeha.conf and also in the same folder where the script is placed.

Thanks for your help.

[j3sk0]

Hello and thank you for your answer @TheRikke

I actually didn't have a "pw" in the code

I have now updated the script.

1. " entered "mqtt_logger": { "enabled": true, "host": "192.168.178.100", "port": 1884, "user_name": "xyz", "password": "xyz",

2. I don't use SSL/TLS

3. my port is 1884

4. where can i find the log?

5. I have the remeha celenta 25ds

update and start the script log is this update correct?

root@ioB:/opt/iobroker# pip install git+https://github.com/TheRikke/remeha_tz.git@master Collecting git+https://github.com/TheRikke/remeha_tz.git@master Cloning https://github.com/TheRikke/remeha_tz.git (to revision master) to /tmp/pip-req-build-xvno7lf_ Running command git clone -q https://github.com/TheRikke/remeha_tz.git /tmp/pip-req-build-xvno7lf_ Requirement already satisfied: dash in /usr/local/lib/python3.9/dist-packages (from remeha-tz==0.11) (2.2.0) Requirement already satisfied: mysql-connector-python in /usr/local/lib/python3.9/dist-packages (from remeha-tz==0.11) (8.0.28) Requirement already satisfied: paho-mqtt in /usr/local/lib/python3.9/dist-packages (from remeha-tz==0.11) (1.6.1) Requirement already satisfied: plotly in /usr/local/lib/python3.9/dist-packages (from remeha-tz==0.11) (5.6.0) Requirement already satisfied: pyserial in /usr/local/lib/python3.9/dist-packages (from remeha-tz==0.11) (3.5) Requirement already satisfied: dash-html-components==2.0.0 in /usr/local/lib/python3.9/dist-packages (from dash->remeha-tz==0.11) (2.0.0) Requirement already satisfied: dash-core-components==2.0.0 in /usr/local/lib/python3.9/dist-packages (from dash->remeha-tz==0.11) (2.0.0) Requirement already satisfied: Flask>=1.0.4 in /usr/local/lib/python3.9/dist-packages (from dash->remeha-tz==0.11) (2.0.3) Requirement already satisfied: flask-compress in /usr/local/lib/python3.9/dist-packages (from dash->remeha-tz==0.11) (1.10.1) Requirement already satisfied: dash-table==5.0.0 in /usr/local/lib/python3.9/dist-packages (from dash->remeha-tz==0.11) (5.0.0) Requirement already satisfied: Werkzeug>=2.0 in /usr/local/lib/python3.9/dist-packages (from Flask>=1.0.4->dash->remeha-tz==0.11) (2.0.3) Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.9/dist-packages (from Flask>=1.0.4->dash->remeha-tz==0.11) (3.0.3) Requirement already satisfied: click>=7.1.2 in /usr/local/lib/python3.9/dist-packages (from Flask>=1.0.4->dash->remeha-tz==0.11) (8.0.4) Requirement already satisfied: itsdangerous>=2.0 in /usr/local/lib/python3.9/dist-packages (from Flask>=1.0.4->dash->remeha-tz==0.11) (2.1.0) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from Jinja2>=3.0->Flask>=1.0.4->dash->remeha-tz==0.11) (2.1.0) Requirement already satisfied: six in /usr/local/lib/python3.9/dist-packages (from plotly->remeha-tz==0.11) (1.16.0) Requirement already satisfied: tenacity>=6.2.0 in /usr/local/lib/python3.9/dist-packages (from plotly->remeha-tz==0.11) (8.0.1) Requirement already satisfied: brotli in /usr/local/lib/python3.9/dist-packages (from flask-compress->dash->remeha-tz==0.11) (1.0.9) Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.9/dist-packages (from mysql-connector-python->remeha-tz==0.11) (3.19.4) root@ioB:/opt/iobroker# remeha.py -d /dev/ttyUSB0 --output data.csv ^CTraceback (most recent call last): File "/usr/local/bin/remeha.py", line 109, in <module> log(args.device, args.output, args.mqtt_freq) File "/usr/local/bin/remeha.py", line 77, in log frame = Frame(io_source=ser) File "/usr/local/lib/python3.9/dist-packages/remeha_core.py", line 46, in __init__ self.frame += io_source.read(size - 7) File "/usr/local/lib/python3.9/dist-packages/serial/serialposix.py", line 565, in read ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left()) KeyboardInterrupt

[TheRikke]

@ciddi89 and @j3sk0 You are both using a very old version, which did not support reading the MQTT parameters (like host and port) from a config. I guess that you initially installed the released version and after that the github version. Since the version numbers of both are the same pip seams to do nothing. Please try this:

pip uninstall remeha_tz pip install git+https://github.com/TheRikke/remeha_tz.git@master

[ciddi89]

You're the best @TheRikke !!! For me, It works perfectly now! :)) Many thanks!

[ciddi89]

hmm I cheered to soon. Something doesn't work well. I got the data to mqtt, this works fine but it's not refreshing and I get this when I start the script:

2022-03-30 00:49:37,314 Thread-1        ERROR    client         :2529     Caught exception in on_message: 'DatabaseLogger' object has no attribute 'manual_log_cursor'
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3591, in _thread_main
    self.loop_forever(retry_first_connection=True)
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1756, in loop_forever
    rc = self._loop(timeout)
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1164, in _loop
    rc = self.loop_read()
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1556, in loop_read
    rc = self._packet_read()
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 2439, in _packet_read
    rc = self._packet_handle()
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3033, in _packet_handle
    return self._handle_publish()
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3327, in _handle_publish
    self._handle_on_message(message)
  File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3570, in _handle_on_message
    on_message(self, self._userdata, message)
  File "/usr/local/lib/python3.9/dist-packages/mqtt_logger.py", line 22, in on_message
    update_function(message.payload.decode('UTF-8'))
  File "/usr/local/bin/remeha.py", line 114, in <lambda>
    log_mqtt = LogToMQtt(config, mqtt_freq, lambda message: log_db.log_manual(message))
  File "/usr/local/lib/python3.9/dist-packages/database_logger.py", line 159, in log_manual
    self.manual_log_cursor.execute(sql_query, params=(message,))
AttributeError: 'DatabaseLogger' object has no attribute 'manual_log_cursor'

[TheRikke]

@ciddi89 Looks like a MQTT message was received. These messages are meant to be persisted in a database, but that fails if not database was configured. I pushed a fix. Please try latest version

[ciddi89]

@TheRikke what should I say? Now it works perfectly. It runs without issues and refreshed every fifteen seconds. Thank you so much for your work and time.

[j3sk0]

unfortunately it doesn't work for me yet, after the new installation and the change in the config, the following error always comes up

root@ioB:/etc/remeha# remeha.py -d /dev/ttyUSB0 --output test.csv
Traceback (most recent call last):
  File "/usr/local/bin/remeha.py", line 196, in <module>
    log_remeha(args.device, args.output, args.mqtt_freq, read_config())
  File "/usr/local/bin/remeha.py", line 168, in read_config
    config = json.load(source)
  File "/usr/lib/python3.9/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 3362: invalid start byte

 "mqtt_logger": {
        "enabled": true,
        "host": "localhost",
        "port": "1884",
        "comment_tls_enabled": "Enables tls encryption with the system cert authority. Don't forget to update the port. The default for unencrypted connection is 1883 and for encrypted connection is 8883",

        "tls_enabled": false,
        "user_name": "xyz",
        "password": "xyz",

[TheRikke]

Looks like your config has an invalid character. At position 3362 the value is 0xb0. That probably is a degree symbol '°' and the config was saved with the wrong encoding. Please open the file with your editor, change the encoding to utf-8 and save it again.

[j3sk0]

done @TheRikke Many thanks!

https://github.com/TheRikke/remeha_tz/issues/7

[j3sk0]

hello can someone help me

[TheRikke]

@j3sk0 Sure, what's the issue?

[j3sk0]

@j3sk0 Sure, what's the issue?

A wrong post I meant the problem in post https://github.com/TheRikke/remeha_tz/issues/7