Struggling to get values from mqtt

[kasper-coding]

Hi! First thank you for this great piece of software. I was so annoyed when I realized SE does only work with that limited set of meters after I installed it... However, I managed to provide all values from my meter to an mqtt topic (mapped to the specific values required by the program), but it seems these values are not really read? I see new values are pushed to mqtt all the time but nothing shows up in the debug log. Will I need to map the mqtt values again in the values procedure of the mqtt.py?

I'm not that deep into programming, but maybe someone can support me on that?

Thanks Malte

[kasper-coding]

I found that there was an issue creating the json output, when subscribing with wildcard to the mqtt topic. I dont understand how the output of lastValue has to look like. Is it sufficient to suply each value as a single json string or will I need to construct the whole output (as listed down below in the mqtt.py)? I modified the onmessage procedure to create and assign the string {'energy_active': 85031.3 } to lastValue. As the message topic already represents the correct names I tried to directly assign it.
Is that fine, respectively working that way? I did not connect the inverter yet to ensure no crappy data is loaded to the SE Portal.

Thank you!

Best Malte

def on_message(client, userdata, message):
    global lastValues

    topic=message.topic.rpartition('/')[-1] 
    m_decode=str(message.payload.decode("utf-8","ignore"))
    lastValues = "{'"+ topic +"': "+ m_decode +" }" 
    logger.debug(f"MQTT message received: {message.topic} ->  {message.payload.decode('utf-8')}")
    logger.debug(f"Object Json constructed: "+ lastValues)

[kasper-coding]

Rewrote the code... finally works now. If anyone is interested in the solution let me know.

Best Malte

[nmakel]

Hi! I don't have much experience with the mqtt device as it was contributed by @marcelrv. If there is anything you can contribute to make the use of this data source easier, please do!

[marcelrv]

It indeed expects a json formatted string with all the values in a single json.

I intentionally did not put too much logic into this, as everyone can have their own sort of mapping / logic. (e.g. on my local implementation I also do some data manipulations like multiplying and some additions/subtractions)

In your case, with single attributes provided in each mqtt message, you prob want to add each message to the lastValues object so you construct a object with all the values

[kasper-coding]

@nmakel @marcelrv Please find attached my current mqtt.py I dont know if this is useful as an example, but feel free to use and add wherever you like. I ended up creating a dictionary (was not aware of that technique before :) ) and initialized it with all the values my meter provides and which are also present on the mqtt server.

I was not so sure about what was really necessary in the end as an output, but this works obviously fine. I needed to set the meter type to wattmeter to get the proxy with my SE3000H.

Thanks again for that nice project!

import datetime
import json
import logging
import sys
import time

import paho.mqtt.client as mqtt

lastValues = {}
logger = logging.getLogger()

def on_connect(client, userdata, flags, rc):
    logger.info(f"Connected to MQTT: {userdata['host']}:{userdata['port']}/{userdata['meterValuesTopic']}")

    client.subscribe(userdata["meterValuesTopic"])

    global dictionary
    dictionary = dict()
    dictionary = {
        'energy_active': 0.00000000000,                               # kWh (total import + export)
        'import_energy_active': 0.00000000000,             # kWh
        'power_active': 0.00000000000,                            # W
        'l1_power_active': 0.00000000000,
        'l2_power_active': 0.00000000000,
        'l3_power_active': 0.00000000000,
        'voltage_ln': 0.00000000000,                               # V
        'l1n_voltage': 0.00000000000,
        'l2n_voltage': 0.00000000000,
        'l3n_voltage': 0.00000000000,
        'frequency': 0.00000000000,                                    # Hz
        'l1_energy_active': 0.00000000000,                            # kWh (import + export)
        'l1_import_energy_active': 0.00000000000,          # kWh
        'export_energy_active': 0.00000000000,               # kWh
        'l1_export_energy_active': 0.00000000000,
        'power_factor': 0.00000000000,
        'l1_power_factor': 0.00000000000,
        'l2_power_factor': 0.00000000000,
        'l3_power_factor': 0.00000000000,
        'power_reactive': 0.00000000000,
        'l1_power_reactive': 0.00000000000,
        'l2_power_reactive': 0.00000000000,
        'l3_power_reactive': 0.00000000000,
        'l1_current': 0.00000000000,
        'l2_current': 0.00000000000,
        'l3_current': 0.00000000000,
    }

    if userdata['willTopic'] is not None:
        client.publish(
            userdata['willTopic'],
            "MeterProxy Connected " + str(datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S"))
        )

def on_message(client, userdata, message):
    global lastValues

    topic=message.topic.rpartition('/')[-1] 
    m_decode=str(message.payload.decode("utf-8","ignore"))
    lastValues = "{'"+ topic +"': "+ m_decode +" }"
    dictionary[topic]= float(m_decode)
    logger.debug(f"MQTT message received: {message.topic} ->  {message.payload.decode('utf-8')}")
    logger.debug(f"Object Json constructed: "+ lastValues)

def on_disconnect(client, userdata, rc):
    if rc != 0:
        logger.info(f"MQTT disconnected unexpectedly: {rc}")

def device(config):

    # Configuration parameters:
    #
    # host              ip or hostname of MQTT server
    # port              port of MQTT server
    # keepalive         keepalive time in seconds for MQTT server
    # meterValuesTopic  MQTT topic to subscribe to to receive meter values

    host = config.get("host", fallback="localhost")
    port = config.getint("port", fallback=1883)
    keepalive = config.getint("keepalive", fallback=60)
    meterValuesTopic = config.get("meterValuesTopic", fallback="/haus/eg/har/hauszaehler/#")
    willTopic = config.get("willTopic", fallback=None)
    willMsg = config.get("willMsg", fallback="MeterProxy Disconnected")

    topics = {
        "host": host,
        "port": port,
        "meterValuesTopic": meterValuesTopic,
        "willTopic": willTopic
    }

    try:
        client = mqtt.Client(userdata=topics)
        client.on_connect = on_connect
        client.on_message = on_message
        client.on_disconnect = on_disconnect

        if willTopic is not None:
            client.will_set(willTopic, payload=willMsg, qos=0, retain=False)

        client.connect(host, port, keepalive)
        client.loop_start()
    except:
        logger.warning(f"MQTT connection failed: {host}:{port}/{meterValuesTopic}")

    return {
        "client": client,
        "host": host,
        "port": port,
        "keepalive": keepalive,
        "meterValuesTopic": meterValuesTopic,
        "willTopic": willTopic,
        "willMsg": willMsg
    }

def values(device):
    if not device:
        return {}

    return(dictionary)
    #  MQTT input is a json with one or more of the below elements
    # "energy_active"
    # "import_energy_active"
    # "power_active"
    # "l1_power_active"
    # "l2_power_active"
    # "l3_power_active"
    # "voltage_ln"
    # "l1n_voltage"
    # "l2n_voltage"
    # "l3n_voltage"
    # "voltage_ll"
    # "l12_voltage"
    # "l23_voltage"
    # "l31_voltage"
    # "frequency"
    # "l1_energy_active"
    # "l2_energy_active"
    # "l3_energy_active"
    # "l1_import_energy_active"
    # "l2_import_energy_active"
    # "l3_import_energy_active"
    # "export_energy_active"
    # "l1_export_energy_active"
    # "l2_export_energy_active"
    # "l3_export_energy_active"
    # "energy_reactive"
    # "l1_energy_reactive"
    # "l2_energy_reactive"
    # "l3_energy_reactive"
    # "energy_apparent"
    # "l1_energy_apparent"
    # "l2_energy_apparent"
    # "l3_energy_apparent"
    # "power_factor"
    # "l1_power_factor"
    # "l2_power_factor"
    # "l3_power_factor"
    # "power_reactive"
    # "l1_power_reactive"
    # "l2_power_reactive"
    # "l3_power_reactive"
    # "power_apparent"
    # "l1_power_apparent"
    # "l2_power_apparent"
    # "l3_power_apparent"
    # "l1_current"
    # "l2_current"
    # "l3_current"
    # "demand_power_active"
    # "minimum_demand_power_active"
    # "maximum_demand_power_active"
    # "demand_power_apparent"
    # "l1_demand_power_active"
    # "l2_demand_power_active"
    # "l3_demand_power_active"

[marcelrv]

Yes, exactly that is the way indeed I would do it as well in your scenario..