Only some OBIS Data Fields are available ISKRA MT175

[HansPeterMauderer]

Hello Team,

First of all many thanks for this great tool :)

running sml2mqtt -c path_to_my_config_file -a results in

| <SmlMessage>
|   transaction_id: 05b61be4
|   group_no      : 0
|   abort_on_error: 0
|   message_body <SmlOpenResponse>
|     codepage   : None
|     client_id  : None
|     req_file_id: 01e75ea2
|     server_id  : 090149534b0004812d55
|     ref_time   : None
|     sml_version: None
|   crc16         : 25225
| <SmlMessage>
|   transaction_id: 05b61be5
|   group_no      : 0
|   abort_on_error: 0
|   message_body <SmlGetListResponse>
|     client_id       : None
|     sever_id        : 090149534b0004812d55
|     list_name       : 0100620affff
|     act_sensor_time : 42350284
|     val_list:
|       <SmlListEntry>
|         obis           : 8181c78203ff (129-129:199.130.3*255)
|         status         : None
|         val_time       : None
|         unit           : None
|         scaler         : None
|         value          : ISK
|         value_signature: None
|         -> (Hersteller-Identifikation)
|       <SmlListEntry>
|         obis           : 0100000009ff (1-0:0.0.9*255)
|         status         : None
|         val_time       : None
|         unit           : None
|         scaler         : None
|         value          : 090149534b0004812d55
|         value_signature: None
|         -> (Geräteeinzelidentifikation)
|       <SmlListEntry>
|         obis           : 0100010800ff (1-0:1.8.0*255)
|         status         : 65920
|         val_time       : None
|         unit           : 30
|         scaler         : -1
|         value          : 56718296
|         value_signature: None
|         -> 5671829.6Wh (Zählerstand Total)
|       <SmlListEntry>
|         obis           : 0100010801ff (1-0:1.8.1*255)
|         status         : None
|         val_time       : None
|         unit           : 30
|         scaler         : -1
|         value          : 56718296
|         value_signature: None
|         -> 5671829.6Wh (Zählerstand Tarif 1)
|       <SmlListEntry>
|         obis           : 0100010802ff (1-0:1.8.2*255)
|         status         : None
|         val_time       : None
|         unit           : 30
|         scaler         : -1
|         value          : 0
|         value_signature: None
|         -> 0.0Wh (Zählerstand Tarif 2)
|       <SmlListEntry>
|         obis           : 0100020800ff (1-0:2.8.0*255)
|         status         : None
|         val_time       : None
|         unit           : 30
|         scaler         : -1
|         value          : 54219342
|         value_signature: None
|         -> 5421934.2Wh
|       <SmlListEntry>
|         obis           : 0100020801ff (1-0:2.8.1*255)
|         status         : None
|         val_time       : None
|         unit           : 30
|         scaler         : -1
|         value          : 54219342
|         value_signature: None
|         -> 5421934.2Wh
|       <SmlListEntry>
|         obis           : 0100020802ff (1-0:2.8.2*255)
|         status         : None
|         val_time       : None
|         unit           : 30
|         scaler         : -1
|         value          : 0
|         value_signature: None
|         -> 0.0Wh
|       <SmlListEntry>
|         obis           : 0100100700ff (1-0:16.7.0*255)
|         status         : None
|         val_time       : None
|         unit           : 27
|         scaler         : 0
|         value          : 23
|         value_signature: None
|         -> 23W (aktuelle Wirkleistung)
|       <SmlListEntry>
|         obis           : 8181c78205ff (129-129:199.130.5*255)
|         status         : None
|         val_time       : None
|         unit           : None
|         scaler         : None
|         value          : 55ee18e385e7aa763de1b81508f198e40e495f1ef707f779be518456b0f293674b06d0ea4060f11f2b6f6fb5d1c7ae62
|         value_signature: None
|         -> (Öffentlicher Schlüssel)
|     list_signature  : None
|     act_gateway_time: None
|   crc16         : 53301
| <SmlMessage>
|   transaction_id: 05b61be6
|   group_no      : 0
|   abort_on_error: 0
|   message_body <SmlCloseResponse>
|     global_signature: None
|   crc16         : 19825
|

cut something out in between here !

| sml2mqtt/ISKRA MT175 Main/0100010800ff (0100010800ff):
|   raw value: 5671.8296
|   pub value: 5671.8296
|   filters:
|     - <Every: 120>
|     - <OnChange>
| sml2mqtt/ISKRA MT175 Main/0100020800ff (0100020800ff):
|   raw value: 5421.9342
|   pub value: 5421.9342
|     - <Every: 120>
|     - <OnChange>
| sml2mqtt/ISKRA MT175 Main/0100100700ff (0100100700ff):
|   raw value: 23
|   pub value: 23
|     - <Every: 120>
|     - <OnChange>

We found 3 issues for this specific SMA meter

1. the obis field name contains spaces i.e obis : 8181c78205ff (129-129:199.130.5*255)

acces to these datafields seems only be possible via config.yml file is we use the first part before the 'space'

2. there are only mqtt-events generated for the following fields sml2mqtt/ISKRA MT175 Main/0100010800ff (0100010800ff): sml2mqtt/ISKRA MT175 Main/0100020800ff (0100020800ff): sml2mqtt/ISKRA MT175 Main/0100100700ff (0100100700ff):

We have NO filters inside our config file Is there a "hidden filter" somewhere else ?

We are interested in some more information, it looks like they are available but not accessable

3. installation on latest raspiban 64bit bullseye required some extra installation of python libraries

   python3 -m pip install --upgrade pip sml2mqtt
   python3 -m pip install typing-extensions --upgrade
   python3 -m pip install ruamel_yaml
   python3 -m pip install smllib
   python3 -m pip install asyncio_mqtt
   python3 -m pip install pyserial_asyncio

br hp

[spacemanspiff2007]

1. the obis field name contains spaces i.e obis : 8181c78205ff (129-129:199.130.5*255)

acces to these datafields seems only be possible via config.yml file is we use the first part before the 'space'

What do you mean exactly? I don't understand the issue. Can you make a full example?

We have NO filters inside our config file Is there a "hidden filter" somewhere else ?

Yes - duplicate values and empty meters are not published. What value specifically are you missing?

installation on latest raspiban 64bit bullseye required some extra installation of python libraries

That is most likely a bug 😕

[HansPeterMauderer]

Attached some more information,

Our setup is: 2 SML Energy meters should be logged -> basically this works despite some information is missing

I reduced the test-case to the minimium setup:

1. Only one SML Meter ist logged

2. removed everything from config-fle (Config-file is attached ) iskra-mt175-smartmeter.yml.txt

3. fired command sml2mqtt -c iskra-mt175-smartmeter.yml -a

The log is attached: sml2mqtt-config-file-a.log

we can see all provided obis fields in the log received from the sml device, but not all are generated as mqtt object

If we are feeding this mqtt stream to IOBroker with mqtt plugin, we see only the mqtt objects reported in the log file attached.

If we are using a SML plugin in IOBroker we are seeing all information including missing ones: (Manufacaturer ID, Public Key, and Power Values = 0, ..) Masking of values with '0' could be a problem for us, because we are installing plugins via scripts and doing script based calculations. If you wer are casacding i.e. some smartmeters, we get errors if we are calculating values via scripts and theese values are not available even if they exist.

Do you need any more information or data ? br hp

[spacemanspiff2007]

(Manufacaturer ID, Public Key, and Power Values = 0, ..)

Sorry - I still don't fully understand the issue. Manufacturer ID and Public Key are constant and do never change. Energy meters that are 0 are constant and also never change. It doesn't make any sense to report any of these values.

If you want do do calculations with these energy meters you can always just assume the non-existing value is 0.

[HansPeterMauderer]

Ok, you are right from technical point, but some more information what we are doing and what we want to do:

We have multiple SML meters, all data is currently logged into influxdb database via iobroker and a plugin. Each time we get a new SML meter, we have to manually install a new instance of this adapter in iobroker and do the configuration via GUI by hand. We want get rid of GUI configuration from the complete system, just only config files. This is just to be able to log the data

Inside the influx database we currently have the values of "Manufacturer ID", "Import Power", "Export Power". Some People have a Photovoltaik-Generator on their roof, others not. Currently we can do all with a simple script:

1. Based on Manufacturer ID (it does never change) we know to which flat the SML Meter is assigned. (there is no other information in the data base available, i.e. USB Port connected) 2 "Consumed Energy" is always "Export-Energy"(could be "0" in case of no PV-Generator) minus "Import-Energy"

We have only one Script currently without adding "support" for each specific SML Configuration. the math is always the same. Each change in the hardware configuratio nwould currently also imply a change in the analytics software ...

If we have the perfect solution already why asking for changes of "sml2mqtt" ?

With your tool we could make live much more easy! We want to get rid of manual configuration of iobroker each time when there is a hardware change or a SML change. The big benefit of your tool is the *.yml configuration file. we can just adapt it, reload and this is it. (also configuration can be stored in our revision control system, no manually clicking ...)

One solution could be to add an extra Key-Word in the Config-File, which just enables "RAW OBIS elements" or "Behaves like current" But adding this switch is from my point of view not necessary (maybe for legacy) , because sml2mqtt has already a "filter" option to mask OBIS elements. It's a really cool tool which could help us a lot

[spacemanspiff2007]

But wouldn't the device ID be a much better way to map Energy Meters to a flat? The device ID is already part of the topic and can even be set to a proper name. That way you can report e.g. sml2mqtt/flat_1/energy

I understand what you are trying to do but still don't understand the problem. Since you process the current energy consumption in a script and mqtt is asynchronous it should be trivial to implement what you are trying to achieve.

Pseudocode

energy_import = 0
energy_export = 0
energy_consumed = 0

function on_mqtt_msg_energy_import(value):
    energy_import = value
    update_energy_consumption()

function on_mqtt_msg_energy_export(value):
    energy_export = value
    update_energy_consumption()

function update_energy_consumption()
    energy_consumed = energy_import - energy_export

    more_logic_here()

You do not know when one of the values will be updated through mqtt so you have to cache the values anyway. By initializing the value to zero everything will work without changes.

Would this not work exactly as you described?

[HansPeterMauderer]

We are running all backend scripts inside influx on the database data of influx. NOT on the received MQTT data before we are writing to influx. (Extra effort)

That's exactly what we want to avoid.

Our current concept relies that we have the same data elements for each SML Device regardless some values are static or not. If not all elements are available in influx we would have moved the logic level of decoding to influx scripting.

dataset consist of: MANUFACTURER_ID DEVICE_ID (Allows via Lookup table to assing to flat) EXPORTED_ENERGY IMPORTED_ENERGY

we are "looping with a flux script inside influx over the database and do all backend services. If we come to the situation that not all elem are available in a dataset we have to start with conditional coding what we want to avoid

[spacemanspiff2007]

We are running all backend scripts inside influx on the database data of influx. NOT on the received MQTT data before we are writing to influx. (Extra effort)

That makes the issue clear - thank you!

I guess it's possible to add a config switch "report empty energy meters" which is set to "False" as a default.

Edit: How so you map topic to influx key?

[HansPeterMauderer]

ok fine thanks, its' a really cool tool :)

Maybe this would cover the complete issue: add a config switch "report static energy meters" which is set to "False" as a default.

This would allow to report all elements, at least one time during connection to the broker Thanks a lot for this tool and your support

br hp from germany

[spacemanspiff2007]

I've added report blank energy meters and report device id under general. It is not created by default, but you can add it manually in your config. Could you please test the current master and see if this works for you as intended?

[HansPeterMauderer]

Hello spaceman yes it's working as intended, thanks a lot br hp