Still work in progress?

[h3llrais3r]

Hi. I'm exploring the ways to read live data from the saj h1 inverter and was wondering if this is a working method? I´ve been looking inside the code and I´m wondering where you got the saj topic from to get the data transmission.https://github.com/paolosabatino/saj-mqtt-ha/blob/20120d2cd51d29196b8a1925d3fcdcf872cdf179/utils/send_register.py#L293 When sniffing my network I only saw mqtt traffic on topic saj/<serial>/realtime_dataPlus wich also seems to be encrypted now (I read on the homeassistant forum that is orginally was not encrypted on the saj/<serial>/realtime_data topic). If you would have some documentation about the mqtt, it would be great if you could share it. Thx.

[paolosabatino]

Hello, there is no documentation about the mqtt protocol; realtime_data is a topic used by spontaneous transmission of data: the wifi dongle will transmit a message on that topic every five minutes by itself and you can't change the granularity in any known way.

data_transmission topic instead is used to initiate a raw modbus request to the inverter which will answer via data_transmission_rsp topic.

You can take a look to this other project for cleaner code about the transmit/receive MQTT protocol, in particular sajmqtt.py is a python library code to read and write modbus registers of the inverter (for the document, you can consult the homeassistant forums (in particular this post)

[h3llrais3r]

Thanks for the info. I have been sniffing my network and the realtime_data topic is no longer used... it's now the realtime_dataPlus topic. (I saw this when sniffing my nework for mqtt messages) And you're right, it's only transmitted every 5 minutes... no way to change it. That's why I'm exploring all other possibilities to get realtime data in realtime (read every few seconds).

So I was really wondering how you found out you had to send data to the data_transmission topic... I assume you found that somewhere on the internet? Any link?

Are you still using this MQTT custom integration yourself? Just wondering if it's still working before trying it out... 😉

PS: I already saw your other repo 'saj-mqtt' with the python scripts to read/write, but since I want to integrate them in HA, I thought I should use this custom integration. Or can those individual scripts also be used from HA to populate the sensors?

[paolosabatino]

Not found anywhere on internet, I did reverse engineering sniffing exchanged messages by one of the SAJ apps and inspected the protocol. At first I thought it was totally useless, but then it turned on that data_transmission and data_transmission_rsp are just encapsulated modbus messages (which is used by SAJ inverter to communicate with the AIO wifi module) into a MQTT message.

Yes, I'm using exactly this code for HA to read the values from the inverter; perhaps your wifi module has an updated software version which encrypts the realtime_data message, mine does not so everything is still plain data on my side.

The other project provides a convenient python library that use data_transmission and data_transmission_rsp mqtt topics and some utilities to read and write modbus registers. In particular, you can pipe registers read with readregs.py to parsedata.py to show a lot of realtime information from the inverter (see the README.md of the other project for usage).

This HA integration uses the same mqtt topics, so you may want to try the "barebones" and see if your wifi module still accepts those topics; if so, then you may want to install this integration.

[h3llrais3r]

This HA integration uses the same mqtt topics, so you may want to try the "barebones" and see if your wifi module still accepts those topics; if so, then you may want to install this integration.

That might indeed be a good idea. Thanks again for the feedback!

[h3llrais3r]

Did you ever think about configuring your local mqtt server as a bridge (to the remote saj mqtt server) so that we can still use the saj app as well? I think if we bridge the realtime data topic to the saj mqtt server, we can still use the app. I think I'm going to try this out, unless you tried it as well and it wouldn't work?

[paolosabatino]

No, I never thought about that because I wanted privacy for my data, hence I use HA in a self-hosted environment.

I don't link my data being held in a chinese server for which I never signed a contract of any kind. More than that, SAJ can control your inverter from remote and I think it is a super violation of private life since, in theory, they could bring down power to my own house. Moreover, since the protocol is plain and there is no encryption, it is very easy for an attacker other than SAJ to craft MQTT packets and misconfigure the inverter from remote, bypassing the simple NAT filter of the router.

[h3llrais3r]

I don't link my data being held in a chinese server for which I never signed a contract of any kind.

Good point. So if I change the mqtt server on the AIO3 module to my own local mqtt server, they won't be able to control my inverter remotely, right? But then that probably also means that I'll never receive firmware/software updates for the inverter? Unless that actually never happens?

Maybe with the bridge, I can configure it in such a way that only the realtime_data is published to the remote server. (need to check). If that would be possible, then only the data is pushed, and no more remote control (over mqtt) should be possible...

PS: Sorry for all the questions. I just have my installation since a few weeks, and I'm exploring all possibilities. 😉

[paolosabatino]

So if I change the mqtt server on the AIO3 module to my own local mqtt server, they won't be able to control my inverter remotely, right?

Can't be sure about that, also I don't know if the AIO3 server can be altered because the Android App allows it, but some people did that without any success.

I instructed my router to redirect messages to my local MQTT broker and trying to getting rid of plethora of broken things this inverter has.

But then that probably also means that I'll never receive firmware/software updates for the inverter? Unless that actually never happens?

Probably correct. And they don't give you the firmware or any chance to update on your own. Also, AFAIK, they update the firmware once you contact your local support center. I had a problem with excessive nightly battery draining, asked the support center and they said the firmware would be updated. It didn't happen immediately though, but after some days and without any kind of notice: the power just went away and got back after a few minutes.

They did not solve the battery draining issue at all, plus now I have the infamous v1.344 firmware with battery and grid current swapping I wrote in the README.md

[h3llrais3r]

Can't be sure about that, also I don't know if the AIO3 server can be altered because the Android App allows it, but some people did that without any success.

I instructed my router to redirect messages to my local MQTT broker and trying to getting rid of plethora of broken things this inverter has.

I might try it out and see what happens. Otherwise I'll try to rewrite the saj mqtt dns to my local mqtt server with adguard (it has the option to rewrite dns).

Hmm, will check this evening what firmware I'm actually running. It's probably only visibible in the app when you directly connect to the device... as I don't see it in the saj home app.

[h3llrais3r]

I might try it out and see what happens. Otherwise I'll try to rewrite the saj mqtt dns to my local mqtt server with adguard (it has the option to rewrite dns).

Rewrite dns with adguard to my local mqtt instance works and I also configured it as a bridge so the data is still pushed to the saj mqtt server (for now 😉) Actually, I only bridged the realtime_data topic, so the remote control by putting something to the data_transmission topic from remote server should not reach my inverter, as my local mqtt does not allow the bridging for it. 😉

I've also been able to test your scripts with success. The topics are still valid and data can be fetched and parsed. 👍

Hmm, will check this evening what firmware I'm actually running.

I seem to be on a older firmware version: v1.202. (You know on which firmware version you were before they pushed the update to that infamous v1.344 version)

[walkjivefly]

@h3llrais3r

Rewrite dns with adguard to my local mqtt instance works and I also configured it as a bridge so the data is still pushed to the saj mqtt server (for now 😉) Actually, I only bridged the realtime_data topic, so the remote control by putting something to the data_transmission topic from remote server should not reach my inverter, as my local mqtt does not allow the bridging for it. 😉

I'd like to try that. I did intercept the mqtt traffic from the inverter once by using Pihole to rewrite the DNS. But had no idea how to then bridge the data to the SAJ servers too, so reverted to the usual behaviour. Can you explain how to do that bridging? I'm running (but not yet actually using) eclipse-mosquitto on the same server as HomeAssistant.

My inverter is an H1-3.6K-S2 with firmware 1.361

[h3llrais3r]

My inverter is an H1-3.6K-S2 with firmware 1.361

Ok, I thought the infamous firmware was for the inverter... but it's for the battery I assume? My inverter is the H1-5K-S2 with firmware 1.202. For the battery I can see a software version, which seems to be 1.152

Can you explain how to do that bridging?

Based on https://solar-assistant.io/help/home-assistant/broker-setup

• Add DNS rewrite in adguard for mqtt.saj-solar.com to your mqtt ip address (I'm running the mosquitto addon in HA, so for me it's the HA ip)
• Add mqtt bridge configuration in HA mqtt addon configuration:
  • add saj mqtt username/password to list of allowed logins in mosquitto config (this is required as the inverter is sending these credentials to the mqtt server and I don't think there is any way to change that):

    logins:
    - username: "!secret mqtt_saj_user"
      password: "!secret mqtt_saj_password"

  • set customize: true and folder: mosquitto to mosquitto config
  • add /share/mosquitto/saj_mqtt_bridge.conf with following content to bridge saj app topic to still have access to the data from the saj app:

    connection saj-mqtt-bridge
    address 47.254.120.144
    remote_username <mqtt_saj_user>
    remote_password <mqtt_saj_password>
    topic saj/<serial>/realtime_data both
    topic saj/<serial>/realtime_dataPlus both

    • address contains the real ip of mqtt.saj-solar.com
    • remote_username contains the username used by saj to connect to mqtt.saj-solar.com
    • remote_password contains the password used by saj to connect to mqtt.saj-solar.com
    • topics are the list of topic we want to bridge (in my case it's saj/<serial>/realtime_dataPlus, but other firmwares might be using saj/<serial>/realtime_data)

With this setup, the app still gets the data, but I don't think remote control is still possible as I'm not bridging the data_transmission topic, so even if it's published on the remote broker, it won't be bridged as it's not part of the bridge topic config.

Only problem I can see now is that the ip address of mqtt.saj-solar.com can change and the bridge woudn't work anymore until the ip is updated in the bridge config. If I would be able to change the mqtt server on the aio3 module directly (I know you can change it via the app, but I never tested it yet) and it actually also keeps that value after a reboot, then the adguard dns rewrite could be removed and in the bridge config, I could put mqtt.saj-solar.com as address instead. Then it should work also when the ip address is changing at saj side.. So I still need to check if the mqtt server can be changed and if it's kept after a restart of the aio3 module.

PS: I suppose you know the credentials for saj mqtt? If not, I can share in PM.

[paolosabatino]

I opened the "github discussions" section, it is better to continue posting there.

Since this is the issue tracker and this is not strictly and issue, I close this but discussion can be continued there: https://github.com/paolosabatino/saj-mqtt-ha/discussions/4