HA mqtt failed to connect to mosquitto

[mjooris]

Hi muppet3000! thx for all the work you have put in to this. i gave it a try but i'm stuck. (probably my fault) maybe you have an idea.

so, i have a working GROTT i think i have a working mosquitto MQTT

Grott gets the data from my shinelink and works as a proxy (server.growatt.com also receives the data so thats good)

MQTT conf file:

Place your local configuration in /etc/mosquitto/conf.d/

#

A full description of the configuration file is at

/usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid allow_anonymous true listener 1883

persistence true persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

the mqtt log:

so the ip 192.168.0.202 is the container where i am having grott and mqtt running. so i believe grott is sending the data to mosquitto.

but when in home assistant i add the integration MQTT as in your documentation.

i'm getting this error:

so. i'm almost there. but here i am stuck.

my HA also runs in a docker container, in the same network segment. (192.168.0.242) so they can communicate with each other.

thx for your help!

[muppet3000]

It looks like you're running MQTT as a local installation, rather than in a container. Am I correct?

Assuming that Grott & MQTT are running on the same machine (192.168.0.202) that explains why Grott has no problem connecting (they're the same machine).

Do you have a firewall running on the 192.168.0.202 machine? Depending on your flavour of linux a command like systemctl status firewalld will give you a good idea (alternatively, just google it for your flavour).

If you have a firewall running you'll likely need to allow access on port 1883 on the local network (again, google should help here).

As a quick test though, you could disable the firewall e.g. systemctl stop firewalld and then re-try your connection information in HA, that would at least give you a quick confirmation that it was the firewall that was the problem. Once you've confirmed that, re-enable the firewall e.g. systemctl start firewalld and then you're good to go.

If I've mis-understood and you are in fact using docker containers for Grott and MQTT let me know and I'll have a re-think about what the issue could be.

[mjooris]

thx for the fast reply. i've downloaded the ledidobe-grott docker image. and made a container. in that container, i also installed the mosquito MQQT. (it works to access it via MQQT explorer) Home assistant does not want to connect to it.

i cannot realy find a way to disable the firewall for now. i thought it would be easyer to do it in one container. But it appears not. if i cant figure this out, i'll create a second docker container for my mqtt. it sounds like that should fix my issues. :)

[muppet3000]

ahhh you've done it all inside the same container.

Can you tell me the command you're using to run the docker container?

Also, if you're trying to wrap all of this into containers, I really recommend using the docker-compose approach to keep things separated and therefore you can take upgrades to individual components. Have you seen my details here (re: docker-compose): https://github.com/muppet3000/homeassistant-grott/blob/main/docs/setup/docker-compose-guide.md

[mjooris]

Hi,

sorry man, i have no idea to see the commands it uses. i'm running docker on my synology NAS so the setup is with the GUI, then i bash in to it to work inside the container. so i think this has some limitations. but i'll setup a separate docker container for MQTT and will test if that works better. i'll keep you posted!

[mjooris]

Alright, i've made a seperate container for mqtt:

docker run -d --name eclipsemosquitto \ -p 1883:1883 \ -p 9005:9005 \ -v /volume1/docker/mosquitto:/config/mosquitto.conf \ -v /volume1/docker/mosquitto:/data \ -v /volume1/docker/mosquitto:/log \ eclipse-mosquitto

with mqtt explorer i am able to access this instance.

Edit the IP in the grott.ini file to the new mqtt container. now i'm getting the error in grott:

• MQTT send failed: [Errno 113] No route to host

but it has a default route to my firewall (and that works, because growatt is still getting data forwarded by grott)

[jkkataja]

@mjooris based on your setup (assuming I understood it correctly), I believe you should set the Broker address in the Home Assistant MQTT configuration be the NAS IP address. Same goes for your grott.ini file for MQTT IP address.

Docker containers by default create the networking so that the address inside the container is not directly accessible but you need to use the host IP address to which you are mapping the ports to with those "-p" parameters -> "-p 1883:1883" effectively maps the host IP and port 1883 to be connected to the process (also using port 1883) inside the container.

[mjooris]

@jkkataja indeed when i use the same network as the host that would be the case. i've tried that.. but i also have a mac vlan in there so i can give every container its own IP address.

so grott is 192.168.0.202 from there i was able to ping 192.168.0.1 8.8.8.8 192.168.0.254 (everything) EXCEPT 192.168.0.242 (ip address of its host, the NAS) so i build a new mqtt container in the mac vlan. it has now ip 192.168.0.203 it is working! grott is receiving data from my inverter, it proxies it to growatt server AND it sends it to MQTT. i can see the growatt data in the MQTT server.

but, still... from my HA, no possible way to add mqtt integration.

as you can see:

my mqtt (192.168.0.203) has the growatt data, so grott forwards it to there, that all goes perfect.

but when adding mqtt in HA:

i assume the problem is the same as i had when running mqtt on the same IP as the host, i was not able to access it. my HA is running on the same IP as the host, so 192.168.0.242 and that is something i cannot change now, to much crap is coded with that ip. so i believe my problem is, that for a unknown reason. it is not possible to talk from the "host" 192.168.0.242 to the containers 192.168.0.203. and i dont understand why.

hope some of you have an idea..

thx already!

[mjooris]

damn it. as i thought, ssh't into my nas, i can ping whatever i want but NOT the docker containers.

so i think my config is correct as muppet3000's docu made us do. the problem is the NAS cannot communicate with the docker hosts.... crap. any ideas?

the network:

[muppet3000]

So I'm writing this from my phone so apologies if it's not formatted nicely.

Generally when you run docker on a single machine all of the containers are able to talk to one another using the bridge network, you don't even have to do port forwarding in a lot of cases.

In your scenario all of the things you want to talk to one another are containers all running on the same machine. Therefore, you should just be able to communicate between them all using their docker container names, these are either assigned to them manually or given dynamically.

My suggestion, exec into one of the containers and then try and ping another one using it's container name. I bet it'll resolve and then work for you. If the ping works then you should just be able to use the container name in your home assistant config and it'll "just work".

In your screen grab earlier from the NAS UI it looked like each of the names in bold is potentially the container name.

[mjooris]

@muppet3000 containers can ping each other, just host/client or client/host does not work. after some googling, the 'macvlan' i use to give containers separate ip's is to blame. i found this: https://blog.oddbit.com/post/2018-03-12-using-docker-macvlan-networks/ create a second macvlan and bridge it and set a static route for that network.... my god, and i was hoping for a simple fix fot those stupid chinese people that keep breaking our growatt integration ...

but, this could be my solution. but after f*cking around with it for hours, i dont dare to start that today. i hope i can try this workaround tomorrow. i'm a network guy but linux is not my stong point. i've learned a lot today by implementing all this :) finally i'll get there!

thx for the help everyone!

[muppet3000]

@muppet3000 containers can ping each other, just host/client or client/host does not work. after some googling, the 'macvlan' i use to give containers separate ip's is to blame. i found this: https://blog.oddbit.com/post/2018-03-12-using-docker-macvlan-networks/ create a second macvlan and bridge it and set a static route for that network.... my god, and i was hoping for a simple fix fot those stupid chinese people that keep breaking our growatt integration ...

but, this could be my solution. but after f*cking around with it for hours, i dont dare to start that today. i hope i can try this workaround tomorrow. i'm a network guy but linux is not my stong point. i've learned a lot today by implementing all this :) finally i'll get there!

thx for the help everyone!

Ok, best of luck, I'm a little confused about why you're trying to get the host to speak to the containers and vice versa. It's just all the containers that need to speak to each other.

To break it down as simple as possible, you just want 3 containers to be able to speak to each other with data coming in from the data logger for 1 of them and then the other one (HomeAssistant) to be accessible from outside of the host. From the way you've spoken you've already got data going in to Grott, and HomeAssistant running as a container viewable from outside. It's just a case of getting mqtt stuck in the middle and away you go. Rather than giving HA and Grott dedicated IPs on your home network can't you just use the default docker bridge network and then use the standard docker port forwarding to expose the necessary ports on the NAS box itself. They'll all have the same IP but the ports will be routed to the respective containers.

[mjooris]

@muppet3000 HA wast my first docker container. And it uses the same IP as the host. nas reachable on port 443, HA on port 8123

after a while i wat installing more containers and needed the same port for other containers, so i needed to give them a separate ip address, then i implemented the mac vlan for new containers.

so my HA has the same IP as the host, of the docker containers. thats why the host ip (HA) needs to talk with the containers :( bad design from my side. if i knew then, what i know now, i would not have done it that way. but, to change the IP of my HA. i'm not sure that will be a quick and easy thing to do... what you suggest could work. To use the bridge network for the mqtt als grott so my HA can access them. just need to make sure those containers dont use any ports, my nas also uses for something else. that thing does like everything in my house :p uses almost no power and can do so many things! i love that thing.

thx for thinking with me!

regards,

[mjooris]

@muppet3000 i rebuild the containers with the bridge network and it is working now! thx for that!!

now.. i have one more issue. i have no data about my battery. with the old integration you made i have all these sensors: with grott i have these:

any idea about how to get my battery data? (have a SPH4600 inverter)

[mjooris]

@muppet3000 figured it out. in your grott.ini example you have invtype=sph so i added that and a whole lot of extra values appeard.

to bad they all have a different name then before. so i have some work adapting things. but i'm so happy i finally have this working!

thx for making all of this!

best of luck with finetuning everything in the future :)

regards,

matthias

[muppet3000]

@muppet3000 figured it out. in your grott.ini example you have invtype=sph so i added that and a whole lot of extra values appeard.

to bad they all have a different name then before. so i have some work adapting things. but i'm so happy i finally have this working!

thx for making all of this!

best of luck with finetuning everything in the future :)

regards,

matthias

Oh wait, but there's more, I too was annoyed by the change of names (but it made sense to start from fresh). Look in the FAQs, it tells you EXACTLY how to template the values so you can have all the old names and not lose any of your history! (This is partly one of the reasons it took me so long to implement, I wanted to make sure it was backwards compatible!) Let me know how you get on with it.

Here: https://github.com/muppet3000/homeassistant-grott/blob/main/docs/FAQ.md

[mjooris]

@muppet3000 figured it out. in your grott.ini example you have invtype=sph so i added that and a whole lot of extra values appeard. to bad they all have a different name then before. so i have some work adapting things. but i'm so happy i finally have this working! thx for making all of this! best of luck with finetuning everything in the future :) regards, matthias

Oh wait, but there's more, I too was annoyed by the change of names (but it made sense to start from fresh). Look in the FAQs, it tells you EXACTLY how to template the values so you can have all the old names and not lose any of your history! (This is partly one of the reasons it took me so long to implement, I wanted to make sure it was backwards compatible!) Let me know how you get on with it.

Here: https://github.com/muppet3000/homeassistant-grott/blob/main/docs/FAQ.md

It wasn't too bad to change the names. I have a second inverter (goodwe) so for most of my automations i use a self-made variable where I sum up the total output of both inverters. so I only had to change it in my formula and all the rest kept working afterward :) it was just for the SOC I had to change it in some places. but after all that f*cking around with docker containers for days (which are definitely NOT my specialty!), I was happy to change some things I am familiar with :p

I LOVE that I have data now every minute. I am very happy with it!. now let's hope our Chinese friends don't push some kind of update to the inverter/shine link/wifi sticks/... that brakes this solution too.

like I said, I also have a Goodwe inverter. I've addid it to HA once. and the integration never stopped working ever since. it is sad Growatt doesn't play along with the people who want to make nice things..

I'ts a good thing we have people like you to find workarounds :D

[muppet3000]

Very happy to hear all of this from you, glad you managed to get it all working nicely, especially with all the information going into HA from both of your inverters. Fingers crossed Growatt don't do anything to upset things again!