jens-maus / RaspberryMatic

:house: A feature-rich but lightweight, buildroot-based Linux operating system alternative for your CloudFree CCU3/ELV-Charly 'homematicIP CCU' IoT smarthome central. Running as a pure virtual appliance (ProxmoxVE, Home Assistant, LXC, Docker/OCI, Kubernetes/K8s, etc.) on a dedicated embedded device (RaspberryPi, etc.) or generic x86/ARM hardware.
https://raspberrymatic.de
Apache License 2.0
1.54k stars 190 forks source link

HmIPW-DRAP/HmIP-HAP cannot identify RaspberryMatic as a HA addon #1373

Open ThomasSiegmund opened 3 years ago

ThomasSiegmund commented 3 years ago

Running RaspberryMatic in a standard Docker subnetwork on the host prevents a DRAP from connecting ...

Steps to reproduce the behavior

  1. Configure a DRAP to connect to a standalone RaspberryMatic. Make sure it can connect and the LED lights up in turquoise
  2. Switch off the DRAP
  3. Move your RaspberryMatic installation into a Docker container, e.g. by creating a backup and restoring from it in a freshly set up Docker installation according to the installation guide. I used the docker-compose variant of the docker setup.
  4. Power up the DRAP The DRAP goes to the usual sequence of yellow, green and turqoise lights. In the end it flashes in turquoise, indicating that it wants to connect to CCU but does not find it.

Expected behavior The DRAP should discover the CCU and connect to it.

System information:

Additional context The issue seems to be related to network discovery by UDP broadcasts. I noticed that RaspberryMatic running on bare metal is visible to EQ3's Netfinder program, but RaspberryMatic running under Docker ist not. So I watched network traffic during Netfinder discovery.

In this example Netfinder is on 10.10.15.1, and two DRAPs are running on 10.10.15.100 und 10.10.15.101

No. Time    Source  Destination Protocol    Length  Info
4   0.648462575 10.10.15.1  255.255.255.255 UDP 52  43537 → 43439 Len=10
6   1.348330504 10.10.15.100    255.255.255.255 UDP 103 43439 → 43537 Len=61
7   1.410798515 10.10.15.101    255.255.255.255 UDP 103 43439 → 43537 Len=61

You can see Netfinder sending a UDP broadcast package, which is immediately answerded by the two DRAPs. I guessed that a similar mechanism might also be used by the DRAP at startup to find the CCU. However, this will fail on a standard Docker setup, because Docker does not support this kind of broadcast messages. The only easy way to fix this I know of, is to run the container in host mode like with this docker compose file:

version: "3.7"
services:
  raspberrymatic:
    image: ghcr.io/jens-maus/raspberrymatic:latest
    container_name: ccu
    hostname: homematic-raspi
    privileged: true
    restart: unless-stopped
    stop_grace_period: 30s
    network_mode: host
    volumes:
      - ccu_data:/usr/local:rw
      - /lib/modules:/lib/modules:ro

Here network_mode: host makes the container use the host network without any internal network or port mapping. Just adding this line to the docker-compose.yaml and restarting the container makes the DRAP connect and to go from flashing to solid turquoise light.

Interestingly, in my experience this problem exists only for the DRAPs. HAPs when run as a range extender to a CCU seem to have a different mechanism of network discovery.

jens-maus commented 3 years ago

Thanks for this nice elaboration, this is highly appreciated.

After some investigation, this (as you outlined) can be quite easily solved with adding network_mode: host in the docker/OCI use case. And I think to "solve" it for native docker/OCI users, we simply have to tune the documentation to some extend and could even adapt the deploy.sh script to allow to run docker using the necessary --net=host option so that the container will use the host network accordingly.

However, the same limitation exists in the HomeAssistant addon use case in principle, but seems to be quite harder to solve, I am afraid. While in principle, the HA addon could also be switched to the same network_mode: host mode using a home assistant addon config setting (see https://developers.home-assistant.io/docs/add-ons/configuration/#add-on-config), this could immediately negatively affect other addons running on the same HomeAssistant host. In addition, the internal RaspberryMatic iptables firewall could then quite easily mixup/break the whole host firewall setup if the RaspberryMatic HA addon will run on the host network of HomeAssistant. AFAIK, there is unfortunately indeed no solution to get a docker container to accept UDP broadcast messages that are received by the actual docker host. Perhaps @pvizeli or some other HA dev has some other idea how to forward UDP broadcast packages to the docker containers in HA without having to set host_network: true in the config.json addon config file. Otherwise, I am currently quite puzzled how to consistently solve this for the HA addon use case.

pvizeli commented 3 years ago

If you know the port, then you just need to set it. If it's multicast, then it could pick-up in future by the new multicast plugin

jens-maus commented 3 years ago

Thanks @pvizeli for the hints. Where exactly do I have to set the multicast port? In the config.json of the addon? Do you have some example you can show? And when will the new multicast plugin be part of HA or does a HA user have to install it himself to get udp multicast routing into a HA addon going?

TVR-X commented 3 years ago

Is it possible to test a modified config.json file with "network_mode: host" inside? I can't find a way to modify this on a Addon from a repository because i think this is the same issue like in #1327.

jens-maus commented 3 years ago

Is it possible to test a modified config.json file with "network_mode: host" inside? I can't find a way to modify this on a Addon from a repository

Sorry, but it is not possible for a user to override the network_mode: host for a published add-on. I, as a add-on developer would have to do that in the config.json of the add-on itself. However, I actually tested this myself and just using network_mode: host will end up in other issues that you seeing because of the add-on generally running on the host network (e.g. the network firewall will be modified by the add-on). Thus, I am still waiting for @pvizeli to answer my question and help in getting this multicast issue somewhat solved in the near future.

pvizeli commented 3 years ago

You can asign the port on options but set it to null for the host, so people can just enable it if they enter the port manual. If it using multicast, then it should be solved with https://github.com/home-assistant/plugin-multicast/issues/17

jens-maus commented 3 years ago

You can asign the port on options but set it to null for the host, so people can just enable it if they enter the port manual.

This is already the case, indeed. However, this alone does not seem to help in the end as this issue here shows.

If it using multicast, then it should be solved with home-assistant/plugin-multicast#17

Sorry to bother you, but where exactly does pimd to be installed so that the multicast routing from the host to/from the add-on container works? I am really lacking some more detailed information/know-how here to get this working with HA. As said, right now the relevant udp multicast port (43438) is already defined null in the config.json file: https://github.com/jens-maus/RaspberryMatic/blob/master/home-assistant-addon/config.json#L78

But it seems this alone does not help in getting the multicast traffic from the HA host to the add-on container and vice versa. So any more detailed hint in getting the udp multicast traffic properly routed between the HA host and the add-on container would be highly appreciated.

dkadioglu commented 3 years ago

I am not sure, if the following is of help or does have any short comings, so I just would like to describe my approach and I would be happy, if it helps as well as about feedback. I have a HMIP-HAP working with Raspberrymatic in a Docker container on a RPi4 without using network_mode: host. Instead, I am using the macvlan driver with a exclusive network for the Raspberrymatic container.

This is my docker-compose file:

version: "3.8"
services:
  raspberrymatic:
    image: ghcr.io/jens-maus/raspberrymatic:latest
    container_name: dkhm
    hostname: dkhm
    privileged: true
    restart: unless-stopped
    stop_grace_period: 30s
    volumes:
      - /home/alarm/dkhm_data:/usr/local:rw
      - /lib/modules:/lib/modules:ro
      - /run/udev/control:/run/udev/control
      - /mnt/storage/HMBackup:/mnt:rw
    networks:
      default:
        ipv4_address: 192.168.1.221
networks:
  default:
    external:
      name: hmnet

Before this works, I created the following network. It is important, that subnet is the same as the hosts network and to take care, that ip-ranges of the host network and the to be created Docker network do not collide:

docker network create -d macvlan -o parent=eth0 \
  --subnet 192.168.1.0/24 \
  --gateway 192.168.1.1 \
  --ip-range 192.168.1.220/30 \
  --aux-address 'host=192.168.1.222' \
  hmnet

To allow communication between the host and the Raspberrymatic container, the following additional steps are necessary - as the Docker firewall configuration seems to block this, as far as I understand it:

ip link add hmnet-shim link eth0 type macvlan mode bridge #create a virtual network device hmnet-shim bound to eth0

ip addr add 192.168.1.222/32 dev hmnet-shim #set the above mentioned (aux-address) ip address of hmnet-shim
ip link set hmnet-shim up

ip route add 192.168.1.220/30 dev hmnet-shim protocol static #add the necessary static route between host and hmnet-shim

As the above four lines don't survive a reboot, I created a systemd service for this:

[Unit]
Description=Create hmnet device and add static route
Requires=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/bin/bash -c '/usr/bin/ip link add hmnet-shim link eth0 type macvlan mode bridge && /usr/bin/ip addr add 192.168.1.222/32 dev hmnet-shim && /usr/bin/ip link set hmnet-shim up && /usr/bin/ip route add 192.168.1.220/30 dev hmnet-shim protocol static'

[Install]
WantedBy=multi-user.target

With this approacv, the communication between Raspberrymatic and the HMIP-HAP works as it should. The turquoise led is solid and not blinking. The only thing I didn't test is, if the setup process (in german "Anlernen") between Raspberrymatic and HMIP-HAP works, as I did that with network_mode: host and I can't redo the process currently. Maybe somebody else can give feedback, if the setup process works with macvlan, too.

I hope this helps.If you've got questions, please ask and I try to follow up timely.

jens-maus commented 2 years ago

FYI: I just tested the issue myself again and tried to use eQ-3 NetFinder to identify a running RaspberryMatic CCU as a HomeAssistant add-on. As soon as I specify port 43438/udp in the add-on config to be forwarded to the add-on NetFinder is perfectly able to identify the RaspberryMatic also within a HomeAssistant environment. See:

img

The only thing that is missing/strange is, that this virtual CCU is listed with "refer to help" in the reachability column. This might be, however, due to the fact that this virtual CCU cannot be pinged/reached directly form the system running thie eQ-3 NetFinder tool.

Regarding reachability of the virtual CCU from a HmIPW-DRAP or HmIP-HAP, the situation can be easily similar and needs to be tested. Note, however, that for the DRAP to identify/find your CCU you would have to open port 43439/udp as well.

THorst92 commented 2 years ago

Hi @jens-maus,

I don't know if you've received already a feedback to the reachability of the virtual CCU from a HmIPW-DRAP or HmIP-HAP. I've tried to connect a HmIP-HAP to RaspberryMatic which runs as AddOn in Home Assistant without success.

SW-Version:

In RaspberryMatic all ports are open. The ports 9293, 43438 and 43439 are activated in the configuration of the RaspberryMatic AddOn in Home Assistant.

When I reset the HMIP-HAP twice and try to activate the pairing mode, the LED stays static blue (or yellow if I block the internet connection). In RaspberryMatic I was not able to teach in the HMIP-HAP (local and with internet connection). Also in the option "Access point with incompatible firmware" the access point was not displayed.

Do you have an idea what I can try next?

jens-maus commented 2 years ago

Do you have an idea what I can try next?

Nothing really. If you read through this issue ticket closely you will find reference to a HomeAssistant related ticket (https://github.com/home-assistant/plugin-multicast/issues/17) which is related to the root cause of this issue: The HmIPW-DRAP and HMIP-HAP rely on UDP multicasting network traffic which is currently not correctly routed to an add-on in HomeAssistant. Thus, the HomeAssistant developers (e.g. @pvizeli) would have to work on this by enhancing their plugin-multicast accordingly. That's the reason why this (no connectivity to a HmIPW-DRAP or HmIP-HAP) is currently a known limitation of using the RaspberryMatic add-on within HomeAssistant.

firstusing commented 2 years ago

@jens-maus thanks for your efforts. I have rebuilt my complete smart home to get the Rasperrymatic installed as Addon within HA. I was reading slowly thru your guideline and had luck to see the issue before I did start. Nothing within my home would work without DRAP and the Second HAP. What can we do to trigger the issue so they will get manpower to the ticket?

Thank you!

jens-maus commented 2 years ago

What can we do to trigger the issue so they will get manpower to the ticket?

If you cannot work on it yourself you can just lean back and wait until someone else is going to implement the missing piece.

vore commented 2 years ago

What can we do to trigger the issue so they will get manpower to the ticket?

Hello! Same here.. I waiting for this in order to switch to Homeassistant. Until I can't use the DRAP I can't change over. Unfortunately this is nothing I could help to implement.

unaiur commented 2 years ago

Warning: multicast and broadcast are not the same!

With multicast, a client has to register to receive the stream of packets. Multicast destination addresses are always in the 224.0.0.0-239.255.255.255 range.

With broadcast, the packet is received by all the hosts in the local network without any extra step. The destination address is the broadcast address of the current subnetwork (for example 192.168.1.255)

After reading this thread, it seems that HMIP uses broadcast, not multicast. So waiting for pmid is pointless.

unaiur commented 2 years ago

Therefore, we just need to use a macvlan docker network in bridge mode as described here: https://docs.docker.com/network/macvlan

Which is the same proposal that @dkadioglu did 8 months ago and all you ignored.

jens-maus commented 2 years ago

Warning: multicast and broadcast are not the same!

Of course not.

After reading this thread, it seems that HMIP uses broadcast, not multicast. So waiting for pmid is pointless.

What exactly from this thread here makes you feel this is actually the case? In fact, I still believe the HAP/DRAP are both relying on UDP multicasting and not broadcasting communication channels and thus haveing pmid running within the HA host should actually solve this issue.

Therefore, we just need to use a macvlan docker network in bridge mode as described here https://docs.docker.com/network/macvlan

Well, this won't help in the HomeAssistant Add-on case, I am afraid since HA does not support to start addons with macvlan. So, yes. the suggestions by @dkadioglu could be added to the documentation so direct docker users could try to get this working the way he was proposing it. But for the HomeAssistant add-on case we need a different solution.

Which is the same proposal that @dkadioglu did 8 months ago and all you ignored.

Well, you can also come up with an own proposal or pull request if you identified a solution for the HAP/DRAP communication issue mentioned here.

ivo7B4 commented 2 years ago

Thank you @jens-maus for all the your efforts and your great work, highly appreciated!

I'm still getting into HA, CCU3 and Raspberrymatic as I'm preparing everything for installation.

There's an observation I have where I'm not sure whether this is related to this issue:

I thus don't know where this issue comes from:

campinge commented 2 years ago

Hello all! This has been pending for a while now and I just encountered the same problem. Does anyone know if it can be solved somehow in the meantime?

jens-maus commented 2 years ago

As long as this ticket is open and not closed there is no other solution than not using the HA addon of RaspberryMatic if you want to connect it to a HmIPW-DRAP or HmIP-HAP.

vore commented 2 years ago

Is it done now? Looks good from what I see. Anybody who can test and confirm?

jens-maus commented 2 years ago

Is it done now?

I don't think so. It's done when it's done, thus when this ticket is closed!

vore commented 1 year ago

Is it done now?

I don't think so. It's done when it's done, thus when this ticket is closed!

Hello! The ticket is still open.. it does not look like it is going to be closed. Should I maybe ask on the current status?

jens-maus commented 1 year ago

Hello! The ticket is still open.. it does not look like it is going to be closed. Should I maybe ask on the current status?

No reason to constantly ask for the status here. As already said in october: It's done when it's done. And if there is something relevant to report it will be and if the issue is solved, the ticket will be closed. Thus, no need to further discuss on the status.

Rainerlan commented 1 year ago

I kind of having the same problem: The HA RaspberryMatic-addon does not find the HMIP-HAP - as I read here, because its in another subnet (The LAN-Gateway is found w/o problems however).

I thought of the following solution - But before trying this out, I wanted to check with more experienced people, if the following setup may work:

Any thoughts if this might be working? (Don't want to risk my running setup, if it may be evident that this is not going to work as intended)

jens-maus commented 1 year ago

I thought of the following solution - But before trying this out, I wanted to check with more experienced people, if the following setup may work:

  • As my HA (OS) runs in a Proxmox VM (already), I would add another LXC to install RaspberryMatic as a docker container.
  • Configure host mode for the Docker container
  • Select the LXC-RaspberryMatic instance in the RaspberryMatic HA integration.

Any thoughts if this might be working? (Don't want to risk my running setup, if it may be evident that this is not going to work as intended)

This is no discussion or support fora here but a bug/issue tracker. But yes, if you install RaspberryMatic as a dedicated VM (LXC does not work) it will work because then of course no docker is involved which is the root cause of the issue that a DRAP/HAP connection does not work with HA (simply because docker containers can't route multicast UDP traffic).

marce14-git commented 1 year ago

Sorry in advance if my response is not appropriate here: i got it working in a docker container in unraid via a macvlan docker network Has to be running in privileged mode

jens-maus commented 1 year ago

Sorry in advance if my response is not appropriate here: i got it working in a docker container in unraid via a macvlan docker network Has to be running in privileged mode

Sure, there are solutions for other docker environment. But the topic of this issue ticket is more than clear: Get it working in a standard HomeAssistant environment. And there, macvlan is a non avail!

lukibazuki commented 1 year ago

So if I understand this correctly, I'll have to run raspberrymatic on a separate device (not as Add-On in HA), so I can use the HAP as a Gateway. With the HACS Integration "Homematic(IP) Local" I'll still have the possibility to control everything through HA, correct? Is there any workaround to get the HAP working at this moment?

jens-maus commented 1 year ago

Please note, that I was finally able to work on that topic again and now at least for the plain docker container / OCI use case it should be possible to use a HmIP-HAP or HmIPW-DRAP if following my updated documentation at https://github.com/jens-maus/RaspberryMatic/wiki/Installation-Docker-OCI and potentially also using the new install-docker.sh helper script as explained there.

This script (and the docu) utilizes a macvlan docker network setup now like previously explained by @dkadioglu (cf. https://github.com/jens-maus/RaspberryMatic/issues/1373#issuecomment-922508274) and thus should allow the docker container to communicate via UDP multicasting to a HmIP-HAP or HmIPW-DRAP and also make it unnecessary to open dedicated ports because the container will receive a dedicated ip address from the same subnet and thus should act as a "real" RaspberryMatic CCU (also with its own firewall setup).

However, please note, that for the HomeAssistant Add-on use case the issue/limitation discussed here still applies because it is currently not possible to setup the HA add-on docker container to work on dedicated docker networks, thus not being able to use a macvlan docker network. This is still something the HA developers (e.g. @agners, @pvizeli) are working on.

jens-maus commented 1 year ago

FYI: After some more work and tests with HomeAssistant I was able to create a manual patch procedure to create the necessary macvlan docker bridge also within HomeAssistant and assign the RaspberryMatic add-on to use this macvlan bridge, thus being directly connected to the respective LAN subnet. See here for the respective documentation on how to apply this patch procedure on your HomeAssistant / RaspberryMatic add-on installation while the HA developers are still working on directly providing this functionality for add-ons like ours here:

https://github.com/jens-maus/RaspberryMatic/wiki/Installation-HomeAssistant#hmip-haphmipw-drap-support-patch

After applying this patch procedure to a running RaspberryMatic add-on it should be indeed reachable via the assigned ip address, thus communication with a local HmIP-HAP or HmIPW-DRAP in the same LAN segment should then work as expected. Note, however, that there still exists a quite strong limitation for this patch procedure, namely the point that upon reboot of HomeAssistant or even restart of the RaspberryMatic add-on one has to manually re-apply this patch procedure.

Nevertheless, if there are some HA/RaspberryMatic users out there waiting for HmIP-HAP/HmIPW-DRAP support, feel free to test this manual patch procedure and please report back if this solves your issues until the HA developers are directly integrating a more convenient solution in a future version.

campinge commented 1 year ago

Very well done, Jens. I Backed up my Raspberrymatic environment and installed HomeAssistant on a fresh hard drive, enabled RPI-RF-MOD. Then added the raspberrymatic addon and restored Raspberrymatic. It restored fine but then got stuck afterwards, so I ignored that, installed the Advanced SSH console via the Store (need to remember setting a password afterwards), launched it and followed your instructions. All went absoluteley fine and I now have a steady connection between the HA Raspberrymatic Addon and my HmIP-DRAP. Let's hope the HA folks are able to pick this up much quicker now. Thanks again for working on it!

dersch81 commented 1 year ago

I followed your patch instructions with macvlan and the HAP directly connected to the RaspberryMatic AddOn. Many thanks. Looking forward to have some persistent patch in the future.

Thorsten1982 commented 1 year ago

Hello, we are currently building our family house and we will use Homematic IP Wired here. I currently own a CCU2 and use Home Assistant on a Rasperry 3. I have now read that there is the Raspberrymagic OS on which I can install the Homematic addon. Then I heard that there are currently problems with the DRAP. Now there seems to be a first solution. Now my question is to assume that I can use it in 2 to 3 months without any problems? Since I want to set up everything from scratch for the new house, I could switch directly to Raspberrymagic.

dersch81 commented 1 year ago

Is it fixed? I have upgraded RasperryMatic to 3.69.7.20230626 and HASS to 2023.6.3 and the HmIP-HAP is reachable without repeating the MAC VLAN Hotfix.

jens-maus commented 1 year ago

Is it fixed?

No, the macvlan solution is still pending / not implemented in HA. Thus, as long as this ticket/issue is open the workaround patch is still required to be executed each time the addon or HA is going to be restarted...

dersch81 commented 1 year ago

Yes, it just was reachable for some minutes. I had to use the hotfix again

Phil7989 commented 1 year ago

FYI: After some more work and tests with HomeAssistant I was able to create a manual patch procedure to create the necessary macvlan docker bridge also within HomeAssistant and assign the RaspberryMatic add-on to use this macvlan bridge, thus being directly connected to the respective LAN subnet. See here for the respective documentation on how to apply this patch procedure on your HomeAssistant / RaspberryMatic add-on installation while the HA developers are still working on directly providing this functionality for add-ons like ours here:

https://github.com/jens-maus/RaspberryMatic/wiki/Installation-HomeAssistant#hmip-haphmipw-drap-support-patch

After applying this patch procedure to a running RaspberryMatic add-on it should be indeed reachable via the assigned ip address, thus communication with a local HmIP-HAP or HmIPW-DRAP in the same LAN segment should then work as expected. Note, however, that there still exists a quite strong limitation for this patch procedure, namely the point that upon reboot of HomeAssistant or even restart of the RaspberryMatic add-on one has to manually re-apply this patch procedure.

Nevertheless, if there are some HA/RaspberryMatic users out there waiting for HmIP-HAP/HmIPW-DRAP support, feel free to test this manual patch procedure and please report back if this solves your issues until the HA developers are directly integrating a more convenient solution in a future version.

Hi all, i have no idea about all this linux stuff. I have execute the support patch and get the error message „RaspberryMatic isn't running or hostname incorrect.„ Do you know this issue? IP ans Host are correct. EDA4F977-80B7-4D83-B3BA-F956AC14FA60

jens-maus commented 1 year ago

Hi all, i have no idea about all this linux stuff.

Don't you think that it would then be better to simply wait until a permanent solution is available? Because you will need to perform these step each time you restart RaspberryMatic or your Home Assistant host. And also note that this here is no support fora, but a bug/issue tracker!

I have execute the support patch and get the error message „RaspberryMatic isn't running or hostname incorrect.„ Do you know this issue? IP ans Host are correct.

No the hostname you have selected does not seem to be correct (see the e.g. XXXXXXX-raspberrymatic pattern), see the add-on page and use the correct hostname, then it should work.

Torben0209 commented 1 year ago

Hallo Jens, wie geht es bzgl. der permanenten Implementierung des Patches in das Add-On weiter? Hab keine Lust mehr nach jedem Neustart den Patch auszuführen :-) Beste Grüße und vielen Dank für die tolle Arbeit!

jens-maus commented 1 year ago

Hallo Jens, wie geht es bzgl. der permanenten Implementierung des Patches in das Add-On weiter? Hab keine Lust mehr nach jedem Neustart den Patch auszuführen :-) Beste Grüße und vielen Dank für die tolle Arbeit!

Zu allererst: Wenn hier primär englisch gesprochen wird bleiben wir auch bei Englisch, bitte.

Und was den Patch angeht, nun dann wende dich an die HA Entwickler. Von RaspberryMatic Seite aus gibt es hier nichts weiter zu tun bzw. kann ich nichts weiter tun. Die HA Entwickler sind gefragt die notwendige Infrastruktur in deren Addon Schnittstellen zur Verfügung zu stellen damit ein Addon eine eigene separate IP-Adresse erhalten kann und das addon eben mit mac-vlan funktionalität gestartet werden kann.

Und zu guter letzt: Das ist kein Diskussionsforum hier sondern ein Bug-Issue Tracker. Wenn du nur nachfragen willst wann es hier weitergeht bzw. wie der stand ist, dann nutze den Diskussionsteil bzw. ein Diskussionsforum.

ThomPre99 commented 1 year ago

Hi Jens,

first of all: great work you're doing!

Is it possible to start the "patch script" in the POST section of the add-on (if configured) and to control the script using configuration parameters? This means that the Raspberrymatic add-on including HmIP-HAP support would be independent of HA development (although an HA solution would of course be better). It is logical that the start time of the addon is extended if the script is activated.

Unfortunately, my docker knowledge is not so good that I could create a solution myself.

Thanks in advance and best regards Thomas

jens-maus commented 1 year ago

Is it possible to start the "patch script" in the POST section of the add-on (if configured) and to control the script using configuration parameters? This means that the Raspberrymatic add-on including HmIP-HAP support would be independent of HA development (although an HA solution would of course be better).

No, this is unfortunately not possible, simply because HomeAssistant does not allow to specify such PRE or POST scripts to be defined by the addon which are then executed upon start/stop in the context of the HomeAssistant host OS. Thus, a HA addon can only modify or affect things within its own limited context (within the docker container) and not outside of that. And that's simply because of security reasons.

So, I am sorry, but the only way to get the necessary macvlan based networking running for being able to communicate with a HmIPW-DRAP or HmIP-HAP from a running RaspberryMatic addon is by either manually running my mentioned patch in the context of the HomeAssistant host each time your RaspberryMatic HA addon is (re-)started or simply wait for the HA developers (@pvizeli, @agners) to actually implement the necessary changes into the HA addon infrastructure so that HA addons can employ such macvlan docker network setups themselves.

caldriks commented 1 year ago

Hello jens-maus,

thank you for your great work at first! Im currently getting in touch with ur Software and running into several Problems, especially with the HmIP-HAP Integration. Do you recommend to setup a standalone raspberrymatic instead of the addon installation in this case?

again, thank you for your great work!

dn-gt commented 1 year ago

Hello,

first of all thanks for all your efforts and for providing the patch!

https://github.com/jens-maus/RaspberryMatic/wiki/Installation-HomeAssistant#hmip-haphmipw-drap-support-patch

Since my Home Assistant and Homematic Installation does not have Internet yet I downloaded the Shell Script and put it to the Config Directory of my Home Assistant Installation. From there I'm running the script using config cat patch.sh | bash

Unfortunately, the script does not find the network parameters itself so I provide them manually and but I can't reach Raspberrymatic under the new static IP Address 192.168.0.3 (also no ping possible). Any idea what's going wrong here?

cat patch.sh | bash ip: can't find device '' HomeAssistant Main Gateway (e.g. 192.168.178.1): 192.168.0.2 RaspberryMatic Add-on Hostname (e.g. 5422eb72-raspberrymatic): de838cd8-raspberrymatic RaspberryMatic Add-on IP (e.g. 192.168.178.4): 192.168.0.3 Creating docker macvlan network every ip-range or gateway must have a corresponding subnet Connecting add-on to macvlan network Error response from daemon: network ccu not found Stopping add-on (addon_de838cd8_raspberrymatic) addon_de838cd8_raspberrymatic Starting add-on (addon_de838cd8_raspberrymatic) addon_de838cd8_raspberrymatic

My HomeAssistant installation is available under 192.168.0.2 - I assume this is the IP I need to provide as "HomeAssistant Main Gateway", correct? Or do I need to supply the Router Gateway which would be 192.168.0.1? I tried this as well but without any effect. Should I worry about the highlighted error message?

Any hints are appreciated. HomeAssistant is running on ODroid Hardware with the latest October Release.

ivo7B4 commented 1 year ago

Hello dn-gt I have user knowledge for homeassistant and the integration only, but to support that jens-maus and other developers here can focus on their great work (also thanks a lot!) and as I think I can answer this question, my comment: Do I see correctly that you have homeassistant and raspberrymatic running on separate devices? If yes, you don't need this add-on; as you have raspberrymatic already running on another device. What you need is the custom homematic integration (which you actually is needed in both cases to integrate CCU3 or raspberrymatic; independent wheter you are running it separately or on the same device). And the custom_homematic integration is a great integration! With having it running on two devices (and you thus don't need this add-on), you also don't need to worry about this patch as the DRAP/HAP are reacheable from the CCU3 or raspberrymatic.

And a P.S.: I also propose to use the support forums as also jens-maus has mentioned above for such topics (Diskussionsforen).

dn-gt commented 1 year ago

... Do I see correctly that you have homeassistant and raspberrymatic running on separate devices? If yes, you don't need this add-on; as you have raspberrymatic already running on another device. What you need is the custom homematic integration (which you actually is needed in both cases to integrate CCU3 or raspberrymatic; independent wheter you are running it separately or on the same device). And the custom_homematic integration is a great integration! With having it running on two devices (and you thus don't need this add-on), you also don't need to worry about this patch as the DRAP/HAP are reacheable from the CCU3 or raspberrymatic.

And a P.S.: I also propose to use the support forums as also jens-maus has mentioned above for such topics (Diskussionsforen).

Thanks for your feedback, I'm using the addon version of Raspberrymatic however and not the integration. I thought it's better and saves additional hardware (the Odroid N2+ is powerful enough). I'll try to find more adequate thread in General Discussion secition. Thanks for the hint.

ivo7B4 commented 1 year ago

I assumed that you have it on a separate device as you've mentioned two different IP addresses. I use it on two different devices; thus I don't use the add-on, only the integration. But I still would expect the same IP address. Add-ons in home assistant you reach typically trough "settings, add-ons", then selecting the corresponding add-on and clicking on "open web-ui" afterwards. Alternatively, you can also actived "show in sidebar". I hope this helps.

Thorsten1982 commented 1 year ago

Hello, I was able to successfully apply the patch and learn the DRAP, DRBL and DSR. Am I correct that I now always have to call Raspberymatic using the static IP. Or can you still access the Raspberrymatic web interface via the home assistant. Does calling the static IP address also work from outside? Also, for example, via VPN or Myfritz, with the background that I use German fiber optics and therefore I don't have an IPv4 address but have to use it via IPv6.

Homeassistant was assigned an Ipv6 address by the Fritzbox but the static IP address of Rapsberymatic is only an Ipv4 address. The point is that the equipment could then be set up at work and not have to be on site. Because it is a new building.

Thorsten1982 commented 1 year ago

Very well done, Jens. I Backed up my Raspberrymatic environment and installed HomeAssistant on a fresh hard drive, enabled RPI-RF-MOD. Then added the raspberrymatic addon and restored Raspberrymatic. It restored fine but then got stuck afterwards, so I ignored that, installed the Advanced SSH console via the Store (need to remember setting a password afterwards), launched it and followed your instructions. All went absoluteley fine and I now have a steady connection between the HA Raspberrymatic Addon and my HmIP-DRAP. Let's hope the HA folks are able to pick this up much quicker now. Thanks again for working on it!

Hello, you say it works for you without any problems. Do you also have a wired 6-fold button in operation? For me, 3 different buttons are not trained, the Falmot c12 could be trained.