Closed omo2002 closed 7 years ago
I could not use /sys directly since, at least with docker swarm, docker was mounting the read-only on top of the shared /sys volume. This is why instead I share the host /sys as /sys_org and /dev as /dev_org and then in entrypoint.sh I use the _org volumes.
Please check /dev/mmd_bidcos : it should point to /dev_org/ttyAMA0. The same way /dev/ccu2-ic200 should point to /sys_org/class/gpio/gpio18/value in a Rasberry PI (I used Raspberries before but I have moved to an Orange PI to have more RAM).
But I think I know where the problem is... Last year homematic added support for Homematic IP to the CCU2. This support requires a special serial device (see https://github.com/angelnu/docker-ccu2/issues/3). They made the new mode the default for rfd but they still support the legacy mode which is what I assume in the docker image. My rfd.conf is set up to use the legacy mode so rfd works for me but you got a default config file that uses the new modes which does not work.
Please edit /var/lib/docker/volumes/ccu2_data/data/etc/config/rfd.conf to look like
# This File was automatically generated
# TCP Port for XmlRpc connections
Listen Port = 2001
#Log Destination = Syslog
#Log Identifier = rfd
Log Level = 1
Persist Keys = 1
#PID File = /var/rfd.pid
#UDS File = /var/socket_rfd
Device Description Dir = /firmware/rftypes
Device Files Dir = /etc/config/rfd
Address File = /etc/config/ids
Key File = /etc/config/keys
Firmware Dir = /firmware
Replacemap File = /firmware/rftypes/replaceMap/rfReplaceMap.xml
Fire NACK Error Events = true
#Improved Coprocessor Initialization = true
[Interface 0]
Type = CCU2
Description = CCU2-Coprocessor
ComPortFile = /dev/mmd_bidcos
AccessFile = /dev/null
ResetFile = /dev/ccu2-ic200
The important lines are AccessFile and ResetFile (please notice the blank at the start of the line or rfd will change it on the next start!) and the commented out Improved Coprocessor Initialization
@Munsio Please notice that I do not longer edit rfd.conf in the entrypoint.sh. Instead I fix the template in the docker build (see Dockerfile) so that I never touch the used rfd.conf at runtime.
The problem is that if the rfd daemon does change the rfd.conf to use the new Homematic IP mode that does not work with a normal Linux kernel (needs device drivers). This is why I modify the template so that rfd would not change it. If you manually modify rfd.conf you need to make sure that you use the same trick to avoid undesired changed.
seems all right ... ls -al /dev/c lrwxrwxrwx 1 root root 32 Jun 11 19:40 /dev/ccu2-ic200 -> /sys_org/class/gpio/gpio18/value lrwxrwxrwx 1 root root 16 Jun 11 19:40 /dev/mmd_bidcos -> /dev_org/ttyAMA0
my rfd.conf also looks ok.
Flashed latest Raspian on the SD-Card and booted it. Installed latest docker Version 17.05-ce and docker-compose
changed the bluetooth and console command in /boot/*.txt files
reboot
Used following docker-compose file for container bootstrap:
version: "3"
services:
ccu2:
image: angelnu/ccu2
hostname: ccu2
ports:
- "2001:2001/tcp"
- "8080:80/tcp"
environment:
- "PERSISTENT_DIR=/mnt"
# - "CHECK_PERSISTENT_DIR=y"
volumes:
- /dev:/dev_org
- /sys:/sys_org
- ./data:/mnt
restart: always
content of rfd.conf
# TCP Port for XmlRpc connections
Listen Port = 2001
Log Destination = Syslog
Log Identifier = rfd
Log Level = 1
Persist Keys = 1
# PID File = /var/rfd.pid
# UDS File = /var/socket_rfd
Device Description Dir = /firmware/rftypes
Device Files Dir = /etc/config/rfd
Key File = /etc/config/keys
Address File = /etc/config/ids
Firmware Dir = /firmware
Replacemap File = /firmware/rftypes/replaceMap/rfReplaceMap.xml
Fire NACK Error Events = true
#Improved Coprocessor Initialization = true
[Interface 0]
Type = CCU2
ComPortFile = /dev/mmd_bidcos
AccessFile = /dev/null
ResetFile = /dev/ccu2-ic200
checked /dev in container: ccu2-ic200 -> /sys_org/class/gpio/gpio18/value mmd_bidcos -> /dev_org/ttyAMA0 -> was red
Found out that on my raspberryPi 3 there is no /dev/ttyAMA0 but an /dev/serial0 - trying it with adjusting those things
@omo2002 when you say that it is not stable: does it work but you get connection problems or it does not work at all? I would suggest taking a look at the rfd log. You should change the log level in rfd.conf to 5 and restart rfd to see more details. Other things to try:
@Munsio Did you check the "How to configure local antenna" ? If you are missing the /dev/ttyAMA0 is because the Raspi is using it as console (see bullets 1 and 2 in the instructions)
Sure sure, as written above i changed the bluetooth and console command in /boot/*.txt files But on an reboot it seems it has no effect. But i also tinkered a little bit with the new kernel for the homeIP thing - so im going to purge the whole sdcard again with an fresh raspbian and following the details from http://spellfoundry.com/2016/05/29/configuring-gpio-serial-port-raspbian-jessie-including-pi-3/ for the serial console. It is at least more verbose why and what to change. After that i will try it again with the latest docker image.
rfd starts and then dies after some seconds, here is some logging
Config file is /etc/config/rfd.conf
2017/06/12 20:34:58.997
After purging everything and setting it up again with the guide from the link above i finally got an /dev/ttyAMA0 but the hmserver itself throws following error:
Jun 13 20:38:02 de.eq3.cbcs.vertx.management.VertxManager WARN [vert.x-eventloop-thread-4] SYSTEM ADVICE: pre-conditions for deployment of LegacyInitializion still not met - check deployment configuration (still unfulfilled: [connector.open])
Jun 13 20:38:06 de.eq3.ccu.bidcos.dispatcher.BidCosRpcDispatcher ERROR [Timer-1] Could not register BidCos-RF_java
Jun 13 20:38:06 de.eq3.ccu.bidcos.dispatcher.BidCosRpcDispatcher ERROR [Timer-2] Could not register HmIP-RF_java
Jun 13 20:38:09 de.eq3.cbcs.vertx.management.VertxManager WARN [vert.x-eventloop-thread-0] SYSTEM ADVICE: long-lasting deployment, check deployment configuration - currently deployed 15 out of 17
Jun 13 20:38:09 de.eq3.cbcs.vertx.management.VertxManager WARN [vert.x-eventloop-thread-0] SYSTEM ADVICE: pre-conditions for deployment of LocalServerFirmwareUpdateInitialization still not met - check deployment configuration (still unfulfilled: [connector.open])
@omo2002 The log shows that rfd is not able to initialise the antenna. You could try to set up the log level to 5 and see if it gives you more info.
Did you try to manually change the value from /dev/ccu2-ic200 and see if that changes Ok? Did you also try to turn off the Raspi for some seconds and try again? This helped me in the past with the Raspis.
@Munsio Glad that you are getting further! Some errors are expected since only the rfd interface works (no wired, no HM-IP). Are ReGaHss and rfd up?
root@orangepi-secundary:~# docker top casa_ccu2.1.goja8fif3k9dnooxcj1iyjq5g
UID PID PPID C STIME TTY TIME CMD
root 2448 4758 0 20:39 ? 00:00:00 sleep 600
root 3949 5133 0 20:43 ? 00:00:00 sleep 60
root 4758 4741 0 13:45 ? 00:00:00 /bin/sh /entrypoint.sh
root 5019 4758 0 13:45 ? 00:00:00 /usr/sbin/crond
root 5023 4758 0 13:45 ? 00:00:01 /sbin/syslogd -m 0
root 5028 4758 0 13:45 ? 00:00:00 /lib/udev/udevd -d
root 5089 4758 0 13:45 ? 00:00:03 /usr/sbin/ifplugd -i eth0 -fI -u0 -d10
root 5133 4758 0 13:45 ? 00:00:02 /bin/sh /bin/setclock
root 5145 4758 0 13:45 ? 00:00:00 /bin/ssdpd
root 5164 4758 0 13:45 ? 00:01:21 /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf
root 5167 4758 0 13:45 ? 00:00:00 /usr/sbin/sshd
root 5256 4758 0 13:46 ? 00:00:04 /bin/multimacd -f /etc/config/multimacd.conf -l 5
root 5333 4758 0 13:46 ? 00:01:41 /bin/rfd -f /etc/config/rfd.conf -l 5
root 5464 4758 1 13:46 ? 00:05:33 java -Xmx64m -Dlog4j.configuration=file:///etc/config/log4j.xml -Dfile.encoding=ISO-8859-1 -jar /opt/HMServer/HMIPServer.jar /etc/crRFD.conf
root 23010 5028 0 20:00 ? 00:00:00 /lib/udev/udevd -d
root 23011 5028 0 20:00 ? 00:00:00 /lib/udev/udevd -d
root 23094 16875 15 14:44 ? 00:56:29 /bin/ReGaHss -f /etc/rega.conf -l 2
I can change the values of /dev/ccu2-ic200 - hope i have done it right ;)
/dev # cat ccu2-ic200
1
/dev # echo 0 > ccu2-ic200
/dev # cat ccu2-ic200
0
/dev # echo 1 > ccu2-ic200
/dev # cat ccu2-ic200
1
EDIT: here my top output
pi@raspberrypi:/opt/ccu2/data/etc/config $ docker top ccu2_ccu2_1
UID PID PPID C STIME TTY TIME CMD
root 4103 4050 0 20:39 ? 00:00:00 /bin/sh /entrypoint.sh
root 4457 4103 0 20:39 ? 00:00:00 /usr/sbin/crond
root 4461 4103 0 20:39 ? 00:00:00 /sbin/syslogd -m 0
root 4466 4103 0 20:39 ? 00:00:00 /lib/udev/udevd -d
root 4532 4103 0 20:39 ? 00:00:00 /usr/sbin/ifplugd -i eth0 -fI -u0 -d10
root 4564 4103 0 20:39 ? 00:00:00 /bin/ssdpd
root 4574 4103 0 20:39 ? 00:00:00 /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf
root 4607 4103 0 20:39 ? 00:00:00 /bin/multimacd -f /etc/config/multimacd.conf -l 5
root 4618 4103 0 20:39 ? 00:00:00 ntpclient -h 172.18.0.1 -l
root 4677 4103 4 20:40 ? 00:00:22 java -Xmx64m -Dlog4j.configuration=file:///etc/config/log4j.xml -Dfile.encoding=ISO-8859-1 -jar /opt/HMServer/HMIPServer.jar /etc/crRFD.conf
root 4687 4103 0 20:40 ? 00:00:00 /bin/ReGaHss -f /etc/rega.conf -l 2
root 4730 4103 0 20:40 ? 00:00:00 sleep 600
root 4837 4821 0 20:40 pts/1 00:00:00 bin/sh
i have no rfd running so i think i have the same problem as @omo2002 The log Destination on rfd.conf is Syslog but there is no syslog file under /var/log
@omo2002 The the reset line is working fine... Did you try to the power off/on cycle?
Turned it off for 1 minuted and then pluggedin again. Still no luck no rfd running when checking with top - same errors on hmipserver.log as above
EDIT: Found the log here is the output when i try to start the rfd service
Jun 13 19:11:54 ccu2 user.info rfd: BidCoS-Service started
Jun 13 19:11:54 ccu2 user.info rfd: XmlRpc Server is listening on TCP port 2001
Jun 13 19:11:54 ccu2 user.warn rfd: opendir(/etc/config/firmware//) failed
Jun 13 19:11:54 ccu2 user.debug rfd: CCU2CommController::init() - Legacy initialization.
Jun 13 19:11:59 ccu2 user.debug rfd: () CCU2CommController::startCoprocessorApp(): Trying to start coprocessor application
Jun 13 19:12:04 ccu2 user.crit rfd: CCU2CommController::init(): Init failed. Cannot start coprocessor application.
Jun 13 19:12:04 ccu2 user.err rfd: () CCU2CommController::setCSMACAEnabled(): Error disabling CSMA/CA.
Jun 13 19:12:04 ccu2 user.err rfd: No BidCoS-Interface available
Jun 13 19:12:04 ccu2 user.warn rfd: Error initializing interfaces
The only thing that could be is that i roasted the pcb with my non existent soldering skills ^^
EDIT:
Im trying a different approach - i backup my current sd card and flash raspberryMatic on it - check if the pcb is working with it and than i know for sure if i made a mistake during soldering or if we have an config error
Edit-Edit: Okay when i use raspberryMatic all the raspberry is starting as expected and the "teach-in devices" method doesn't throw any errors. I currently don't have any homematic device to test if the connection is successful. If all work as expected i can test this in 2 days an post the result.
I tried the same thing as Munsio. Same results. Manual access to ccu2-ic200 works. Increasing loglevel does not help obviously. Loglevel 5 prints less logging than loglevel 1 (crazy!) Regarding the poweron/off. I did this in the past, no success.
What is really strange: I have another image, which I built on my own some months ago using the Dockerfile from angelnu, but I modified the entrypoint.sh and I believe also the Dockerfile itself using the same firmware version. This image works, no resets, poweron/offs required.
here again is logging, with logvel 0!
/etc/init.d # rfd -l 0 -c
Config file is /etc/config/rfd.conf
2017/06/13 21:34:42.306
last comment for today on this (being off for 5 days on vaction now:-)) I checked the setup my working image. Differences are so far: the ccu2_ic200 points to /sys/class.... and not to /sys_org/class (this is ok) the mmd_bidcos device is not linked to /dev_org/ttyAMA0 but is just a device. the rfs.conf files are identical.
Maybe the error is in the device ttyAMA0. Is it busy by bluetooth? (my cmdline.txt is fine, remember my other ccu2 image is running on the same docker engine properly)
You use plain docker (no docker swarm), right? Could you try to start the container with --privileged ? Another option is --device=/dev/ttyAMA0:/dev_org/ttyAMA0:rwm
I am guessing that docker shares ttyS1 (orange PI) but not ttyAMA0 by default.
HEY MAN - YOU SOLVED IT the --privileged option made my raspi reboot !! But the devicemapping option --device=/dev/ttyAMA0:/dev_org/ttyAMA0:rwm solved it.
Thanks a lot for your patience and your support. I recommend to adjust the options string on your prebuilt image dockerhub site.
Okay looks like i need a second microSD card to try it out. If it work - next step homematicIP support? @omo2002 @angelnu
@omo2002 Glad that we found the problem! So you just need the device mapping option, right?
Actually I wonder why the device option is needed for raspis/plain docker and not for my setup with OrangePI/docker swarm... The device option should be needed in both cases.
My objective with the docker image is that it works in swam with any type of HW. This is why I moved the HW detection into the docker container itself instead of doing it at build time as before.
So let us try to start the container with multiple device mappings for all known HW serial ports. Could you please check to start your container with "--device=/dev/ttyAMA0:/dev_org/ttyAMA0:rwm --device=/dev/ttyS1:/dev_org/ttyS1:rwm" and let me know if 1) it works 2) ll /dev_org/ttyS1 ? Thanks
@Munsio The homematicIP requires a kernel module, not just the docker container so not a simple think to do as mentioned at #3 This requires several days of work to try so not the highest thing in my prios since 1) I do not have any HM IP device 2) I do not longer use raspis so I would need a different HW.
But if anyone feels like doing a PR I would take it :-)
@angelnu i know your statement about the homematicIP and i will definitly give it a try but all this kernel and gpio thing is new for me so it would be great to have someone with advices behind. It would also be giving a boost to your project if there is an testet hmip solution/guide imho.
@Munsio I am not a kernel expert either, this is why I am also hesitating to drive this. Maybe you could try the instructions in the forum I pointed to and eventually contact the person that shared them. Once you have rfd working in non-legacy mode I can integrate it in the docker image.
Btw: one option that would work with todays docker images is to use the HM LAN gateway. Would this be an option instead of the local antenna?
I also see that EQ-3 is releasing a lot of new HM IP devices. For the time being I do not see anything unique that is not available in HM but it is an indicator of where they are investing. At one point I expect them to stop selling old HM devices since they made very simple to consume. At this point better to have HM IP support available...
Okay already had the possibility to test both the wireless and hmIP Switch actuator with the raspberryMatic Image and they are working. Also got a second microSD card to swap with an Raspbian Image and trying in the next days the solution with the --device option.
Okay tried it today with the device option but it wont work:
Jun 15 07:00:41 ccu2 user.info rfd: BidCoS-Service started
Jun 15 07:00:41 ccu2 user.info rfd: XmlRpc Server is listening on TCP port 2001
Jun 15 07:00:41 ccu2 user.warn rfd: opendir(/etc/config/firmware//) failed
Jun 15 07:00:41 ccu2 user.debug rfd: CCU2CommController::init() - Legacy initialization.
Jun 15 07:00:46 ccu2 user.debug rfd: () CCU2CommController::startCoprocessorApp(): Trying to start coprocessor application
Jun 15 07:00:47 ccu2 user.debug rfd: CCU2CommController::startCoprocessorAppThreadFunction(): Send start command
Jun 15 07:00:47 ccu2 user.debug rfd: CCU2CommController::SendSystemCommdand()
Jun 15 07:00:47 ccu2 user.debug rfd: CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 00 03 18 0a
Jun 15 07:00:47 ccu2 user.debug rfd: RX SERIAL: fd 00 02 ff 00 02 00
Jun 15 07:00:47 ccu2 user.debug rfd: RX FRAME: ff 00
Jun 15 07:00:47 ccu2 user.debug rfd: CCU2CoprocessorCommand::CCU2CoprocessorCommand(): Message too small: 2 bytes.
Jun 15 07:00:47 ccu2 user.debug rfd: () CCU2CommController::handleIncomingSerialFrame(): Command not parseable.
Jun 15 07:00:48 ccu2 user.debug rfd: () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
Jun 15 07:00:50 ccu2 user.debug rfd: CCU2CommController::startCoprocessorAppThreadFunction(): Retrying to send start command
Jun 15 07:00:50 ccu2 user.debug rfd: CCU2CommController::startCoprocessorAppThreadFunction(): Send start command
Jun 15 07:00:50 ccu2 user.debug rfd: CCU2CommController::SendSystemCommdand()
Jun 15 07:00:50 ccu2 user.debug rfd: CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 01 03 9e 09
Jun 15 07:00:50 ccu2 user.debug rfd: RX SERIAL: fd 00 02 ff 01 82 05
Jun 15 07:00:50 ccu2 user.debug rfd: RX FRAME: ff 01
Jun 15 07:00:50 ccu2 user.debug rfd: CCU2CoprocessorCommand::CCU2CoprocessorCommand(): Message too small: 2 bytes.
Jun 15 07:00:50 ccu2 user.debug rfd: () CCU2CommController::handleIncomingSerialFrame(): Command not parseable.
Jun 15 07:00:51 ccu2 user.debug rfd: () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
Jun 15 07:00:51 ccu2 user.crit rfd: CCU2CommController::init(): Init failed. Cannot start coprocessor application.
Jun 15 07:00:51 ccu2 user.debug rfd: CCU2CommController::SendSystemCommdand()
Jun 15 07:00:51 ccu2 user.debug rfd: CCU2CommController::SendSystemCommdand() sending: fd 00 04 00 02 0a 00 3d 10
Jun 15 07:00:51 ccu2 user.debug rfd: RX SERIAL: fd 00 02 ff 02 82 0f
Jun 15 07:00:51 ccu2 user.debug rfd: RX FRAME: ff 02
Jun 15 07:00:51 ccu2 user.debug rfd: CCU2CoprocessorCommand::CCU2CoprocessorCommand(): Message too small: 2 bytes.
Jun 15 07:00:51 ccu2 user.debug rfd: () CCU2CommController::handleIncomingSerialFrame(): Command not parseable.
Jun 15 07:00:52 ccu2 user.debug rfd: () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
Jun 15 07:00:52 ccu2 user.err rfd: () CCU2CommController::setCSMACAEnabled(): Error disabling CSMA/CA.
Jun 15 07:00:52 ccu2 user.err rfd: No BidCoS-Interface available
Jun 15 07:00:52 ccu2 user.warn rfd: Error initializing interfaces
This is my actuall docker-compose file:
version: "3"
services:
ccu2:
image: angelnu/ccu2
hostname: ccu2
ports:
- "2001:2001/tcp"
- "8080:80/tcp"
devices:
- "/dev/ttyAMA0:/dev_org/ttyAMA0:rwm"
- "/dev/ttyS0:/dev_org/ttyS0:rwm"
environment:
- "PERSISTENT_DIR=/mnt"
# - "CHECK_PERSISTENT_DIR=y"
volumes:
- /dev:/dev_org
- /sys:/sys_org
- ./data:/mnt
restart: always
@omo2002 - it would be great if you can provide the command how you starting the docker-container maybe there is something wrong
Could it be that cause i used raspberyMattic that it has updated the firmware of the module and cause the firmware is now the dualcoprocessor one it won't work? Currently i have Version: 2.8.4
I found that the serial interface was not working for me with Docker swarm: I had not noticed because in addition to the local antenna I have a HM LAN gateway...
So if you need to use the local antenna then for the time being is not possible to use docker swarm directly. There is a circumvention here that I will try next. For the time being I would recommend to start the CCU docker image with docker run where the -device is supported.
If you use raspberyMattic as base OS then I would expect the HM IP devices to work on the docker image: you would then need to remove the blanks preceding the AccessFile and ResetFile lines and comment out the "Improved Coprocessor Initialization = true" line. Afterwards the rfd service will update the rfd.conf to use the new method that allows HM IP devices to work.
After we confirm that HM IP devices work when using raspberyMattic as host then we would know that the only missing part is the device driver. For raspis I could extend build.sh to build the kernel module so it works with other raspi images.
Another option worth mentioning is that raspberyMattic also supports running in LAN GW mode. So it would be possible to use the CCU docker image to access raspi powered LAN GWs. Since you can add multiple LAN GWs to a CCU docker image then if one raspi dies the CCU docker swarm would still have access to at least a LAN GW. I think I will try that with the raspis I have not used for a while :-)
@Munsio in your case it looks like you have access to the serial interface since you get some bytes back but they contain invalid messages. So either the reset did not work, the antenna is broken or the serial port is being used for other things. If you are using raspberyMattic you might have two instances of the rfd using the serial interface in parallel.
I see, we are discussing two things in one thread. Therefore I opened a new issue for the prebuilt image one, okay ? History can be seen at #7.
I used your latest prebuilt image and your run command: docker run --name ccu2 -d --restart=always -p 8080:80 -p 2001:2001 -e PERSISTENT_DIR=/mnt -v /dev:/dev_org -v /sys:/sys_org -v ccu2_data:/mnt --hostname ccu2 angelnu/ccu2
the volumes sys_org and dev_org are available inside the container. But the rfd does not run stable. I am using a raspberry pi 3 with the latest docker version.
Is the image really running in your environment ? in my eyes you should not mount the sys volume as /sys_org but as /sys, because access to GPIO port 18 does not work (read-only filesystem). Curious about your comment.