search

john30 / ebusd

daemon for communication with eBUS heating systems
GNU General Public License v3.0
561 stars 130 forks source link

ebusd --configpath does not accept https #649

Closed eh-gh closed 1 year ago

eh-gh commented 1 year ago

Description

--configpath does not allow https URLs as given in the documentation https://github.com/john30/ebusd/wiki/2.-Run

Actual behavior

root@raspberrypi:/home/pi/repos/ebusd# ebusd --configpath=https://cfg.ebusd.eu --scanconfig 2022-09-25 11:59:46.005 [main error] invalid configPath URL

but --configpath=http://cfg.ebusd.eu works

Expected behavior

https URL should work.

ebusd version

current source from git

ebusd arguments

ebusd --configpath=https://cfg.ebusd.eu --scanconfig

Operating system

other

CPU architecture

armv7l

Dockerized

No response

Hardware interface

adapter 2

Related integration

other

Logs

2022-09-25 11:59:46.005 [main error] invalid configPath URL

eh-gh commented 1 year ago

just saw in https://github.com/john30/ebusd/commit/c3935d494619203dac308a197e3554786098fa01

most probably it has connection to libseccomp library

i do no use docker, the current stable version

dpkg -l | grep seccomp

ii libseccomp2:armhf 2.5.1-1+rpi1+deb11u1 armhf high level interface to Linux seccomp filter

john30 commented 1 year ago

did you compile yourself or use a debian package?

john30 commented 1 year ago

you need to have the necessary libraries installed when building from source in order to have SSL support included. running "ebusd --help" reveals in the device section whether https is supported

I2e4per commented 1 year ago

Hi,

I ran in same troubles. I used the apt-get update function, therefore I assume it was the precompiled *.deb package of this repro. So I went back manually to 22.2. which is (I believe) I came from. But it' doesn't help.

So I assume it's due to the fact that my installation previous to the update had the necessary csv files locally, once downloaded, but now this also fails.

Next I'm trying to pull those csv's locally and point the config to them.

john30 commented 1 year ago

the provider has some DDoS issues since Friday, so maybe this is related to it. You may want to try again after restarting ebusd

I2e4per commented 1 year ago

That's of course bad, but I doubt this to be the reason. I could access everything yesterday and today several times. If this would be related, then I shouldn't ether be able to gather things. Hope the provider will manage this soon.

At the moment it seems to work again. But it's still curious.

If someone else will have the need to pull config files local (*.csv) make sure to clone them via git. I found it here around somewhere.

Furthermore I am seeing symptoms like in #414 and #320. I'm suffering under sudden bus signal losts and SYN Err entries in the ebusd logs. Furthermore some read commands from ioBroker are failing.

I didn't touch my hw setup so I did not expect to have an issue right here. Today I checked the bus with scope and I couldn't find something obvious. Seems all fine, was running for years. Will go in deeper next days. I got the 2.2 Adapter with the USB/Serial Interface connected to a RPI 2

Any idea where I should start looking into deeper ?

My plan for now is checking wiring and directly solder the Interface shield on the 2.2 Board. as the bus wires also. See if this will change things.

I2e4per commented 1 year ago

What a journey.

Ok, just to finish up things and hopefully of any use for others running in same troubles.

My Setup: I am using a RPI 2 with some Homeautomation py -scripts, Volkszaehler and also ebusd. The RPi get's the ebusd connection via the 2.2 Adapter over the CP210x UART. That was running nearly for one year without issues. RPI2 (running ebusd, serving Data to ioBroker adapter) -> connected via USB to CP210x UART -> connected to the EBUSD Adapter v2.2 -> Vaillant geoTherm+

What happened: I went through my IoT's and updated them as I do from time to time. Coming to my RPI2 (running ebusd) I just flat did a sudo apt-get update / upgrade on the Pi via ssh. So I did not touch anything physical at all. It all ran through without issues at all, but afterwards I faced problems in ioBroker concerning read statements on the ebusd. They basically all failed.

So I found that the ioBroker Adapter doesn't support the current ebusd version of 22.4. It just supports 22.3 yet. So I manually installed the 22.3 version on the RPI. But this didn't change anything beside satisfying the ioBroker adapter not yelling anymore for the 22.3 version.

So I started to look into the ebusd logs and it seems running. At this point I ignored bus error messages and arbitration lost notifications... Some data came through. I just concentrated on the fact, that the retrieving of the online config files failed.

This was were I started investigating. After some time I managed to get them locally but this did also not bring the big advantage. Then I came up that there might be issues with the bus communication, which prevents ebusd from recognizing the correct configs, and therefore doesn't load / use them. As result I had still many unknown device messages. ebusd couldn't translate them. Then I wondered if I do have an issue on the bus. As I went through my heater's Frontpanel Menu I discovered connection errors with the OMU device, which is the outside unit of my heater and this is communicating via ebus. That gives me headaches, because I also had the feeling that my heater isn't operating properly.

I disconnected the EBUS Adapter from the ebus and viewed the bus with my oscilloscope. -> Nothing obvious. Then i attached my Adapter back to the bus and also nothing changed until it tried to write to the bus. Every time the Adapterboard red (Tx) LED went on, I noticed some sort of BUS reset and error messages in the logs.

So i checket my Adapterboard with the scope. Couldn't find anything. All should be fine.

So what the heck is this ?! I started reading and find some hints from John30 according to timing problems.

My guess now was, that related to the update on the Pi I did, the timings of the serial interface are somehow affected. No clue how to sort this out. So I decided to set up a NodeMCU ESP 8266 which I had luckily lying around and connected it to the EBUS Adapterboard 2.2 Flashed latest firmware on it, got the config right, changed the ebus config on the RPI and bang -> It's almost working again.

For now I only see arbitration lost after some write attempts.

Conclusion: Update of the RPI2 caused timing problems on the ebus.

mwildbolz commented 1 year ago

Very interesting. When I read through your post, I find some similarities to my problem. I also did an update on my Debian based PC on Sep 05th and have trouble since this point in time - see also #717

But i wasn't able to find a package related to usb or serial or something like that. I use ebusd within a docker container. Do you have any idea, which package update is the problem? Maybe you could check this with your list (in my case, logfile is /var/log/dpkg.log)

I updated the following packages:

libpython2.7-minimal:amd64 | <none> | 2.7.18-8
python2.7-minimal:amd64 | <none> | 2.7.18-8
python2-minimal:amd64 | <none> | 2.7.18-3
libpython2.7-stdlib:amd64 | <none> | 2.7.18-8
python2.7:amd64 | <none> | 2.7.18-8
libpython2-stdlib:amd64 | <none> | 2.7.18-3
python2:amd64 | <none> | 2.7.18-3
python-is-python2:all | <none> | 2.7.18-9
libc-dev-bin:amd64 | <none> | 2.31-13+deb11u3
linux-libc-dev:amd64 | <none> | 5.10.127-2
libcrypt-dev:amd64 | <none> | 1:4.4.18-4
libtirpc-dev:amd64 | <none> | 1.3.1-1+deb11u1
libnsl-dev:amd64 | <none> | 1.3.0-2
libc6-dev:amd64 | <none> | 2.31-13+deb11u3
libisl23:amd64 | <none> | 0.23-1
libmpfr6:amd64 | <none> | 4.1.0-3
libmpc3:amd64 | <none> | 1.2.0-1
cpp-10:amd64 | <none> | 10.2.1-6
cpp:amd64 | <none> | 02.01.2001 04:10
libcc1-0:amd64 | <none> | 10.2.1-6
libgomp1:amd64 | <none> | 10.2.1-6
libitm1:amd64 | <none> | 10.2.1-6
libatomic1:amd64 | <none> | 10.2.1-6
libasan6:amd64 | <none> | 10.2.1-6
liblsan0:amd64 | <none> | 10.2.1-6
libtsan0:amd64 | <none> | 10.2.1-6
libubsan1:amd64 | <none> | 10.2.1-6
libquadmath0:amd64 | <none> | 10.2.1-6
libgcc-10-dev:amd64 | <none> | 10.2.1-6
gcc-10:amd64 | <none> | 10.2.1-6
gcc:amd64 | <none> | 02.01.2001 04:10
libstdc++-10-dev:amd64 | <none> | 10.2.1-6
g++-10:amd64 | <none> | 10.2.1-6
g++:amd64 | <none> | 02.01.2001 04:10
make:amd64 | <none> | 4.3-4.1
libdpkg-perl:all | <none> | 1.20.11
dpkg-dev:all | <none> | 1.20.11
build-essential:amd64 | <none> | 12.Sep
libfakeroot:amd64 | <none> | 1.25.3-1.1
fakeroot:amd64 | <none> | 1.25.3-1.1
fonts-dejavu-core:all | <none> | 2.37-2
fontconfig-config:all | <none> | 2.13.1-4.2
javascript-common:all | <none> | 11+nmu1
libalgorithm-diff-perl:all | <none> | 1.201-1
libalgorithm-diff-xs-perl:amd64 | <none> | 0.04-6+b1
libalgorithm-merge-perl:all | <none> | 0.08-3
libfontconfig1:amd64 | <none> | 2.13.1-4.2
libjpeg62-turbo:amd64 | <none> | 1:2.0.6-4
libdeflate0:amd64 | <none> | 01.07.2001
libjbig0:amd64 | <none> | 2.1-3.1+b2
libwebp6:amd64 | <none> | 0.6.1-2.1
libtiff5:amd64 | <none> | 4.2.0-1+deb11u1
libxpm4:amd64 | <none> | 05.12.2001 01:03
libgd3:amd64 | <none> | 2.3.0-2
libc-devtools:amd64 | <none> | 2.31-13+deb11u3
libexpat1-dev:amd64 | <none> | 2.2.10-2+deb11u3
libfile-fcntllock-perl:amd64 | <none> | 0.22-3+b7
libjs-jquery:all | <none> | 3.5.1+dfsg+~3.5.5-7
libjs-underscore:all | <none> | 1.9.1~dfsg-3
libjs-sphinxdoc:all | <none> | 3.4.3-2
libpython3.9-dev:amd64 | <none> | 3.9.2-1
libpython3-dev:amd64 | <none> | 3.9.2-3
manpages-dev:all | <none> | 05.10.2001
python-pip-whl:all | <none> | 20.3.4-4+deb11u1
zlib1g-dev:amd64 | <none> | 1:1.2.11.dfsg-2+deb11u1
python3.9-dev:amd64 | <none> | 3.9.2-1
python3-dev:amd64 | <none> | 3.9.2-3
python3-wheel:all | <none> | 0.34.2-1
python3-pip:all | <none> | 20.3.4-4+deb11u1
mwildbolz commented 1 year ago

Just did a test with an older RaspberryPi with an older version of Debian and ebusd - also no difference in behaviour. Therefore I'm again going into the direction of a faulty eBus-coupler. Looks like I have to get out my oscilloscope... :-(

I2e4per commented 1 year ago

Hi mwildbolz,

You nailed it down. Same question I had in mind, but didn't think of looking up the logs. Now I throwed your log and mine into a excel spreadsheet to show some similarities.

Here is the complete result:

user installed package
me avahi-daemon_0.6.32-2+deb9u1_armhf.deb ...
mwildbolz build-essential:amd64 | | 12.Sep
me cifs-utils (2:6.7-1+deb9u1) ...
mwildbolz cpp-10:amd64 | | 10.2.1-6
mwildbolz cpp:amd64 | | 02.01.2001 04:10
me dbus (1.10.32-0+deb9u1) ...
me dpkg_1.18.26_armhf.deb ...
me dpkg-dev_1.18.26_all.deb ...  
mwildbolz dpkg-dev:all | | 1.20.11  
me ebusd_22.4_armhf.deb ...  
mwildbolz fakeroot:amd64 | | 1.25.3-1.1  
mwildbolz fontconfig-config:all | | 2.13.1-4.2  
mwildbolz fonts-dejavu-core:all | | 2.37-2  
mwildbolz g++-10:amd64 | | 10.2.1-6  
mwildbolz g++:amd64 | | 02.01.2001 04:10  
mwildbolz gcc-10:amd64 | | 10.2.1-6  
mwildbolz gcc:amd64 | | 02.01.2001 04:10  
me golang-1.7_1.7.4-2+rpi1+deb9u5_all.deb ...  
me golang-1.7-doc_1.7.4-2+rpi1+deb9u5_all.deb ...  
me golang-1.7-go_1.7.4-2+rpi1+deb9u5_armhf.deb ...  
me golang-1.7-src_1.7.4-2+rpi1+deb9u5_armhf.deb ...  
me gzip_1.6-5+deb9u1_armhf.deb ...  
me initramfs-tools (0.130) ...  
me install-info (6.3.0.dfsg.1-1+b1) ...  
mwildbolz javascript-common:all | | 11+nmu1  
mwildbolz libalgorithm-diff-perl:all | | 1.201-1  
mwildbolz libalgorithm-diff-xs-perl:amd64 | | 0.04-6+b1  
mwildbolz libalgorithm-merge-perl:all | | 0.08-3  
me libarchive13_3.2.2-2+deb9u3_armhf.deb ...  
mwildbolz libasan6:amd64 | | 10.2.1-6  
mwildbolz libatomic1:amd64 | | 10.2.1-6  
me libavahi-common3_0.6.32-2+deb9u1_armhf.deb ...  
me libavahi-core7:armhf (0.6.32-2+deb9u1) ...  
me libc-bin (2.24-11+deb9u4) ... only apartly similar
mwildbolz libc-dev-bin:amd64 | | 2.31-13+deb11u3  
mwildbolz libc-devtools:amd64 | | 2.31-13+deb11u3  
mwildbolz libc6-dev:amd64 | | 2.31-13+deb11u3  
mwildbolz libcc1-0:amd64 | | 10.2.1-6  
mwildbolz libcrypt-dev:amd64 | | 1:4.4.18-4  
me libdbi-perl_1.636-1+deb9u2_armhf.deb ...  
mwildbolz libdeflate0:amd64 | | 01.07.2001  
me libdpkg-perl_1.18.26_all.deb ...  
mwildbolz libdpkg-perl:all | | 1.20.11  
mwildbolz libexpat1-dev:amd64 | | 2.2.10-2+deb11u3  
mwildbolz libfakeroot:amd64 | | 1.25.3-1.1  
mwildbolz libfile-fcntllock-perl:amd64 | | 0.22-3+b7  
mwildbolz libfontconfig1:amd64 | | 2.13.1-4.2  
mwildbolz libgcc-10-dev:amd64 | | 10.2.1-6  
mwildbolz libgd3:amd64 | | 2.3.0-2  
me libglib2.0-0_2.50.3-2+deb9u3_armhf.deb ...  
me libglib2.0-data_2.50.3-2+deb9u3_all.deb ...  
mwildbolz libgomp1:amd64 | | 10.2.1-6  
mwildbolz libisl23:amd64 | | 0.23-1  
mwildbolz libitm1:amd64 | | 10.2.1-6  
mwildbolz libjbig0:amd64 | | 2.1-3.1+b2  
mwildbolz libjpeg62-turbo:amd64 | | 1:2.0.6-4
me libjpeg62-turbo:armhf (1:1.5.1-2+deb9u2) ...  
mwildbolz libjs-jquery:all | | 3.5.1+dfsg+~3.5.5-7  
mwildbolz libjs-sphinxdoc:all | | 3.4.3-2  
mwildbolz libjs-underscore:all | | 1.9.1~dfsg-3  
mwildbolz liblsan0:amd64 | | 10.2.1-6  
me liblzma5_5.2.2-1.2+deb9u1_armhf.deb ...  
me libmariadbclient18_10.1.48-0+deb9u2_armhf.deb ...  
mwildbolz libmpc3:amd64 | | 1.2.0-1  
mwildbolz libmpfr6:amd64 | | 4.1.0-3  
mwildbolz libnsl-dev:amd64 | | 1.3.0-2  
me libopenjp2-7_2.1.2-1.1+deb9u7_armhf.deb ...  
me libpam-systemd_232-25+deb9u14_armhf.deb ...  
mwildbolz libpython2-stdlib:amd64 | | 2.7.18-3  
mwildbolz libpython2.7-minimal:amd64 | | 2.7.18-8  
mwildbolz libpython2.7-stdlib:amd64 | | 2.7.18-8  
mwildbolz libpython3-dev:amd64 | | 3.9.2-3  
mwildbolz libpython3.9-dev:amd64 | | 3.9.2-1  
mwildbolz libquadmath0:amd64 | | 10.2.1-6  
me libssl-dev:armhf (1.1.0l-1~deb9u6) ...  
me libssl-doc_1.1.0l-1~deb9u6_all.deb ...  
me libssl1.1:armhf (1.1.0l-1~deb9u6) ...  
mwildbolz libstdc++-10-dev:amd64 | | 10.2.1-6  
me libsystemd0_232-25+deb9u14_armhf.deb ...  
mwildbolz libtiff5:amd64 | | 4.2.0-1+deb11u1  
mwildbolz libtirpc-dev:amd64 | | 1.3.1-1+deb11u1  
mwildbolz libtsan0:amd64 | | 10.2.1-6  
mwildbolz libubsan1:amd64 | | 10.2.1-6  
me libudev1_232-25+deb9u14_armhf.deb ...
mwildbolz libwebp6:amd64 | | 0.6.1-2.1  
me libxml2_2.9.4+dfsg1-2.2+deb9u7_armhf.deb ...  
mwildbolz libxpm4:amd64 | | 05.12.2001 01:03  
mwildbolz linux-libc-dev:amd64 | | 5.10.127-2  
mwildbolz make:amd64 | | 4.3-4.1  
me man-db (2.7.6.1-2) ...  
mwildbolz manpages-dev:all | | 05.10.2001  
me mariadb-client-10.1_10.1.48-0+deb9u2_armhf.deb …  
me mariadb-client-core-10.1_10.1.48-0+deb9u2_armhf.deb ...  
me mariadb-common_10.1.48-0+deb9u2_all.deb ...  
me mariadb-server-10.1_10.1.48-0+deb9u2_armhf.deb ...  
me mariadb-server-core-10.1_10.1.48-0+deb9u2_armhf.deb ...  
me mime-support (3.60) ...  
me openssl_1.1.0l-1~deb9u6_armhf.deb ...  
me Prcifs-utils_2%3a6.7-1+deb9u1_armhf.deb ...  
me Preparlibjpeg62-turbo_1%3a1.5.1-2+deb9u2_armhf.deb ...  
me Preplibavahi-core7_0.6.32-2+deb9u1_armhf.deb ...  
me Preplibssl1.1_1.1.0l-1~deb9u6_armhf.deb ...  
me Prersyslog_8.24.0-1+deb9u3_armhf.deb ...  
me Pribavahi-common-data_0.6.32-2+deb9u1_armhf.deb ...  
me Prlibssl-dev_1.1.0l-1~deb9u6_armhf.deb ...  
mwildbolz python-is-python2:all | | 2.7.18-9  
mwildbolz python-pip-whl:all | | 20.3.4-4+deb11u1  
mwildbolz python2-minimal:amd64 | | 2.7.18-3  
mwildbolz python2:amd64 | | 2.7.18-3  
mwildbolz python2.7-minimal:amd64 | | 2.7.18-8  
mwildbolz python2.7:amd64 | | 2.7.18-8  
mwildbolz python3-dev:amd64 | | 3.9.2-3  
mwildbolz python3-pip:all | | 20.3.4-4+deb11u1  
mwildbolz python3-wheel:all | | 0.34.2-1  
mwildbolz python3.9-dev:amd64 | | 3.9.2-1  
me rsyslog (8.24.0-1+deb9u3) ...  
me systemd_232-25+deb9u14_armhf.deb ...
me systemd-sysv_232-25+deb9u14_armhf.deb ...  
me tzdata_2021a-0+deb9u4_all.deb ...  
me udev_232-25+deb9u14_armhf.deb ...  
me update-initramfs: deferring update (trigger activated)  
me vim-common_2%3a8.0.0197-4+deb9u7_all.deb ...  
me vim-tiny_2%3a8.0.0197-4+deb9u7_armhf.deb ...  
me xxd_2%3a8.0.0197-4+deb9u7_armhf.deb ...  
me xz-utils_5.2.2-1.2+deb9u1_armhf.deb ...  
mwildbolz zlib1g-dev:amd64 | | 1:1.2.11.dfsg-2+deb11u1  

What I found most interesting are following events:

user installed package remarks
me dpkg-dev_1.18.26_all.deb ...  
mwildbolz dpkg-dev:all | | 1.20.11  
me libc-bin (2.24-11+deb9u4) ... only apartly similar
mwildbolz libc-dev-bin:amd64 | | 2.31-13+deb11u3  
mwildbolz libjpeg62-turbo:amd64 | | 1:2.0.6-4 sems to be related to jpeg compression
me libjpeg62-turbo:armhf (1:1.5.1-2+deb9u2) ...  
me libudev1_232-25+deb9u14_armhf.deb ... maybe a rs232 device -> Serial interface ? ? ?
me systemd_232-25+deb9u14_armhf.deb ... System and Services Management
me systemd-sysv_232-25+deb9u14_armhf.deb ...  
me udev_232-25+deb9u14_armhf.deb ...  

I think that the libc and systemd 232 packages could be candidates for further examination. The libjpeg thing seems to be related to jpeg compression, therefore I would sort it out.

Regarding the systemd and libudev 232 things I could image that they are related to rs232 interface which is the serial port (GPIO's). If so, that would be of high interest but might also be only a coincidence in versioning leading the wrong way.

And finally libc as general management of the system and services.

So now we would need somebody who is more familiar with those packages to sort this out furthermore.

You also mentioned cabling. I also thought about this already. In my case I do have a cable run from my heater to my main power distribution panel were the circuit breakers are, as I had my installation there. The cable I used back in the days is a KNX non shielded one. I also considered a shielded cat cable but found this to be oversized at this time as we have rather slow datarates and high level differences on the ebus side. And of course this cable is only a few meters long but apparently next to power cables. And this worked for month/years solid.

Next point is, that I used this cable when I added the esp8266 to the ebus adapterboard, instead of having the pi directly via usb connected to it and the situation got instantly better. Remember, I used the ebus adapter board 2.2 with an UART connected via usb to the RPI and not the RPi Hat version.

Now I do have the setup directly next to the heater and I am not using the cable anymore. As the heater is at the next room the esp8266 enabled me now to use wifi. And it looks very stable. Suddenly I see some errors on the bus, but mainly it seems to be ok.

As I am interested in this I might get a pi3 the next days and check again against the initial setup. I will see.

As I furthermore think about it, I have to admit that it might be a combination of the update and the eventually software timing issue and the non optimal cable which then turned out this issue. But if so the Pi3 should bring more clarification. I would expect this symptoms to be gone because of the more power the pi brings in.

I2e4per commented 1 year ago

And then i found this.

[https://github.com/eBUS/ttyebus]

I did not use it. Maybe this is a try worth.

EDIT: I think I misread this the first time. This is only related to the UART hw portion of the Rasperry Pi itself (GPIO's)

mwildbolz commented 1 year ago

I did some further investigations with my oscilloscope. It turned out, that the ESERA coupler is not able to pull down the bus voltage low enough for correct transmission of the data. So, it is a hardware problem. Just installed a new adapter - works like a charm now.

So, I don't think that the mentioned software packages have any influence. Also, none of the packages have a hardware relationship. libudev1_232 has nothing todo with RS232 - 232 is a version number! The others also not - as you have mentioned too.

Regarding cabling: I did some tests - no difference in behaviour. Best thing in my opinion is to use a network cable, CAT6 or similar...

john30 commented 1 year ago

is the original subject (HTTPS) still an issue? if not, this can be closed I guess

eh-gh commented 1 year ago

the orig problem with the version: ebusd 22.4.v22.4 could not be reproduced anymore, so i close the issue.