Closed edenhaus closed 3 years ago
Good news I was able to connect successfully with the app v2 to bumper with mitmproxy in between. I was able to run some basic commands (auto clean, locate,...) but until bumper supports the whole app it will take some time. Still some endpoints doesn't exist.
As the new version expects the MQTT server also on port 443, I have placed a nginx in the front. See #103 and #102 Here the relevant part of the nginx config (Thanks @puddly):
stream {
map_hash_bucket_size 64;
map $ssl_preread_server_name $internalport {
# redirect all requests, which contain "mq" in the SNI -> MQTT
~^.*(mq).*\.eco(vacs|user)\.(net|com)$ 8883;
# the rest of eco(user|vacs) requests
~^.*\.eco(vacs|user)\.(net|com)$ 443;
# mapping default to MQTT as the bots are connecting directly to the ip without SNI
default 8883;
}
server {
listen 443;
ssl_preread on;
proxy_pass 172.19.200.10:$internalport;
}
server {
listen 5223;
proxy_pass 172.19.200.10:5223;
}
server {
listen 8007;
proxy_pass 172.19.200.10:8007;
}
server {
listen 8883;
proxy_pass 172.19.200.10:8883;
}
}
I will create a complete docker-compose example in the next days.
Ecovacs uses now a lot of sub sub domains and for them the current certs are not valid. We need to find a way to generate them dynamically or find out all used subdomains
Unfortunately I didn't get this working after following your instructions. But I have to admit, I'm not into docker at all. Previously I did setup bumper manually.
Now I always get this error, as far as I can tell, it may be related to the architecture? Is it correct, that the image was not build for ARM? I'm trying to use it on a raspi. So maybe that's the problem?!
bumper_1 | standard_init_linux.go:219: exec user process caused: exec format error bumper_bumper_1 exited with code 1
I also had to change the nginx path to get rid of the following error.
nginx_1 | 2037/04/23 10:58:16 [emerg] 1#1: open() "/etc/nginx/nginx.conf" failed (2: No such file or directory) nginx_1 | nginx: [emerg] open() "/etc/nginx/nginx.conf" failed (2: No such file or directory)
from
- ./nginx/:/etc/nginx:ro # See config file below
to
- /etc/nginx/:/etc/nginx:ro # See config file below
Don't quite understand this either, as I would have expect it to use exactly the same path for inside the container, no matter how my system is set up in the end... Isn't this the idea of docker?!
However, my "current issue" is the first error.
Unfortunately I didn't get this working after following your instructions. But I have to admit, I'm not into docker at all. Previously I did setup bumper manually.
Now I always get this error, as far as I can tell, it may be related to the architecture? Is it correct, that the image was not build for ARM? I'm trying to use it on a raspi. So maybe that's the problem?!
Please get familiar with docker and docker compose. You need to mount the folder containing the certs to the container. I also have provided only a image for x64, but you can create a image for the raspi yourself with the source code or run it manually without docker
When you certs are found, bumper will call a go program to generate it. As I‘m not familiar with go, I can‘t change the go code. Also a bug there exists #117. I only tested the manual way, which is described in the docs.
Don't quite understand this either, as I would have expect it to use exactly the same path for inside the container, no matter how my system is set up in the end... Isn't this the idea of docker?!
For your second issue I will update the docs above, but you should get familiar with docker first. In the folder, where you have created the docker-compose.yaml, create a new folder named „nginx“ and within create a file nginx.conf and place there the whole Nginx configuration from above.
You don‘t docker to run bumper. You can still download the code of this pr and run it manually. Be aware that you need probably configure nginx manually as at least my deebot 950 is connecting on port 443 instead of 8883
Is the edenhaus/bumper
image rpi compatible? The container instantly dies with standard_init_linux.go:219: exec user process caused: exec format error
.
I guess the go code was not compiled for ARM.
Is the
edenhaus/bumper
image rpi compatible? The container instantly dies withstandard_init_linux.go:219: exec user process caused: exec format error
.I guess the go code was not compiled for ARM.
I have tested it only on my linux x64 server.
I have no idea about the go program... But if you create the certs manually and map it as volume inside the container the go program will not be executed. I would recommend it to create it manually as a bug in the go program exists #117
Does not seem to be an issue with the go code in bumper but rather docker itself. I tried to build the image on my rpi4 and the build failed with the same error:
pi@rpi4:~/smarthome/bumper-dev $ docker build -t foobar/bumper .
Sending build context to Docker daemon 41.99MB
Step 1/16 : ARG FROM_ARCH=amd64
Step 2/16 : FROM alpine as builder
---> b6342193e7c0
Step 3/16 : ADD https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz .
Downloading [==================================================>] 1.719MB/1.719MB
---> Using cache
---> 4b90884cb9bd
Step 4/16 : RUN tar zxvf qemu-3.0.0+resin-arm.tar.gz --strip-components 1
---> Using cache
---> 8552add857b4
Step 5/16 : ADD https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-aarch64.tar.gz .
Downloading [==================================================>] 1.857MB/1.857MB
---> Using cache
---> 56fb8e80c4f2
Step 6/16 : RUN tar zxvf qemu-3.0.0+resin-aarch64.tar.gz --strip-components 1
---> Using cache
---> e0a169240d89
Step 7/16 : FROM $FROM_ARCH/python:3.7-alpine as base
---> ec8ed031b5be
Step 8/16 : COPY --from=builder qemu-arm-static /usr/bin
---> Using cache
---> 1ce35e30773f
Step 9/16 : COPY --from=builder qemu-aarch64-static /usr/bin
---> Using cache
---> b9d7e4a15f11
Step 10/16 : FROM base as builderfinal
---> b9d7e4a15f11
Step 11/16 : RUN apk add build-base
---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
---> Running in a64e938c918e
standard_init_linux.go:219: exec user process caused: exec format error
The command '/bin/sh -c apk add build-base' returned a non-zero code: 1
I tried different FROM_ARCH values and --platform options but I cannot build the image. Ill try again tomorrow on my laptop.
Does not seem to be an issue with the go code in bumper but rather docker itself. I tried to build the image on my rpi4 and the build failed with the same error:
pi@rpi4:~/smarthome/bumper-dev $ docker build -t foobar/bumper . Sending build context to Docker daemon 41.99MB Step 1/16 : ARG FROM_ARCH=amd64 Step 2/16 : FROM alpine as builder ---> b6342193e7c0 Step 3/16 : ADD https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz . Downloading [==================================================>] 1.719MB/1.719MB ---> Using cache ---> 4b90884cb9bd Step 4/16 : RUN tar zxvf qemu-3.0.0+resin-arm.tar.gz --strip-components 1 ---> Using cache ---> 8552add857b4 Step 5/16 : ADD https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-aarch64.tar.gz . Downloading [==================================================>] 1.857MB/1.857MB ---> Using cache ---> 56fb8e80c4f2 Step 6/16 : RUN tar zxvf qemu-3.0.0+resin-aarch64.tar.gz --strip-components 1 ---> Using cache ---> e0a169240d89 Step 7/16 : FROM $FROM_ARCH/python:3.7-alpine as base ---> ec8ed031b5be Step 8/16 : COPY --from=builder qemu-arm-static /usr/bin ---> Using cache ---> 1ce35e30773f Step 9/16 : COPY --from=builder qemu-aarch64-static /usr/bin ---> Using cache ---> b9d7e4a15f11 Step 10/16 : FROM base as builderfinal ---> b9d7e4a15f11 Step 11/16 : RUN apk add build-base ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested ---> Running in a64e938c918e standard_init_linux.go:219: exec user process caused: exec format error The command '/bin/sh -c apk add build-base' returned a non-zero code: 1
I tried different FROM_ARCH values and --platform options but I cannot build the image. Ill try again tomorrow on my laptop.
Please give a look at https://github.com/bmartin5692/bumper/blob/9b1ae262a807ad9fe09b33841ba7054b23c91453/hooks/build
I used the hook scripts to push the image for the same arch as the official one. So probably you should be able to use it on the rpi
Thank you for the hint! I managed to get it running for the rpi4 by running this on my laptop:
docker build --build-arg FROM_ARCH=arm32v7 . --tag bumper-arm32
and the using docker save
and docker image load
to get the image onto the rpi.
However, after recreating the bumper.crt the app now shows an empty screen.
Do I need to update the CA cert as well? I had already reduced the validity before.
Do I need to update the CA cert as well? I had already reduced the validity before.
I don't think so but I have recreate it also as I was not sure what the go program does.
Also I have tested only the iOS App version. Maybe the android one uses different endpoints. As I don't have a rooted phone I couldn't test it
I checked the logs via adb
and the app throws RobotUI Error
s for the following endpoints:
If I try them out manually they all return 404. Are those different in the iOS app?
I checked the logs via
adb
and the app throwsRobotUI Error
s for the following endpoints:* https://api-app.dc-eu.ww.ecouser.net/api/rapp/sds/user/data/map/get?key=EcovacsHome&subKeys=name * https://api-app.dc-eu.ww.ecouser.net/api/pim/product/software/config/batch * https://api-app.dc-eu.ww.ecouser.net/api/ecms/app/ad/res?locations=robot_device_list_first
If I try them out manually they all return 404. Are those different in the iOS app?
The endpoints exists and return also some data when called correctly. Be aware that you need to call them correctly.
For example: https://api-app.dc-eu.ww.ecouser.net/api/pim/product/software/config/batch
must be called as post request with some pid
in the body.
A GET request on https://api-app.dc-eu.ww.ecouser.net/api/rapp/sds/user/data/map/get?key=EcovacsHome&subKeys=name returns
{"code": 0, "data": {"data": {"name": "My Home"}, "tag": null}, "message": "success"}
Probably you don't have setup bumper correctly or you don't use the version with the code of this PR....
Probably you don't have setup bumper correctly or you don't use the version with the code of this PR....
Just had the same realization and noticed that I build the main branch instead of the PR branch of your repo. Sorry for wasting your time on this :see_no_evil:
Also the manual build was not necessary. I just had to run docker pull with the correct platform on the rpi: docker pull edenhaus/bumper --platform linux/arm/v7
. I naively assumed that docker is able to figure out which platform it is running on...
~Now the Deebot is shown in the app but marked as offline. Bumper status shows that the client did not connect to MQTT, but I dont see any errors~ After restarting the app a couple times I just had my first successful cleaning run :tada:
After pulling the docker image like @Bustel suggested, and after fixing https://github.com/linuxserver/docker-papermerge/issues/4#issuecomment-735236388, I finally got the docker containers running on my rpi using docker-compose up
.
Unfortunately the services are not reachable on their ports (either not running, or not exposed, or otherwise "blocked"?).
While docker-compose ps
shows the containers running and the ports:
docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------------------------------
bumper_bumper_1 python3 -m bumper Up
bumper_nginx_1 /docker-entrypoint.sh ngin ... Up 443/tcp, 5223/tcp, 80/tcp, 8007/tcp, 8883/tcp
They won't appear in netstat
, e.g.:
sudo netstat -ntl | grep "8007"
sudo netstat -ntl | grep "443"
And of course everything else cannot reach them as well (e.g. curl):
curl -X GET https://api-app.dc-eu.ww.ecouser.net/api/rapp/sds/user/data/map/get?key=EcovacsHome&subKeys=name
curl: (7) Failed to connect to api-app.dc-eu.ww.ecouser.net port 443: Connection refused
The domain is resolved correctly by pi-hole:
host api-app.dc-eu.ww.ecouser.net
api-app.dc-eu.ww.ecouser.net has address 192.168.178.3
Stopping docker and starting up the old "bumper.service" works just fine.
sudo netstat -ntl | grep 8007
tcp 0 0 192.168.178.3:8007 0.0.0.0:* LISTEN
Note: I don't want to keep using the "old service" approach, as I would have to setup nginx etc. manually and actually therefore prefer the docker-way.
Since docker seems to add a lot of iptables
rules to the system and therefore broke my piVCCU installation, I had to modify the rules like here: https://github.com/alexreinert/piVCCU/issues/264#issuecomment-753907782
But I already reverted all those changes and still none of the ports is reachable for me :/
Did anyone else face a similar issue? Or does anyone have an idea what might go wrong here? Any help is highly appreciated! Thx in advance.
@d4rkd3v1l I have updated the docker-compose example. The problem was that I only exposed the ports in my example as I use in my setup macvlan.
A working example can be found here
Thx @edenhaus
I got it working now as well, after using ports
instead of expose
. Also be aware that the example docker-compose.yaml
uses the original docker image from bmartin5692, and not the one you provided on docker hub, so this must also be changed to edenhaus/bumper
in order to make it work (for the moment, until everything got merged, I guess).
@d4rkd3v1l I know that, put I don't want to change it as my images are only test images, which can be updated and also be broken anytime
@bmartin5692 Have you time for a review? I think some people are waiting for this PR
I'll try to get this tested and reviewed this weekend.
@edenhaus system: rpi 4 @arm32 with deebot 901 and adguard for dns rewrites
I had to add 'privileged: true' to start bumper from composer. The iobroker plugin worked and i see the login in the logs. But the IOS app didnt not work, timeout -> no login attempt in the logs.
Do u have any suggestions?
I use the docker-compose setup described in https://github.com/bmartin5692/bumper/tree/master/example/docker-compose
I have no idea what is wrong with your setup, but it can be a lot.... Wrong certs, wrong dns config, endpoints not reachable,....
Also I don't have your model and as I can remember it correctly your robot uses the protocol MQTT/xml
and my 950 uses MQTT/json
. So it can also be that there are missing endpoints...
Also you need to trust the certificate on your apple device.
As you can see there can be a lot and sorry but I currently don't have time to give you support.
Adding the new endpoints for app version 2 Fixes https://github.com/bmartin5692/bumper/issues/114 Fixes https://github.com/bmartin5692/bumper/issues/107
Feel free to test it :). For testing purpose I have also created a new docker image
edenhaus/bumper
, which will be always up to date with this PR.Testing instructions
The easiest ways to test is using the docker-compose example below. Please adjust it for your needs **Note:** You need to recreate the certs as some new sub domains have been added. I have only tested the manual ways (see updated docs) **Currently only domains/urls for europe are included. Please comment if your app uses different ones** A docker-compose example can be found in the folder `example` The docker-compose example has an nginx proxy, which redirects mqtt traffic on port `443` to port `8883` The redirection is required as the app v2+ and robots with a newer firmware are connecting to the mqtt server on port 433.