[Bug]: doc up ingester fails with "Please make sure the libxml2 and libxslt development packages are installed."

[bkallw]

What happened?

doc up ingester fails with "Please make sure the libxml2 and libxslt development packages are installed."

What am I doing wrong?

Thx! Bernd

Environment

Linux pi4 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

Relevant log output

pi@pi4:/opt/containers/apple-health $ doc up ingester
Creating network "apple-health_default" with the default driver
Building ingester
Step 1/5 : FROM python:3.11-alpine
 ---> c33d907e4a1d
Step 2/5 : COPY requirements.txt .
 ---> Using cache
 ---> ea9eb21f3e79
Step 3/5 : RUN pip3 install -r requirements.txt
 ---> Running in 3888ff75b972
Collecting influxdb==5.3.1 (from -r requirements.txt (line 1))
  Downloading influxdb-5.3.1-py2.py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 2.5 MB/s eta 0:00:00
Collecting gpxpy==1.5.0 (from -r requirements.txt (line 2))
  Downloading gpxpy-1.5.0.tar.gz (111 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.6/111.6 kB 5.9 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting lxml==4.9.3 (from -r requirements.txt (line 3))
  Downloading lxml-4.9.3.tar.gz (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 6.4 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [3 lines of output]
      Building lxml version 4.9.3.
      Building without Cython.
      Error: Please make sure the libxml2 and libxslt development packages are installed.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
ERROR: Service 'ingester' failed to build: The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1

docker-compose.yml

pi@pi4:/opt/containers/apple-health $ more docker-compose.yml
version: "3"
services:
  grafana:
    image: grafana/grafana:10.0.3
    ports:
    - 3000:3000
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=health
      - GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/etc/grafana/provisioning/dashboards/dashs/sp
ecific-metrics.json
    volumes:
    - ./provisioning:/etc/grafana/provisioning
  influx:
    image: influxdb:1.8
    restart: always
    environment:
      - INFLUXDB_DB=health
    volumes:
    - influx-data:/var/lib/influxdb
  ingester:
    image: k0rventen/apple-health-grafana-ingester
    build: ./ingester
    volumes:
    - ./health-export.zip:/export.zip

volumes:
  influx-data:

[k0rventen]

Hi ! Thx for giving this project a try ! You are not doing anything wrong, this is not expected behavior. You should not be building the ingester image locally, they are already built for amd64/arm64.

Something in the recent version of docker might have changed (same behavior as issue #21). I've removed the build instruction from the compose file for now (b74323edd81f000c42df95da919192c66cdf2616), please git pull and try again !

I'll keep the issue open, please report back on your progress !

[bkallw]

Hi and thanks for the quick reply!

I now receive an error: pi@pi4:/opt/containers/apple-health $ doc up ingester Pulling ingester (k0rventen/apple-health-grafana-ingester:)... latest: Pulling from k0rventen/apple-health-grafana-ingester ERROR: no matching manifest for linux/arm/v8 in the manifest list entries

Thanks! Bernd

[k0rventen]

Could you share more about your setup ? (Os, version, and the output of docker info)

If I test on my pi4 (debian 11, uname -a Linux carina 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux, which is the same as yours), it pulls the linux/arm64 version (this one).

I do not understand why your pi is trying to pull a linux/arm/v8 platform..

Could you try adding --platform=arm64 ? A potential explanation could be due to a mismatched 64b kernel / 32b userland/docker seen here

If that doesn't work, I'll see if I can build this specific v8 variant.

Regards,

[bkallw]

Sure.

Linux pi4 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

Client: Docker Engine - Community Version: 24.0.7 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.11.2 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.21.0 Path: /usr/libexec/docker/cli-plugins/docker-compose

Server: Containers: 7 Running: 7 Paused: 0 Stopped: 0 Images: 7 Server Version: 24.0.7 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523 runc version: v1.1.9-0-gccaecfc init version: de40ad0 Security Options: seccomp Profile: builtin cgroupns Kernel Version: 6.1.21-v8+ Operating System: Raspbian GNU/Linux 11 (bullseye) OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 1.805GiB Name: pi4 ID: 5WYG:HCW6:3DMD:D6RI:KXV6:QQSQ:YYTQ:FNVT:GRMW:XRVJ:SSN4:XRIA Docker Root Dir: /var/lib/docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

WARNING: No memory limit support WARNING: No swap limit support

---

I was able to pull your image using docker pull --platform=arm64.

When I try to start it, I end up with pi@pi4:/opt/containers/apple-health $ doc up ingester Creating apple-health_ingester_1 ... done Attaching to apple-health_ingester_1 apple-health_ingester_1 exited with code 159 pi@pi4:/opt/containers/apple-health $ doc ps Name Command State Ports

apple-health_grafana_1 /run.sh Up 0.0.0.0:3000->3000/tcp,:::3 000->3000/tcp apple-health_influx_1 /entrypoint.sh influxd Up 8086/tcp apple-health_ingester_1 /bin/sh -c python3 -u app.py Exit 159

[k0rventen]

Ok so the issue regarding the platform seems to come from somewhere else. I'll investigate this issue in //.

If possible, check if the container produced any logs (docker logs <ingester_id>).

Exit code 159 may come from the same 'issue' as previously, a 64b kernel running on a 32b userland. Could you run dpkg --print-architecture and paste back the output ? Did you recall installing the 32b version of raspberry pi OS ?

[bkallw]

There seem to be no logs available.

I used a fresh 64bit Bullseye when I build my Raspi. uname -m says 'aarch64', dpkg 'armhf'.

In docker version I can see Server: Docker Engine - Community Engine: Version: 24.0.7 API version: 1.43 (minimum version 1.12) Go version: go1.20.10 Git commit: 311b9ff Built: Thu Oct 26 09:08:26 2023 OS/Arch: linux/arm Experimental: false which may indicate a 32bit docker ('linux/arm'). (?)

[bkallw]

Apparently not only docker is 32bit (armhf), but all platform-dependent apt packages. Sorry, never really looked into this.

[k0rventen]

Yeah the container can't even start properly due to the platform/arch difference, between what the container was built for and what your docker engine can run. And i'm not sure building an image for linux/arm/v7l would fix the problem as your docker engine was requesting v8.

fwiw on my PI:

> dpkg --print-architecture
arm64

Yes linux/arm indicates an non 64b installs, which is most likely the root cause of the errors you are encountering.

I would suggest reinstalling, but more importantly I don't understand how a 64bit version of raspian would create such behavior. IIRC at the start of the armhf to arm64 transition there was a period of time where you could enable manually a 64bit kernel using arm_64bit=1 even on a 32bit version, but i can't remember if that was on debian 11 or 10.

[k0rventen]

Apparently not only docker is 32bit (armhf), but all platform-dependent apt packages. Sorry, never really looked into this.

no problem. And now that I've thought about it, I'm not quite sure if you can switch from 32 to 64 bit userland.. that would mean the only way out is effectively to reflash a newer, 64bit version on your pi..

[bkallw]

Hi!

I spent some more time on this. Using your Dockerfile I was able to succesfully build my first container, ingester; only needed to add g++ and the libs. Here is the Dockerfile working for me:

FROM python:3.11-alpine

RUN apk add g++ libxslt-dev libxml2-dev

COPY requirements.txt .
RUN pip3 install -r requirements.txt

COPY *.py ./
ENTRYPOINT python3 -u app.py

It took quite a while to build lxml ... there might be a better approach.

Thanks for your good work and kind support!

[k0rventen]

Great news ! Hopefully you'll be able to run the whole thing and import your data !

I remember having similar issues years ago when building docker container on a raspberry pi3. Alpine does provide prebuilt packages for popular python packages, so to skip the build phase, you could try using the py3-lxml package (from this package search).

I'll close the issue, again thanks for reporting the problem !