Error starting built image for arm64 bookworm - failed to process planet file

[jata1]

Hello. I am trying to build / start Overpass-api using docker compose on a rpi5 (arm64v8/bookworm) and have made good progress but now I am stuck...

I have edited the dockerfile to use arm64v8 and added python3-pip (as it was failing to build without it)- see below.

The image builds successfully but when I then use the build image to start the container I get the following error.

Any thoughts on why this is happening very much appreciated.

error on container up/start:

Reading XML file ... elapsed node 2426135824. Compute current ... ready. /app/bin/init_osm3s.sh: line 44:   132 Broken pipe             bunzip2 < $PLANET_FILE
133 Killed                  | $EXEC_DIR/bin/update_database --db-dir=$DB_DIR/ $META $COMPRESSION
Failed to process planet file

compose:

version: '3.3'

services:
  overpass:
    image: overpass-api:latest
    container_name: overpass
    hostname: overpass_db
    build: .
    ports:
      - 8084:80
    volumes:
      - /symlinks/overpass-data:/db
    environment:
      - OVERPASS_META=yes
      - OVERPASS_MODE=init
      - OVERPASS_PLANET_URL=https://download.geofabrik.de/australia-oceania/australia-latest.osm.bz2
   ##   - OVERPASS_PLANET_URL=https://download.geofabrik.de/australia-oceania/australia-latest.osm.pbf
      - OVERPASS_DIFF_URL=https://planet.openstreetmap.org/replication/minute/
      - OVERPASS_COMPRESSION=gz
   ##   - OVERPASS_PLANET_PREPROCESS='mv /db/planet.osm.bz2 /db/planet.osm.pbf && osmium cat -o /db/planet.osm.bz2 /db/planet.osm.pbf && rm /db/planet.osm.pbf' \
      - OVERPASS_RULES_LOAD=10
      - OVERPASS_UPDATE_SLEEP=3600
      - OVERPASS_USE_AREAS=false
    networks:
      - overpassnet

networks:
  overpassnet:
    driver: bridge

dockerfile:

FROM arm64v8/nginx AS builder

RUN apt-get update \
    && apt-get upgrade -y \
    && apt-get install --no-install-recommends --no-install-suggests -y \
        autoconf \
        automake \
        bash \
        bzip2 \
        ca-certificates \
        curl \
        expat \
        fcgiwrap \
        g++ \
        libexpat1-dev \
        liblz4-1 \
        liblz4-dev \
        libtool \
        m4 \
        make \
        osmium-tool \
        python3 \
        python3-venv \
        supervisor \
        wget \
        zlib1g \
        zlib1g-dev

ADD http://dev.overpass-api.de/releases/osm-3s_latest.tar.gz /app/src.tar.gz

RUN  mkdir -p /app/src \
    && cd /app/src \
    && tar -x -z --strip-components 1 -f ../src.tar.gz \
    && autoscan \
    && aclocal \
    && autoheader \
    && libtoolize \
    && automake --add-missing  \
    && autoconf \
    && CXXFLAGS='-O2' CFLAGS='-O2' ./configure --prefix=/app --enable-lz4 \
    && make dist install clean \
    && mkdir -p /db/diffs /app/etc \
    && cp -r /app/src/rules /app/etc/rules \
    && rm -rf /app/src /app/src.tar.gz

FROM arm64v8/nginx

RUN apt-get update -y \
    && apt-get install -y \
    build-essential \
    cmake \
    libboost-dev \
    libexpat1-dev \
    zlib1g-dev \
    libbz2-dev \
    python3-pip \
    python3-venv

COPY --from=builder /app /app

COPY requirements.txt /app/

RUN python3 -m venv /app/venv && /app/venv/bin/pip install -r /app/requirements.txt

RUN apt-get update \
    && apt-get upgrade -y \
    && apt-get install --no-install-recommends --no-install-suggests -y \
        bash \
        bzip2 \
        ca-certificates \
        curl \
        expat \
        fcgiwrap \
        jq \
        liblz4-1 \
        osmium-tool \
        python3 \
        python3-venv \
        supervisor \
        wget \
        zlib1g \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /app /app

ADD https://raw.githubusercontent.com/geofabrik/sendfile_osm_oauth_protector/master/oauth_cookie_client.py \
    /app/bin/
RUN sed -i -e 's/allow_read_prefs": "yes"/allow_read_prefs": "1"/g' /app/bin/oauth_cookie_client.py
RUN addgroup overpass && adduser --home /db --disabled-password --gecos overpass --ingroup overpass overpass

COPY requirements.txt /app/

RUN python3 -m venv /app/venv \
    && /app/venv/bin/pip install -r /app/requirements.txt --only-binary osmium

RUN mkdir /nginx /docker-entrypoint-initdb.d && chown nginx:nginx /nginx && chown -R overpass:overpass /db

COPY etc/supervisord.conf /etc/supervisor/conf.d/supervisord.conf

COPY etc/nginx-overpass.conf.template /etc/nginx/nginx.conf.template

COPY bin/update_overpass.sh bin/update_overpass_loop.sh bin/rules_loop.sh bin/dispatcher_start.sh bin/start_fcgiwarp.sh /app/bin/

COPY docker-entrypoint.sh docker-healthcheck.sh /app/

RUN chmod a+rx /app/docker-entrypoint.sh /app/bin/update_overpass.sh /app/bin/rules_loop.sh /app/bin/dispatcher_start.sh \
    /app/bin/oauth_cookie_client.py /app/bin/start_fcgiwarp.sh

ENV OVERPASS_RULES_LOAD=1
ENV OVERPASS_USE_AREAS=true
ENV OVERPASS_ALLOW_DUPLICATE_QUERIES=no

EXPOSE 80

HEALTHCHECK --start-period=48h CMD /app/docker-healthcheck.sh

CMD ["/app/docker-entrypoint.sh"]

[jata1]

Just an update but I switched my system disk from a 128GB SSD to a 500GB SSD and now the container started without error and is currently processing the updates from https://planet.openstreetmap.org/replication/minute/

I also stopped all of the other containers when starting overpass so it might have been caused by low memory (no swap file on my rpi5).

Not sure which of these two possibilities but thought I would share in case it helps other folk.

[wiktorn]

I recommend starting with a small extract (like Monaco) to check if everything works fine. Then you can go with larger extracts and any errors then suggests lack of resources (usually memory, but can also be disk).

[jata1]

Thanks @wiktorn. This is a great docker project by the way.

I have it running but needed to download the entire planet in clone mode. I only want speed limit data for Australia so a bit silly 😂

I’m now working on getting an init container with just Australia and it’s processing the diffs at the moment so I think it will work.

I’m 99% sure now my main issue was ram / buffer related.