octodns / octodns-docker

OctoDNS – DNS as code – bundled as Docker images
12 stars 2 forks source link

Multi Arch Images? #60

Closed rjocoleman closed 2 months ago

rjocoleman commented 1 year ago

This image is very handy (thank you!) it currently supports amd64 architecture.

Would you be open to a PR that added support to the publish-on-push-to-main action that adds multi-arch support via the additional steps of:

https://github.com/docker/setup-qemu-action & https://github.com/docker/setup-buildx-action

and modification to the build script: e.g.

docker buildx build --tag "$TAG" \
    --label org.opencontainers.image.created="$(date --rfc-3339=seconds)" \
    --label org.opencontainers.image.url="https://github.com/${GITHUB_REPOSITORY}" \
    --label org.opencontainers.image.documentation="https://github.com/octodns/octodns#readme" \
    --label org.opencontainers.image.source="https://github.com/${GITHUB_REPOSITORY}" \
    --label org.opencontainers.image.revision="$GITHUB_SHA" \
    --label org.opencontainers.image.licenses=MIT \
    --platform linux/amd64,linux/arm/v7,linux/arm64 \
    "$FLAVOR"

Update:

After some experimenting, I've found that this isn't going to work easily due to the decoupled push step and the cibuild / release as the images aren't loaded into the local store and there is some incompatibility around this: https://github.com/docker/roadmap/issues/371 (basically can't load the image into local store to test and/or push later if it's multi-arch so would require larger rework of this ci/cd pipeline or waiting until docker supports this, perhaps..)

parkr commented 2 months ago

linux/arm/v7 has a known issue with the cryptography Python package: https://github.com/docker/buildx/issues/395

Basically, it uses Rust and therefore needs to compile Rust code with cargo, but cargo cannot build on arm/v7:

#26 160.4 Collecting cryptography>=2.5 (from azure-identity>=1.16.1->octodns-azure==0.0.9->-r requirements.txt (line 2))
#26 160.4   Downloading cryptography-43.0.1.tar.gz (686 kB)
#26 160.6      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 686.9/686.9 kB 5.7 MB/s eta 0:00:00
#26 161.4   Installing build dependencies: started
#26 233.2   Installing build dependencies: still running...
#26 242.0   Installing build dependencies: finished with status 'done'
#26 242.0   Getting requirements to build wheel: started
#26 242.7   Getting requirements to build wheel: finished with status 'done'
#26 242.7   Preparing metadata (pyproject.toml): started
#26 246.2   Preparing metadata (pyproject.toml): finished with status 'error'
#26 246.2   error: subprocess-exited-with-error
#26 246.2   
#26 246.2   × Preparing metadata (pyproject.toml) did not run successfully.
#26 246.2   │ exit code: 1
#26 246.2   ╰─> [16 lines of output]
#26 246.2           Updating crates.io index
#26 246.2       warning: spurious network error (2 tries remaining): could not read directory '/root/.cargo/registry/index/github.com-1285ae84e5963aae/.git/refs': Value too large for defined data type; class=Os (2)
#26 246.2       warning: spurious network error (1 tries remaining): could not read directory '/root/.cargo/registry/index/github.com-1285ae84e5963aae/.git/refs': Value too large for defined data type; class=Os (2)
#26 246.2       error: Unable to update registry `crates-io`
#26 246.2       
#26 246.2       Caused by:
#26 246.2         failed to fetch `[https://github.com/rust-lang/crates.io-index`](https://github.com/rust-lang/crates.io-index%60)
#26 246.2       
#26 246.2       Caused by:
#26 246.2         could not read directory '/root/.cargo/registry/index/github.com-1285ae84e5963aae/.git/refs': Value too large for defined data type; class=Os (2)
#26 246.2       💥 maturin failed
#26 246.2         Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
#26 246.2         Caused by: `cargo metadata` exited with an error:
#26 246.2       Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-9dl_sbyq', '--interpreter', '/opt/octodns/env/bin/python']' returned non-zero exit status 1.
#26 246.2       Checking for Rust toolchain....
#26 246.2       Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-9dl_sbyq --interpreter /opt/octodns/env/bin/python`
#26 246.2       [end of output]
#26 246.2   
#26 246.2   note: This error originates from a subprocess, and is likely not a problem with pip.
#26 247.7 error: metadata-generation-failed
#26 247.7 
#26 247.7 × Encountered error while generating package metadata.
#26 247.7 ╰─> See above for output.
#26 247.7 
#26 247.7 note: This is an issue with the package mentioned above, not pip.
#26 247.7 hint: See above for details.

Another project documented their workarounds here: https://github.com/flo-at/minmon/issues/14

One includes using a tmpfs mount for /var/lib/docker, which, amazingly, solves the issue (documented here: https://github.com/JonasAlfredsson/docker-on-tmpfs).

I think arm/v7 is a nice-to-have at the moment, so in order to unlock arm64, I've omitted it from my PR https://github.com/octodns/octodns-docker/pull/74.