search

terascope / base-docker-image

Base docker image for Node
MIT License
0 stars 0 forks source link

add support for MacOs arm64 platform #14

Closed sotojn closed 9 months ago

sotojn commented 9 months ago

When trying to run current base image on an arm Mac, we receive the following error

jsoto@Josephs-MacBook-Pro teraslice % docker run terascope/node-base:14.21.3
Unable to find image 'terascope/node-base:14.21.3' locally
14.21.3: Pulling from terascope/node-base
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.

We need to figure out how to support multiple platforms with our docker images.

sotojn commented 9 months ago

Here is docker documentation on supporting multiple images

https://docs.docker.com/build/building/multi-platform/

godber commented 9 months ago

I've spent a little more time reviewing the output of the earlier build processes and I have some concerns. Specifically, it looks like when node-rdkafka is installed, it is detecting it's architecture as AMD64 here:

https://github.com/terascope/base-docker-image/actions/runs/6054662342/job/16432423437#step:5:323

So I will be interested to know if this image does run on ARM and if it can successfully require the node-rdkafka module.

I have also thought of a way for us to "get images out" of our build.yml workflows. I think we can upload them as artifacts to github using something like docker save, then we can download and try them out before merging next time.

godber commented 9 months ago

Ok, we're still definitely missing something, after getting both the build and release workflows updated we have the following situation:

docker buildx imagetools inspect terascope/node-base:14.21.3
Name:      docker.io/terascope/node-base:14.21.3
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:833e6f7530b9fe922f6bccd8c52c84065441821eccade8ed605114aaf9a0b353

Manifests:
  Name:        docker.io/terascope/node-base:14.21.3@sha256:a656df7ef8a426005761dbb0063e368636614ebc53135059c588f3cccd0e828f
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/arm64

  Name:        docker.io/terascope/node-base:14.21.3@sha256:29ce4c6a1ab90cc403646d00c3a3e0665149c7e49c7888048973c7e9c28cecf2
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations:
    vnd.docker.reference.digest: sha256:a656df7ef8a426005761dbb0063e368636614ebc53135059c588f3cccd0e828f
    vnd.docker.reference.type:   attestation-manifest

When running docker run terascope/node-base:14.21.3 on MacOS ARM, I get the tini output as we would hope.

When running docker run terascope/node-base:14.21.3 on MacOS Intel, I get the tini output as we would hope.

docker run terascope/node-base:14.21.3
Unable to find image 'terascope/node-base:14.21.3' locally
14.21.3: Pulling from terascope/node-base
docker: no matching manifest for linux/amd64 in the manifest list entries.
See 'docker run --help'.

So, now we've broken the important one.

I will rollback both the changes so we have working Intel images which are priority.

godber commented 9 months ago

I reverted the platform changes and there is now an AMD64 image:

https://github.com/terascope/base-docker-image/pull/17

docker buildx imagetools inspect terascope/node-base:14.21.3
Name:      docker.io/terascope/node-base:14.21.3
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:4c05d4e96fe5b980f9de784180fee9ab0a594addf6d836d4f155ac23a39e902a

Manifests:
  Name:        docker.io/terascope/node-base:14.21.3@sha256:f9ab98bc184b88494dcc2c24aacf857fdc4cfff95b760a386ac5308282d2a5c4
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/amd64

  Name:        docker.io/terascope/node-base:14.21.3@sha256:7c339e6ada43522b667d3a2de001519dd4f12af39ad453986632fd18f9103f00
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations:
    vnd.docker.reference.digest: sha256:f9ab98bc184b88494dcc2c24aacf857fdc4cfff95b760a386ac5308282d2a5c4
    vnd.docker.reference.type:   attestation-manifest
godber commented 9 months ago

I have also thought of a way for us to "get images out" of our build.yml workflows. I think we can upload them as artifacts to github using something like docker save, then we can download and try them out before merging next time.

Rather than this, we should change the build workflow to publish images to a dev or test tag. So the "prod" images have the names:

terascope/node-base:14.21.3
terascope/node-base:16.20.0
terascope/node-base:18.16.0
terascope/node-base:14.21.3-core
terascope/node-base:16.20.0-core
terascope/node-base:18.16.0-core

Maybe we make some called:

terascope/node-base:14.21.3-test
terascope/node-base:16.20.0-test
terascope/node-base:18.16.0-test
terascope/node-base:14.21.3-core-test
terascope/node-base:16.20.0-core-test
terascope/node-base:18.16.0-core-test
busma13 commented 9 months ago

https://docs.docker.com/build/guide/multi-platform/

https://docs.docker.com/build/ci/github-actions/multi-platform/

11:02 https://docs.docker.com/build/ci/github-actions/multi-platform/

godber commented 9 months ago

This was completed in https://github.com/terascope/base-docker-image/pull/19