search

Sage-Bionetworks / sage-monorepo

Where OpenChallenges, Schematic, and other Sage open source apps are built
https://sage-bionetworks.github.io/sage-monorepo/
Apache License 2.0
23 stars 12 forks source link

[Story] Publish images without rebuilding them in the CI workflow #1744

Closed tschaffter closed 1 year ago

tschaffter commented 1 year ago

What projects is this story for?

No response

As a user, I want

NA

Description

The CI workflow has the following three steps for images: build, scan and publish.

The same tool is used to build and publish the images, the main difference being the value of push:. The motivation for separating the two is that we can do a scan after building the images, and depending on the result of the scan, make the workflow fail and not push images.

Running the step responsible for the publishing the images attempt to build the images again, though this operation goes much faster because layers previously built are cached.

In at least one occurrence, the publish step failed because "no space available" for four images that fail at a COPY operation.

The goal of this ticket is to identify if there is a way to push the images with the same tool without attempting to rebuild the images. The main benefit would be to make the workflow a bit faster.

Acceptance criteria

No response

Tasks

No response

Anything else?

No response

Have you linked this story to a GitHub Project?

tschaffter commented 1 year ago

Prototype

Build an image:

$ nx build-image openchallenges-zipking

$ docker images
REPOSITORY                                       TAG           IMAGE ID       CREATED       SIZE
ghcr.io/sage-bionetworks/openchallenges-zipkin   local         01befd50513c   5 weeks ago   157MB
ghcr.io/sage-bionetworks/openchallenges-zipkin   sha-06558c9   01befd50513c   5 weeks ago   157MB

Remove the image by name:

vscode@3be174c018e7:/workspaces/sage-monorepo$ docker rmi $(docker images --filter=reference="ghcr.io/sage-bionetworks/openchallenges-zipkin:*" --quiet) --force
Untagged: ghcr.io/sage-bionetworks/openchallenges-zipkin:local
Untagged: ghcr.io/sage-bionetworks/openchallenges-zipkin:sha-06558c9
Deleted: sha256:a5fd2343980c1b912ee3d06774a2ef3893b7d73155b3c3628efaf457fd653cb4
Error: No such image: a5fd2343980c
vscode@3be174c018e7:/workspaces/sage-monorepo$ echo $?
0
tschaffter commented 1 year ago

The error thrown when trying to push an image to GHCR from a fork (expected):

>  Nx Container  Removing temp folder /tmp/docker-build-push-nkqKbx

 >  NX   buildx failed with: ERROR: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed