Empty docker content digest causing service creation/update impossible - No 2

[stefanmichalk]

Hi,

we want to use swarmpit for our system, but there's an error, we don't get rid of.

Service creation failed. rpc error: code = InvalidArgument desc = ContainerSpec: "***/package-name:main@" is not a valid repository/tag

We searched the issues and there was the same issue, but no solution for this (https://github.com/swarmpit/swarmpit/issues/312)

We're using Docker Hub (private), Auth went well, swarmpit searches the Hub, shows the Repo, etc. We tagged the package with 'latest', 'main', etc. and nothing seems to work.

We tried swarmpit v1.9, also Edge (currently v1.10-850a7f9). No changes at all.

Our Github Action is pretty simple

name: create and push docker image to Docker Hub

on:
  push:
    branches: [ 'main', 'staging', 'develop' ]

env:
  IMAGE_NAME: ${{ github.event.repository.name }}

jobs:
  build-and-push-image:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - name: checkout repository
        uses: actions/checkout@v4

      - name: Docker meta
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ secrets.DOCKER_HUB_USERNAME }}/${{ env.IMAGE_NAME }}

      - name: set up docker buildx
        uses: docker/setup-buildx-action@v3

      - name: login to docker hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_PASSWORD }}

      - name: build and push docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: |
            ${{ secrets.DOCKER_HUB_USERNAME }}/${{ env.IMAGE_NAME }}:${{ github.ref_name }}
            ${{ secrets.DOCKER_HUB_USERNAME }}/${{ env.IMAGE_NAME }}:latest

The latest for each branch is currently just a test ;-)

Fun fact If we create a stack, it can pull the image, create a service, everything is fine. But when you edit the service and toggle Auto-Deploy for example, you will run in the same error message as above.

Does anyone run in the same problem and can provide a solution for this?

All the best Stefan

[mrspartak]

Just ran into the same issue. Everything was fine, then servers spiked in resource usage and now I'm in this situation too

[Jognu]

Hi, Same here with rabbitmq:3.12.11-management from dockerhub.

[stefanmichalk]

This is urgent - any ideas how to fix this?

[stefanmichalk]

Hello :-)

It seems to be a problem with the version of GitHub Actions (Metadata, etc.).

We used Docker Hub as container registry, but out of the blue I changed it to ghcr.io (GitHub Container Registry).

I followed the guides Working with github packages registry and publishing and installing with github actions.

Add a Registry v2 in Swarmpit

1. use https://ghcr.io as URL
2. make it secured
3. use your PERSONAL username and the created token like described

For creating a service

1. just enable "specify repository manually"
2. use https://ghcr.io/YOUR-USERNAME-OR-ORGANIZATION/THE-REPOSITORY
3. click next
4. select the tag you want to use
5. Deploy

After Deploying you can make changes to the service and all works like expected.

[mrq1911]

workaround is to the edit whole stack

[ganicus]

Not a great workaround - why is there an "@" appended at the end of the tagname?

[stefanmichalk]

Not a great workaround - why is there an "@" appended at the end of the tagname?

What's the matter?

The @ is tagging the version of the image. Seems like swarmpit has trouble reading meta information.

If you have a better workaround, share it.

[ganicus]

I don't have a better workaround. Just is a hassle to not be able to update a single service.

Why can't the "@" be omitted if "imageDigest" is nil?

I don't write in clojure or I'd love to contribute.

[ganicus]

@stefanmichalk @mrq1911

Within the docker/engine/mapper/inbound.clj image-digest is defined from a string split that will return an empty string if not present instead of nil. It is the same way for defn ->task starting on line 127 and defn ->service on line 382.

Here is the shared logic

image-info (str/split image #"@")
        image-name (first image-info)
        image-digest (second image-info)

The ternary evaluation on the in outbound.clj for digest is here on line 193

(defn ->service-image
  [service digest?]
  (let [repository (get-in service [:repository :name])
        tag (get-in service [:repository :tag])
        digest (get-in service [:repository :imageDigest])]
    (if digest?
      (str repository ":" tag "@" digest)
      (str repository ":" tag))))

The empty string will evaluate true giving a repo + tagname + "@" with no digest. Resulting in an invalid tagname error.

suggest changing in the inbound func defs to

(let [image (get-in task [:Spec :ContainerSpec :Image])
      image-info (str/split image #"@")
      image-name (first image-info)
      image-digest (if (> (count image-info) 1) (second image-info) nil)]
  ...
  :repository {:image       image-name
               :imageDigest image-digest}
  ...)

Like I said, i don't write or build in clojure so I don't have the time to setup a dev env to create a pull request.

[mrspartak]

Sad this seem to never be fixed. I encounter it with services like nginx etc. I can update the version directly from a VM, but updating big configs is just impossible now. Not sure what to do

[jcnmsg]

Found myself looking at the exact same portion of code mentioned in the comments above. I don't write closure either but it seems to me the @ is being appended to the string even if digest is nil so this simple edit would probably fix it?

(defn ->service-image
  [service digest?]
  (let [repository (get-in service [:repository :name])
        tag (get-in service [:repository :tag])
        digest (get-in service [:repository :imageDigest])]
    (if (and digest? digest)
      (str repository ":" tag "@" digest)
      (str repository ":" tag))))