Please provide a unique docker tag:

[jseiser]

Have you read the Important information text above

• [X] Yes i did

Current behavior

You tag everything latest which makes pulling new images harder than it needs to be

Desired behavior

Every time you push to dockerhub there should be a unique tag.

Links to screenshots

No response

To Reproduce

Steps to reproduce the behavior:

1. Docker Hub

Software setup

• OS: Kubernetes

Hardware setup

A bunch of nodes

Additional context

https://medium.com/@mccode/the-misunderstood-docker-tag-latest-af3babfd6375

In short, if you are re-using the latest tag, my system has no idea it should be pulling a newer image. If you do not want to look into versioning, just including the git short sha or something would make pulling the image easier.

Thanks.

[jseiser]

You could do something like..

      - name: Docker meta
        id: meta
        uses: docker/metadata-action@v5
        with:
          # list of Docker images to use as base name for tags
          images: |
            jammsen/palworld-dedicated-server         
          # generate Docker tags based on the following events/attributes
          tags: |
            type=sha  
      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

That would give you something like

jammsen/palworld-dedicated-server :latest jammsen/palworld-dedicated-server:8676523

[jammsen]

Hey @jseiser im about 75% sure what you mean, are you able to talk with me in Discord or Teamspeak to clear a few of my questions?

[pchang388]

From other projects I've seen, you can accomplish this by triggering a build on tag pushes.

git tag v1.X.X

git push origin --tags

Then in your github ci, you can have something that triggers on tag creations

---
name: Create Official Release and Push Artifacts

on:
  push:
    tags:
      - v*

And then use the docker-meta action to create a tagged release and a latest if preferred

- name: Docker meta
    id: meta
    uses: docker/metadata-action@v5
    with:
      images: docker.io/${{ github.repository }}
      flavor: |
        latest=${{ fromJSON(inputs.latest) }}
      # for some reason can't get this to show up from docker image labels
      # placing here for now
      labels: |
        org.opencontainers.image.description=Prometheus exporter for tdarr
      tags: |
        ## add the event types that should be added as tags
        ## on release - for use by users
        ## version ; shorthand for {{major}}.{{minor}}.{{patch}} (can include pre-release)
        type=semver,pattern={{ version }} 

Then you can use the docker-build action to create the new image with latest and semantic versioning tag

- name: Build Docker Image
    if: github.event_name != 'pull_request'
    uses: docker/build-push-action@v5
    with:
      context: .
      file: ./Dockerfile
      platforms: linux/amd64,linux/arm64
      push: ${{ fromJSON(inputs.push) }}
      tags: ${{ steps.meta.outputs.tags }}
      labels: ${{ steps.meta.outputs.labels }}
      build-args: |
        VERSION=${{ steps.meta.outputs.version }}
        BUILDTIME=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.created'] }}
        REVISION=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}

I've seen a similar pattern in other projects and added it to my own here as a reference: https://github.com/homeylab/tdarr-exporter/blob/main/.github/actions/docker/action.yml

[pchang388]

In the end, much of this is personal/project preference based on what you need most. For smaller projects, I would recommend a similar pattern to what I implemented here: https://github.com/homeylab/tdarr-exporter/tree/main/.github/workflows

Essentially actions on PR open, branch merge, Tagged release.

PR open -> Test/lint, etc. Merge into main -> trigger build and deploy to main tag in dockerhub. This tag is for latest code in repo and not considered stable. Tag Push -> trigger build of latest and X.X.X tag build. Users can pin app to the tag or pull latest.

Example:	tag	description
latest	Latest stable release and is updated with each new release.
X.X.X	Semantic versioned releases are also provided if preferred for stability or other reasons.
main	This tag reflects the main branch of this repository and may not be stable

This pattern works fine (imo) for small projects. Industry CI/CD is much more focused on testing and leveraging temporary/JIT compute to do more integration and E2E testing before reaching production but for github projects, the method I described may work out well for you

Hope this helps!

[pchang388]

If I get some time, I'll open a PR to get the ball rolling, if needed/wanted, to help get the outline of this process in the yaml github files.

[jammsen]

This pattern works fine (imo) for small projects. Industry CI/CD is much more focused on testing and leveraging temporary/JIT compute to do more integration and E2E testing before reaching production but for github projects, the method I described may work out well for you

Hmmm yeah im doing DevOps work at my company i know and teach this, though with 2 other products then GH. Im just thinking about what i want here (some easy, maybe shortended version of sha-commit-hash) and what i really like (https://hub.docker.com/_/debian and https://hub.docker.com/_/ubuntu have a really well organized site with multiplate tags to stay up to date, but also version pin if needed very hard) and which version it should be (as a single guy doing this, i would have it rather small and automated heavily, than very complex). And yes sometimes im one of the guys who surely forgets to tag or to push with tags, happend in the past and will in the future 😎

If I get some time, I'll open a PR to get the ball rolling, if needed/wanted, to help get the outline of this process in the yaml github files.

Dont just spend hours of work on a PR that i might not want to use, thats why i asked @jseiser if he has time to do a call in voice to better unterstand this and have someone to brainstorm on this. I would rather have this than the code i maybe dont use and someone is mad at me 😭

[jseiser]

Sorry im traveling for work this week.

In our projects we generate tags based off of git commits, thats probably over kill for something of this size. Just any unique tag would be great, so we dont run into issues of docker caching the latest tag.

[jammsen]

Sorry im traveling for work this week.

In our projects we generate tags based off of git commits, thats probably over kill for something of this size. Just any unique tag would be great, so we dont run into issues of docker caching the latest tag.

What do you mean by that? Isnt Docker supposed to cache this? Or what do you consider problems?

[jammsen]

This looks promising i think:

Not sure how this should be helping in your request and the cache-problem i just dont understand, please explain in voice when you back at home.

[jammsen]

latest as always, master for branch name and GIT_SHA shortend to 7 chars

[jseiser]

what you just posted ill work perfect.

[jammsen]

@jseiser happy that helps, but please help me solve my confusion

Not sure how this should be helping in your request and the cache-problem i just dont understand, please explain in voice when you back at home.

Please answer this.

[jseiser]

The tag latest is mutable, it changes what it references over time.

1. You run this image and a valid config on server1. It pulls down latest. You know take the same image tag and config to server2, and it would pull down latest, but you have no promise that its the same image.
2. When running in kuberenetes, it will normally check if the image already exists on the host before it tries to download it, so you can end up in a situation here it already sees image:latest and decides it doesnt need to pull a new image, putting you back in the same situation as #1

[jammsen]

1. You run this image and a valid config on server1. It pulls down latest. You know take the same image tag and config to server2, and it would pull down latest, but you have no promise that its the same image.

Oh yeah then now you can just Hard-Version-Pin by sha-has and make sure you have the right version on every node of your cluster. Seems legit.

2. When running in kuberenetes, it will normally check if the image already exists on the host before it tries to download it, so you can end up in a situation here it already sees image:latest and decides it doesnt need to pull a new image ....

Ooooh now i starting to see/get your point, well in my firm where i educate people on using docker and registries we use Harbor all the time, we do always pull from Harbor in ci-context, because it has the information if its current or not and the cache-proxies check on every request in the upstream if the sha has changed and if the new image needs to be downloaded and be cached. I think i just thought about this a little bit to ?"enterprisy"? i guess? 🤔

[uncaught]

Guess it depends on your work bubble then :D

In my firm we avoid latest for anything that goes in production at all cost.

You have no way of rolling back to a different version if everything is latest ;)

Not to mention, that you can never figure out what version you are actually running, unless you added some different information like docker labels.

[jseiser]

We actually use harbor, but anything tagged latest is actually prevented from starting up.

https://kyverno.io/policies/best-practices/disallow-latest-tag/disallow-latest-tag/

[jammsen]

Guess it depends on your work bubble then :D

In my firm we avoid latest for anything that goes in production at all cost.

You have no way of rolling back to a different version if everything is latest ;)

Not to mention, that you can never figure out what version you are actually running, unless you added some different information like docker labels.

Thats why im trying to teach everyone to hardcore use Version-Pinning and not even like Node:18 tags and stuff like that. Sorry if that came across as weird written.

[jammsen]

Thanks for the awesome exchange of experiences! I value that very highly!

Is this issue resolved? Please consider giving this repo and the docker-hub-repo a star.