Image version "7" being reported as having upgrade, even though it's the same sha256 as "7.0.14"

[LorenKeagle]

I'm using WUD to monitor several dozen containers, and it's working great for all except a mongodb container. Here's the relevant compose entry:

  mongo-db:
    image: mongo:7
    labels:
      - 'wud.tag.include=^\d+\.\d+\.\d+$'
      - 'wud.tag.exclude=windowsservercore'

The apps using this container do not support mongo 8, so I have to limit to mongo 7. But I still want to pull the latest version of 7.x.x. The issue is that even after a pull, WUD still reports that there's an upgrade available, even though the running image has the same hash as the latest. Note that I had to explicitly exclude 'windowsservercore' tags, because for some reason WUD was always suggesting the latest tag of that release as an upgrade.

I'm not sure that this is a bug, because I know the hazards around 'latest', and "mongo:7" is basically just "latest" for a previous major version. However, I would have expected WUD to notice that they're the same image and not indicate an update.

Is there a workaround for this, as it's a pretty common tagging strategy for many images (especially databases and OS base images).

[robross0606]

I am seeing the same problem with mariadb.

mariadb:    image: mariadb:11
    labels:
        - wud.tag.exclude=^.+-rc$$

I have the latest version of mariadb:11 image deployed with the same SHA as "11.5.2" image. I can also confirm by looking at the deployed environment that it is running 11.5.2.

However, whats-up-docker still flags 11.5.2 as needing an update because of a semver mismatch. With most of my deployed images, this works fine and whats-up-docker understands the SHA match. However, if you have a wud.tag.exclude label, it appears to flag semver mismatches even though the SHA is the same. The only reason I need this wud.tag is because whats-up-docker is also flagging release candidates as an update.

[fmartinou]

Hi,

When tags are semver (which is true in your situation with mongo versions like 7 or 7.x.y), wud doesn't compare digests. (it compares them only when the tag is not semver (latest, dev...))

That means that, wud will:

• find from the registry all tags matching ^\d+\.\d+\.\d+$
• sort them desc
• compare until one of them is semantically greater than 7
• report if a greater one is found

From my understanding, you're in a "special" case where you want an extra check to ensure that the candidate image found has another hash. It should be doable but it will be an opt-in option because it causes extra requests to the registry (requests that are counted from the Docker hub request quota)

[fmartinou]

Hi,

I think I get it working 😃

It will be part of the next release. Please notice, that to make it work, you'll also need to add the already existing wud.watch.digest=true label on these containers.

(removing all wud data to start from a fresh state will also be needed)

I tested it with both images mongo:7 and mongo:8.

docker run -d --name mongo7 --label 'wud.tag.include=\d+\.\d+\.\d+' --label 'wud.watch.digest=true' m
ongo:7
docker run -d --name mongo8 --label 'wud.tag.include=\d+\.\d+\.\d+' --label 'wud.watch.digest=true' m
ongo:8

• For mongo:7, wud found the mongo:8.0.0 tag
• For mongo:8, wud found no update.

[robross0606]

I can confirm this appears to have worked for mariadb. For the record, this is not really much of an "edge case". In my experience, releasing the same digest SHA as "latest", "8", "8.1" and "8.1.5" is fairly common practice for Docker images. I'm glad this works. 😄

[LorenKeagle]

This doesn't fully solve my issue, although it's a step in the right direction! Thank you for explaining the algorithm to us.

My issue is that I have to pin my image at the 7.x.x branch, because the application does not yet support mongo 8.

With the recent change, WUD now indicates that there is an upgrade from my current (latest 7.x.x) image to "8.0.0". Not only is this not the latest 8.x image, but it doesn't pin the version as expected.

Is this something that can be solved with a simple "startsWith()" call? If I've indicates "7" or "7.1" can we not simply use that version as the prefix for the regex of accepted image versions? It occurs to me that I might be able to do that currently just by putting my prefix in the includes parameter, but I think this could be automatic is we're already making the assumption around semver.

[fmartinou]

It occurs to me that I might be able to do that currently just by putting my prefix in the includes parameter, but I think this could be automatic is we're already making the assumption around semver.

Unfortunately 7 or 7.1 versions are not semver valid.

In my experience, releasing the same digest SHA as "latest", "8", "8.1" and "8.1.5" is fairly common practice for Docker images

I agree; pretty common in Docker world but still a bad practice in general 😃

My personal view:

Tag mutability is an anti-pattern. Semver is a good practice but lots of apps published as Docker images don't comply with it.

Regarding images tagged as x, x.y and x.y.z at the same time (only x and x.y are mutable whereas x.y.z is immutable). Image publishers propose them to meet various user needs:

• "I'm interested to stick to a x major version and I don't care about any particular minor or patch version" => So go for the x tag
• "I'm interested to stick to a x.y major + minor version and I don't care about any particular patch version" => So go for the x.y tag
• "I'm interested to stick to a x.y.z major + minor + patch version" => So go for the x.y.z tag

The principle of wud is to fully embrace the concept of immutable tags without the hassle of manually tracking new versions over time.

Is this something that can be solved with a simple "startsWith()" call? If I've indicates "7" or "7.1" can we not simply use that version as the prefix for the regex of accepted image versions?

The wud.tag.include label already meets the need. I don't want to add some other mechanism on top; it wouldn't be easily understable.

For your situation

docker run -d --name mongo7 --label 'wud.tag.include=7+\.\d+\.\d+' --label 'wud.watch.digest=true' 

will perfectly be working (you'll get only new 7.x.y available updates.

[LorenKeagle]

I've tried it with the 'feature__427_semver_digest_check' wud image from your feature build, and the labels suggested on my mongo container.

I believe it's working, based on the web interface. However, it did not update my HomeAssistant entity for this container. I clicked 'skip' for now. I guess we'll see what it looks like the next time there's a minor mongodb release on the 7.x branch!

[robross0606]

Unfortunately 7 or 7.1 versions are not semver valid.

I think herein lies the problem with this approach in whats-up-docker. Docker tags do not have to comply with semver. They never have complied with semver. So this app is trying to do two things that don't make complete sense:

1. Applying semver filtering logic by default against a system that is not semver compliant and has never claimed to be semver compliant.
2. Applying semver filtering logic without offering configuration that allows the user to control semver rules. The only rules are regex but, behind the scenes, whats-up-docker appears to be trying to apply real semver "greater than" and "less than" rules.

If you're going to apply semver rules, it should really be "opt in" and not by default and, if so, allow the user to pass in real semver rule syntax (^, ~, >=, etc.) instead of only being able to pass in regex matching. When semver rule are not "opt in", rules could maybe be based on regex and image date/time?

What seems to be a corner case to me would be someone wanting to be notified about an image that has the exact same SHA as the one they already have installed. Instead, the system does this by default and the user has to opt-in for what seems to me would be the typical use case. But maybe I have blinders on here. I can't see the benefit of being notified about the image I already have running, regardless of the tag.

[fmartinou]

Docker tags do not have to comply with semver. They never have complied with semver. So this app is trying to do two things that don't make complete sense

Sure Docker registry doesn't enforce semver tagging but most image publishers comply with it because it's the "state of the art" of versioning from ages.

So I'm convinced it's the good approach for wud to work with semver tags at the first place. All other naming can be supported as most as possible but they'll be treated as 2nd class citizens.

What seems to be a corner case to me would be someone wanting to be notified about an image that has the exact same SHA as the one they already have installed. Instead, the system does this by default and the user has to opt-in for what seems to me would be the typical use case. But maybe I have blinders on here. I can't see the benefit of being notified about the image I already have running, regardless of the tag.

I'm not sure to get your point 🤔 :

The default logic currently implemented:

1. The system checks if the tag used in local is semver valid 2a. If so, it compares only the tag value to find if there is a semver greater tag on the registry 2b. If not, it compares the digest used in local to the digest associated to the same tag on the registry
2. It notified only if they're different; you should never be notified about a version on the registry which is the same as yours 🤔

[robross0606]

I'm not sure to get your point 🤔 :

True image uniqueness is not determined by the "tag". It is determined by the SHA. The same SHA can be tagged numerous times, all pointing to the same image. What I'm saying is that your default logic as you described here doesn't consider the fact that the same image may be tagged numerous ways by default.

Also, you said this:

Unfortunately 7 or 7.1 versions are not semver valid.

Then this:

For mongo:7, wud found the mongo:8.0.0 tag

Then this:

The system checks if the tag used in local is semver valid

All of these statements cannot be true. If we hold that "7" is not semver valid then your system should have failed logic step "1" of the rules you listed and not compared "only the tag" to find "8.0.0" to begin with. The same goes for "mariadb:11". If you do not consider "11" to be semantically valid semver, then why is the system not failing step "1"?

[fmartinou]

True image uniqueness is not determined by the "tag". It is determined by the SHA. The same SHA can be tagged numerous times, all pointing to the same image. What I'm saying is that your default logic as you described here doesn't consider the fact that the same image may be tagged numerous ways by default.

Sure but we aren't interested to find if other tags are also pointing to the same SHA as the tag currently used; am I wrong? I don't understand why the fact that the same image may be tagged numerous ways caused wud to report wrong results. There is maybe something I don't see 🤔

If you do not consider "11" to be semantically valid semver, then why is the system not failing step "1"?

Right. The fact is that I parse tags with semver lenient options to maximize chances.

[robross0606]

Ahh, okay. So it is considering "11" to be a valid semantic version with leniency (i.e. the same as "11.0.0"). But this is not really how Docker tags work in most cases. The "11" tag in Docker semantics means "the latest version of 11" which equates to "^11.0.0" and not "11.0.0". This is why it finds "11.5.2" and thinks that is newer than "11" even though they actually have the same SHA.

[fmartinou]

Absolutely.

I can add an option to give the choice to enable or disable "leniency" parsing if it can help.

[robross0606]

For my use case, adding wud.watch.digest did the trick and I don't need anything else right now. I can't speak for others on this ticket. Thanks!

[fmartinou]

Available in 6.6.0

[robross0606]

I just found that this isn't work after all. I have this in my compose file:

services:
  mariadb:
    container_name: mariadb
    image: mariadb:11
    labels:
      - wud.watch.digest=true
      - wud.tag.exclude=^.+-rc$$

Regardless, whats-up-docker is still telling me that I have an upgrade from "11" to "11.5.2" even though 11.5.2 is already deployed and I have the exact same confirmed SHA.