Implemented check 5.27

[badamowicz]

Hi all! This PR is just a proposal serving as a basis for some discussions. It's not a ready-to-merge PR at all!

Basic idea of implementing check 5.27 was to retrieve the latest version tag from the registry and compare it with the image tag under test. In my proposal this is basically implemented using AWS ECR.

Of course it is not acceptable to have some hard coded commands serving only AWS. We need a generic solution for this. Some ideas to address this might be:

• Provide a list of commands to be filled with arguments at runtime for other registries.
• Additionally provide a command line switch like -r <registry-type> to specify which command to use.

Or:

• Make the user provide his own command on command line or via environment variable.

Other things to consider are:

    # Bad:
    # * hard coded command
    # * How to reliably retrieve the repository's name?
    # * How to configure region?
    latestTag=$(
      aws ecr describe-images --repository-name "mycompany/openjdk11-base" --filter tagStatus=TAGGED \
        --region eu-central-1 --query 'sort_by(imageDetails, &imagePushedAt)[-1].imageTags[]' \
        | jq '.[] | select(. != "latest")'
      )

Maybe this proposal is too complicated to be implemented at all. However, I'd like to hear your opinions.

Looking forward to your ideas/suggestions.

[rhatdan]

What about using the podman images?

quay.io/podman/stable? quay.io/podman/hello?

[rhatdan]

@vrothberg @lsm5 WDYT?

[badamowicz]

What about using the podman images?

quay.io/podman/stable? quay.io/podman/hello?

What do you mean? Where and how could this help?

[lsm5]

RE: aws ecr, I haven't yet messed around enough with the aws cli yet, so no strong opinions so far. @cevich, any thoughts?

[cevich]

I've gotten as far as running help, does that count?

note: I don't think the need here is aws-specific. In fact it seems like skopeo list-tags could be used for this?

[badamowicz]

note: I don't think the need here is aws-specific. In fact it seems like skopeo list-tags could be used for this?

Good idea! Will have a closer look at it.

[rhatdan]

@badamowicz I thought that you wanted an image to test against.

[badamowicz]

@rhatdan No. The image is not the issue here.

[badamowicz]

With my latest commit I've now introduced a combination of skopeo list-tags and jq:

latestTag=$(skopeo list-tags "${imgUrl}" | jq -r '[.Tags[] | select(. != "latest")] | max')

This of course a much better solution than using individual registry commands. However, there are still a few things to consider. For example the max functions of jq will no work in all cases. At least not if the image tag contains a Git commit like 4.2.0-befee09a. It's impossible to say if this tag is smaller or larger than for example 4.2.0-b6598f21.

I thought about appending a grep command like:

latestTag=$(skopeo list-tags "${imgUrl}" | jq -r '[.Tags[] | select(. != "latest")] | max' | grep -Eo "^[[:digit:]]*.[[:digit:]]*.[[:digit:]]*")

This will isolate the semantic versioning part of the tag but will not ensure at all that we will really get the latest tag. So I'm still thinking about a better solution. Any ideas?

[badamowicz]

At least not if the image tag contains a Git commit like 4.2.0-befee09a. It's impossible to say if this tag is smaller or larger than for example 4.2.0-b6598f21.

Meanwhile I think this is just a homemade problem within my current project. No need to discuss this here I guess.

[badamowicz]

Another thing I'd like to discuss is that the implementation of 5.27 as I did it now deviates from what is laid out inside the official Docker Benchmark Document. Chapter 5.27 says:

Audit:
You should carry out the following steps:
Step 1: Open your image repository and list the image version history for the image you are inspecting.
Step 2: Observe the status when the docker pull command is triggered. If the status is shown as Image is up to date, it means that you are getting the cached version of the image.
Step 3: Match the version of the image you are running to the latest version reported in your repository and this will tell you whether you are running the cached version or the latest copy.

So it's about comparing the cached versus the actual version. Whereas my implementation simply tries to retrieve the highest (latest) tag from the repository and compares it with the current tag. Is this basically acceptable? Maybe we could rename this test to something like check_5_27a in order to avoid confusion. What do you think?

[badamowicz]

Is this basically acceptable? Maybe we could rename this test to something like check_5_27a in order to avoid confusion. What do you think?

No replies so far so I think we could leave it as it is. Will change to "Ready for review".

[rhatdan]

LGTM

[rhatdan]

@cevich @lsm5 PTAL

[badamowicz]

Guys, can we merge?

[cevich]

I'm good. @rhatdan ?