Feature Request: Configurable Latest Tags Count for ImageRepository

[kahirokunn]

Current Behavior

Currently, the image-reflector-controller in FluxCD limits the number of latest tags to 10 for all ImageRepository resources. This is hardcoded in the getLatestTags function:

https://github.com/fluxcd/image-reflector-controller/blob/15a1b0968c3a935bbffefdc9d0ca5ae36854f3c5/internal/controller/imagerepository_controller.go#L61

https://github.com/fluxcd/image-reflector-controller/blob/15a1b0968c3a935bbffefdc9d0ca5ae36854f3c5/internal/controller/imagerepository_controller.go#L630-L642

Desired Behavior

We would like to make the number of latest tags configurable on a per-ImageRepository basis, with a default of 10. This would allow users to retrieve more (or fewer) latest tags as needed for specific repositories.

Proposed Solution

We propose adding a latestTagsCount field to the ImageRepository spec:

apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
  name: app1
  namespace: apps
spec:
  latestTagsCount: 100  # New field
  interval: 1h
  image: docker.io/org/image
  exclusionList:
    - "^.*\\.sig$"
    - "1.0.2"
    - "1.1.1|1.0.0"

Implementation Details

1. Update the ImageRepository CRD to include the new latestTagsCount field.
2. Modify the getLatestTags function to accept a parameter for the count:

func getLatestTags(tags []string, count int) []string {
    // ... implementation using the provided count
}

3. Update the controller logic to use the specified count from the ImageRepository spec, falling back to the default of 10 if not specified.

Benefits of Increasing Latest Tags Count

1. Better visibility into recent releases: For projects with frequent releases or using date-based versioning, having access to more than 10 latest tags provides a more comprehensive view of recent project history.

2. Improved automation capabilities: With more tags available, users can create more sophisticated automation workflows, such as rolling back to specific versions or tracking longer-term trends in releases.

3. Enhanced compatibility with various versioning schemes: Some projects may use versioning schemes that produce many tags in a short time (e.g., build numbers or git commit hashes). A higher tag count allows for better compatibility with these schemes.

4. More flexible image selection: Users can implement more nuanced policies for selecting images based on a larger pool of recent tags.

5. Better support for canary releases and A/B testing: With access to more tags, users can more easily manage multiple versions of an application for testing or gradual rollout purposes.

Using CEL for Validation

To ensure that the latestTagsCount is not set to a value less than 10, we can use Common Expression Language (CEL) for validation. Here's an example of how to implement this validation:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: imagerepositories.image.toolkit.fluxcd.io
spec:
  # ... other CRD fields ...
  versions:
    - name: v1beta2
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                latestTagsCount:
                  type: integer
                  default: 10
              x-kubernetes-validations:
                - rule: "has(self.latestTagsCount) ? self.latestTagsCount >= 10 : true"
                  message: "latestTagsCount must be greater than or equal to 10"

This CEL rule does the following:

1. It checks if latestTagsCount is present in the spec.
2. If it is present, it validates that the value is greater than or equal to 10.
3. If it's not present, the validation passes (allowing the default value of 10 to be used).

This approach ensures that users can increase the latestTagsCount beyond 10, but prevents them from setting it to a value less than 10, maintaining a minimum level of recent tag history.

[stefanprodan]

The latest tags number is just for display, the controller stores all tags of an image, not only the latest 10. The policies for a specific image repo, are executed on the full list of tags.

[kahirokunn]

@stefanprodan Thank you for the clarification regarding the storage and display of image tags. There are situations during troubleshooting when we may need to reference a broader range of tags. Additionally, we are interested in creating a controller to automate this process.

[stefanprodan]

The assumptions made in this issue are quite wrong, the number of tags from status play no role in the controller's behaviour. Also the tags listed in status can't be used for any decision making because the upstream registry can serve pages in any order, what you think it's latest can be oldest. The proper way to select "latest" is using an ImagePolicy that looks at all tags and selects latest based on the regex/semver.