konflux-ci / rpm-lockfile-prototype

GNU General Public License v3.0
3 stars 10 forks source link

Use the same repo file as the base image #26

Closed msrb closed 2 months ago

msrb commented 2 months ago

The problem

During the Y-Stream development, the compose changes often and thus we need to update the repo file(s) used by rpm-lockfile-prototype. We don't use the "latest" repo URLs -- we use specific nightlies.

The challenge is that both base images and leaf images should be using the same repo URLs. Otherwise we might run into problems like not being able to re-install packages because they simply don't exist in newer repos. Installing subpackages when other RPMs from the older build already exist in the base image can be also problematic (subpackages often have strict version requires on other subpackages from the same build).

Implementation idea(s)

Images have labels, like for example:

               "vcs-ref": "d8b50ec88c18e2e97648c3391a8709b598f4d95a",
               "vcs-type": "git",

The information from these labels could be used to locate a repo file in the remote git repository (base image repo).

lubomir commented 2 months ago

This would be fairly easy if the image was built in OSBS and Koji. The labels give NVR, which can be used to find the repos. But if the image is built in Konflux, there's no kojihub to get the information from. The labels only have the vcs-ref and vcs-type, but no information about where the git repo actually is.

Do you have any suggestion on how to find out where the git repo is?

The git repo probably contains a repofile, and that could be used by the resolver. But then it's tricky to find out the exact branch. Maybe this could be done with something like

contentOrigin:
  repofiles:
    - location: https://gitlab.com/<path-to-repo>/-/blob/{vcs-ref}/<repofile>
      # One of the following could be provided
      varsFromContainerfile: Containerfile
      varsFromImage: registry.example.com/some-image:latest
lubomir commented 2 months ago

PR #27 has a draft that should make this use case easier. It seems to work in my testing, but it has an issue if the git repo requires authentication. The resolver doesn't support any way of logging in.

Please test and share any feedback on the PR.

lubomir commented 2 months ago

I have updated the PR with additional commit. With that, the config file can specify this:

contentOrigin:
  repofiles:
    - giturl: https://$USER:$PASS@git.example.com/my-repo.git
      file: my.repo
      gitref: "{vcs-ref}"
      varsFromContainerfile: Containerfile

The url can reference environment variables. For example gitlab.com would support my username + personal access token instead of the password. All of giturl, file and gitref can also reference labels from the image (though I don't know if that's useful in other places than gitref).

If you want to run this with a bot, the bot would need to have an access token usable for the repo, and you would need to know in what env var it makes it available. An alternative implementation would allow configuring a credentials helper, and the calling bot would need to make one available.

msrb commented 2 months ago

Thank you!

I think this should do the trick ;)