support hostname + base path rewrite

cppforlife commented 3 years ago

Describe the problem/challenge you have

jfrog supports a way to proxy image downloads from a particular hostname + base path (e.g. jfrog.corp.com/project-x proxies all requests to upstream-registry.com/some-path). it should be possible to rewrite lock file to rewrite lock entries.

Describe the solution you'd like

if user is using imgpkg bundles then i think there would be no need for this since bundle would be proxied and found in the virtual repo. .imgpkg/images.yml should be relative to that location.

should we have a feature in kbld that facilities rewriting hostnames, not sure...

Anything else you would like to add:

https://www.jfrog.com/confluence/display/JFROG/Docker+Registry#DockerRegistry-VirtualDockerRepositories ?

pivotaljohn commented 3 years ago

As someone installing an app containing a bunch of images And I have a virtual repository configured (which includes all of the possible source registries that I'd pull my images from) I want to be able to rewrite my ImagesLock such so I'd obtain all of those images from that virtual repository

For example:

Given an initial ImagesLock:

# images.yml
apiVersion: imgpkg.carvel.dev/v1alpha1
kind: ImagesLock
images:
- image: index.docker.io/dkalinin/k8s-simple-app@sha256:4c8b96d4fffdfae29258d94a22ae4ad1fe36139d47288b8960d9958d1e63a9d0
  annotations:
    kbld.carvel.dev/id: docker.io/dkalinin/k8s-simple-app
- image: gcr.io/k8slt/k8s-simple-app2@sha256:deadbeef4c8b96d4fffdfae29258d94a22ae4ad1fe36139d47288b8960d9958d
  annotations:
    kbld.carvel.dev/id: gcr.io/k8slt/k8s-simple-app2

JFrog Artifactory is configured with with this Virtual Repository:

Repository Key: jfrog.corp.com/k8slt Selected repositories:
- index.docker.io/dkalinin/k8s-simple-app
- gcr.io/k8slt/k8s-simple-app2

$ kbld rewrite -f images.yml --to-repo jfrog.corp.com/k8slt
apiVersion: imgpkg.carvel.dev/v1alpha1
kind: ImagesLock
images:
- image: jfrog.corp.com/k8slt@sha256:4c8b96d4fffdfae29258d94a22ae4ad1fe36139d47288b8960d9958d1e63a9d0
  annotations:
    kbld.carvel.dev/id: docker.io/dkalinin/k8s-simple-app
- image: jfrog.corp.com/k8slt@sha256:deadbeef4c8b96d4fffdfae29258d94a22ae4ad1fe36139d47288b8960d9958d0
  annotations:
    kbld.carvel.dev/id: gcr.io/k8slt/k8s-simple-app2

`imgpkg` already handles this... for bundles

Since imgpkg pull will detect whether or not all of the images were pulled from the same repository, given the Artifactory configuration, above, it will rewrite the ImagesLock as described above. Assuming the user is using bundles.

It does the thing that's requested.

Should `kbld`?

So, the question is:

Is it in kbld's scope to support this scenario, independently (i.e. not relying on imgpkg copy/pull for the relocation)?

pivotaljohn commented 3 years ago

Part of the answer is: if we value the convenience of such a feature over the compactness of the tool suite.

If a user has opted-out of using bundles, the request of this issue is the one remaining gap in functionality that would allow them to complete their workflow with just kbld. For convenience's sake, yes, we'd include this in kbld.

However, imgpkg is technically the tool that owns the task of "rewrite the ImagesLock". Duplicating this functionality in kbld may blur the lines between these two tools even more.

A strict regard of each tool's responsibilities: kbld's responsibilities are:

to resolve image references to digests
if necessary (and configured to do so), build the image

imgpkg's responsibilities are:

to simplify performing operations over a collection of images (along with attendant app configuration), including relocation.
as a side-effect, keep up-to-date the "bill of materials" (i.e. ImagesLock)

Including this "rewrite the lock file" functionality in imgpkg amounts to making that second responsibility of imgpkg an explicit action. That is: to also update the "bill of materials" to reflect that images have been relocated via some other mechanism.

However, today, imgpkg does not perform operations on an ImageLock file, directly; users are targeting bundles. Adding this functionality broadens the tool's API (and might make it more confusing to use).

What next?

We suspect it would help us tip the scales if we learned more about the lived experience of those who would want such functionality in kbld.
- are they adverse to imgpkg at all? does that represent undue complexity in their workflow?
- what are the important factors in their world?
Explore the actual impact on imgpkg's UI if we did include operating on ImagesLock files.

cc: @StevenLocke

pivotaljohn commented 3 years ago

A need like this was mentioned in #carvel.

pivotaljohn commented 3 years ago

In a side conversation about that request ☝🏼 , @cppforlife pointed out:

i would argue that this should be in scope of kbld since it's more about rewriting hostnames that kbld knows about.
keep in mind that ImagesLock is just one source of image overrides.
this part (i.e. the rewriting of hostname) is resolving though. if user says: trust me this image is actually here, then kbld should make the change. (this is an extension of image overrides feature which is unrelated to relocation)

pivotaljohn commented 2 years ago

We're seeing a growing number of users doing some DNS gymnastics to produce an imgpkg lock file whose references appear to point to a production registry. One such example here: https://github.com/vmware-labs/service-bindings/blob/main/.github/workflows/ci.yaml#L28-L99

There are other reasons for such a setup: for example being able to produce images and a bundle using a short-lived, air-gapped registry means it can more likely be insecure (a risk one needs to assess).

Regardless, this feature would effectively be a supported way of doing manual relocation.

This is different than a some pattern-matching "override" feature in that we'd want the origins (formerly "metas") to be preserved.

rickhlx commented 9 months ago

We're also interested in using kbld to override the image registry only. We use artifactory as a pull-through cache, currently we need to maintain a list of images which needs to match images defined in k8s manifests. Having the ability to override the registry only would allow us to cover all cases where an image's registry should be overridden.

carvel-dev / kbld