mirroring and builds v1

gabemontero commented 4 years ago

today openshift build v1 can build registries.conf files with associated auth and certs to ensure they are available to the build process and its invocation of buidah

ultimately buildv2 should have some form of this

it very well might make sense for DEVEX to help abstact out logic used for build v1 for use by buildv2 .... https://github.com/gabemontero/obu is a prototype of such an endeavor

Update by shoubhik

If https://github.com/shipwright-io/build/issues/537 takes care of this use case, we should document how to use the same to accomplish this use case

sbose78 commented 4 years ago

today openshift build v1 can build registries.conf files with associated auth and certs

Gabe, could you please provide details about the interaction points which lead to this?

gabemontero commented 4 years ago

today openshift build v1 can build registries.conf files with associated auth and certs

Gabe, could you please provide details about the interaction points which lead to this?

Sure @sbose78 ... so again I've captured the more novel things in obu ... see https://github.com/gabemontero/obu/blob/master/pkg/cmd/cli/cmd/mirrors.go

So you have

the obu mirror --docker-cfg-file which builds you the registries.conf file that buildah understands ... that translates to the --registries-conf and --registries-conf-dir options for buildah
obu mirror --ca-data captures the necessary certs for the mirrored registries
I didn't add any special logic, at least yet, for getting the auth ...
the secrets users create for their different registries can be associated with the pipeline service account for use by build v2 somewhat similar to what the operator does for the internal registry today ...
but in fact an aggregated auth file with all possible registries (i.e. the mirror registry and whatever registry the mirror is overriding) translating to one secret is best, as that seems to make things easier for buildah
since there are no well known locations for the mirror auth secrets, right now this is a manual step in openshift, so I don't see how the pipeline operator would do it automatically.

I have no idea what the level of support for mirroring is with the other build tools besides buildah ... certainly s2i doe not do anything with mirrors

gabemontero commented 3 years ago

According to @sbose78 we have api fields to supply/override the command line parameters supplied to the image building command (whether it is buildah, buildpacks, kaniko, etc.)

https://github.com/shipwright-io/build/master/pkg/apis/build/v1alpha1/Fbuild_types.go

But the reconciler/controller does not yet process whatever fields exist.

A possible MVP line that would be to complete the loop on this, and in this way, buildah users would have a path for supplying --registries-conf and --registries-config-dir, where mounting volumes/secrets/configsmaps would be the way to supply the actual contents.

Then, items

sbose78 commented 3 years ago

A more detailed rationale available here https://github.com/shipwright-io/build/blob/master/docs/proposals/buildstrategy.md#parameterization

gabemontero commented 3 years ago

Most likely we will open up a new feature around parametrization. And @sbose78 is going to bring this up in the community meeting.

qu1queee commented 3 years ago

My understanding is that an strategy can be extended to support any params we want to define for the tooling, e.g. buildah. Wondering if this issue is stale and can be closed? @gabemontero @sbose78

gabemontero commented 3 years ago

If the more generic parameterization feature is opened to complete the work @sbose78 noted still needs to be done, I'm good with closing this.

@sbose78 can you open that feature with the details you had in mind?

sbose78 commented 3 years ago

The discussion on spec.parameters is here https://github.com/shipwright-io/build/issues/184

sbose78 commented 3 years ago

https://github.com/shipwright-io/build/issues/537 Created a fresh issue.

I added a note in the description. If this is fixed by #537 , we should keep this open as a way to track the use case & document once #537 is in.

gabemontero commented 3 years ago

thanks @sbose78

closing this out

shipwright-io / build

mirroring and builds v1 #58