Open ArcticXWolf opened 1 year ago
Just to clarify our usecase:
We need to build an image which uses python and one of the pip packages is inside a private pipy-index. So we need to include the credentials for this during the pip install
step of the build. However copying or setting the credentials via envs will leak them in the image layer history. Thus we want to use the intended mechanism of buildkit secrets.
So we need devspace to:
All three together are currently not possible, because the upload to local registry is bound to its own set of builders (docker or remote buildkit).
Hello! Thanks for submitting an issue. This is something we will work on enabling.
also ran into this and spent a lot of time against it before I realized what was happening
What happened?
I want to add specific buildKit options (
--secret ...
) to the image build process, but also deploy to the local registry. However, enabling the localRegistry weirdly overwrites the complete buildKit stanza in the devspace.yaml and no args are being applied.What did you expect to happen instead?
That devspace applies
images.[imagename].buildKit.args
even when enabling localRegistry.How can we reproduce the bug? (as minimally and precisely as possible)
Create the following three files in a new folder:
Dockerfile
secretfile
devspace.yaml
Output on
devspace build
:Local Environment:
Anything else we need to know?
I have already debugged the issue in devspace code. The reason is that when you enable the localRegistry, then unintuitively a different docker builder is being used (localregistry vs buildkit). The localregistry builder also uses buildkit (for online builds) or docker (for local builds), but does not reuse the code from the real buildkit or docker builder.
Thus the localregistry builder does not have any access to the
images.[imagename].buildKit.args
config parameters and cannot apply those to the build. This is also the same when settinglocalRegistry.localbuild=true
.The problem is: We need a local build with buildkit (which works when disabling localRegistry) AND need to push the image to the local registry. But currently you cannot use both together.
My proposal (but since I do not know the devspace code well you might have different opinions/reasons) would be to isolate the build and push parts of the
devspace build
pipeline, so any builder can be used with localRegistry. This also makes localRegistry more DRY, since you do not implement the full docker build pipeline there AND in the docker/buildKit builder.Also: is there a workaround to already use this now? (creating a new pipeline or else)