bentoml / yatai-image-builder

🐳 Build OCI images for Bentos in k8s
14 stars 9 forks source link

Passing Custom Parameters to the Image Building Job Container #68

Closed pepesi closed 7 months ago

pepesi commented 7 months ago

My base image is in an internal Docker registry without TLS. When the image builder (Kaniko) operates, it defaults to pulling images via HTTPS, which leads to a failure in building the image. I noticed that the parameters for Kaniko are hardcoded in the Bentorequest's controller, and it seems there's no option to add the --insecure-pull or --insecure-registry parameters.

Currently, I have modified the Kaniko image, using something like /kaniko/origin-executor --insecure-registry=MYREGISTRY.REPO $@ to work around this issue.

Is this by design? If not, is it possible to add this parameter through an environment variable, similar to DOCKER_INSECURE_REGISTRY?

yetone commented 7 months ago

β€Œβ€Œβ€Œβ€Œβ€Œβ€ŒHello, we already possess the pertinent installation parameter DOCKER_REGISTRY_SECURE. For more details, please refer to the installation documentation:

https://docs.yatai.io/en/latest/installation/yatai_image_builder.html#prepare-container-registry

pepesi commented 7 months ago

I was following the instructions from the document you posted, but I specified a docker.base_image in the bentomlfile, which is an image from my private.repo. When the image-builder tries to pull this image, it encounters an error, similar to "requested HTTPS but received an HTTP response".

After looking at the command-line help for kaniko, it mentions the following parameter:

      --insecure                                  Push to insecure registry using plain HTTP
      --insecure-pull                             Pull from insecure registry using plain HTTP
      --insecure-registry multi-arg type          Insecure registry using plain HTTP to push and pull. Set it repeatedly for multiple registries.

According to its description, the --insecure parameter only controls the push operation, not the pull operation. To control the pull operation, you must use the --insecure-pull or --insecure-registry parameter.

I noticed that the current controller's code has hardcoded kaniko parameters. What I would like is the ability to pass the --insecure-pull or --insecure-registry parameter from somewhere. Considering the need to better support other custom behaviors, perhaps allowing the passing of more key-value formatted parameters would be even better.

pepesi commented 7 months ago

Sorry for my oversight, there actually is an implementation for this "extra parameter". It might not have been mentioned in the documentation.

It is possible to achieve this through the builder_args in the configMap yatai-image-builder-config.

https://github.com/bentoml/yatai-image-builder/blob/d4ab40b368867ffee62ae100cb929a175c808e99/controllers/resources/bentorequest_controller.go#L2452-L2460