PESTO provides a set of tools to ease the process of packaging a Python algorithm as a processing web service into a docker image. The deployment of a web service becomes now as easy as filling few configuration files.
This pull request is proposed to resolve issue #9.
Some element for the context: we use PESTO in combination with a pip registry on a Artifactory instance. And the current implementation of the PIP_EXTRA_INDEX_URL is leaking any credentials from this URL in the final image.
The main modifications are:
A new mode to configure the pip command run when building a service. In addition to the environment variable PIP_EXTRA_INDEX_URL that was already sent to the service Dockerfile, the variable PIP_CONFIG_FILE is now detected and the file pointed is exposed in the service container during pesto build. This file can be used to fully configure pip (index-url, extra-index-url, path to certificates, ...).
Support for the BuildKit mode. This mode introduce a more secure way to pass secrets and credentials during a docker build, without leaking them in the final image. When this mode is enabled ( DOCKER_BUILDKIT=1 ), the content of PIP_EXTRA_INDEX_URL and PIP_CONFIG_FILE is exposed during docker build using --secret option:
the extra-index-url is exported at the beginning of each line with a pip install
the config file is mounted on /etc/pip.conf before each pip install
When BuildKit is not enabled the credentials are passed:
with a ARG command for the extra-index-url
by a COPY to /etc/pip.conf for the config file
Two tests have been added to verify the generation of the service Dockerfile. However, no test is added to validate a pesto build command.
Regarding the --network option, a small modification is made. Now, giving --network '' to pesto will remove any --network option in the actual docker build command line. This is helpful to let docker use its default networking mode.
This pull request is proposed to resolve issue #9.
Some element for the context: we use PESTO in combination with a pip registry on a Artifactory instance. And the current implementation of the
PIP_EXTRA_INDEX_URLis leaking any credentials from this URL in the final image.The main modifications are:
pipcommand run when building a service. In addition to the environment variablePIP_EXTRA_INDEX_URLthat was already sent to the service Dockerfile, the variablePIP_CONFIG_FILEis now detected and the file pointed is exposed in the service container duringpesto build. This file can be used to fully configure pip (index-url, extra-index-url, path to certificates, ...).DOCKER_BUILDKIT=1), the content of PIP_EXTRA_INDEX_URL and PIP_CONFIG_FILE is exposed during docker build using--secretoption:pip install/etc/pip.confbefore eachpip installARGcommand for the extra-index-urlCOPYto/etc/pip.conffor the config fileTwo tests have been added to verify the generation of the service Dockerfile. However, no test is added to validate a
pesto buildcommand.Regarding the
--networkoption, a small modification is made. Now, giving--network ''to pesto will remove any--networkoption in the actualdocker buildcommand line. This is helpful to let docker use its default networking mode.Feel free to comment.