RFC: Support Dockerfiles

[sclevine]

Readable

This proposal is part of a larger initiative to reduce complexity originally outlined in https://github.com/buildpacks/rfcs/pull/167

[cmoulliard]

Will it be possible to run the build using a root user as proposed here: https://github.com/buildpacks/rfcs/blob/root-buildpacks/text/0000-root-buildpacks.md ?

[sclevine]

Will it be possible to run the build using a root user as proposed here: https://github.com/buildpacks/rfcs/blob/root-buildpacks/text/0000-root-buildpacks.md ?

Only the Dockerfiles can run as root. This proposal does not allow buildpacks to run as root.

[cmoulliard]

Only the Dockerfiles can run as root. This proposal does not allow buildpacks to run as root.

So you confirm that the limitation (see hereafter) will not be fixed by this RFC ? @sclevine

_"The execution of a privileged command such as "yum install", as declared within a "build bash script", will fail due to the following error This command has to be run with superuser privileges (under the root user on most systems). The error is due to the fact that the container created by "pack build" is using the UID/GUID of the build image and never the root user"_

If the answer is Yes, how will it be then possible to support such a use case (see example hereafter) ?

Example proposed within the RFC69:

[buildpack]
id = "example/cacerts"
privileged = true

[[stacks]]
id = "io.buildpacks.stacks.bionic"

...

bin/build

#!/usr/bin/env bash

# filter to the cert
for file in $(cat $3 | yj -t | jq -r ".entries | .[] | select(.name==\"cacert\") | .metadata | .file"); do
 cert="$(cat $3 | yj -t | jq -r ".entries | .[] | select(.name==\"cacert\") | .metadata | select(.file==\"$file\") | .content")"
 echo $cert > /usr/share/ca-certificates/$file
 chmod 644 /usr/share/ca-certificates/$file
done

update-ca-certificates

Using the RFC69 ? Is it approved and implemented ?

@sclevine @jkutner

[sclevine]

RFC0069 creates a special type of buildpack that can run as root. This proposal would revert RFC0069 and instead require Dockerfiles to run commands as root.

[cmoulliard]

instead require Dockerfiles to run commands as root.

Can you please describe which system and which dockerfile (run or build) is doing what exactly as this is not clear ?

Example : If I use as platform pack, will it be able with this RFC to create a container as done by the command pack build using the build.dockerfile and launch the container using as user root ...

[sclevine]

As specified in the RFC, for the Dockerfiles adjacent to project.toml:

A run.Dockerfile is applied to the selected runtime base image after the detection phase. A build.Dockerfile is applied to the build-time base image before the detection phase.

So both Dockerfiles are applied automatically during pack build.

And for creating/extending stack images in advance:

The same Dockerfiles may be used to create new base images or modify existing base images outside of the app build process.

[cmoulliard]

So both Dockerfiles are applied automatically during pack build.

This is not yet clear to me. If a Build.Dockerfile exists within the app's project, will the image be built during the execution of the command pack build. Will next the container started using the root user and not as this is the case today using the CNB_UID - https://github.com/buildpacks/pack/blob/main/internal/build/lifecycle_execution.go#L210

Question: What will happen if no build.Dockerfile is defined with the app's project when we execute the command pack build ?

[dmikusa]

My understandings based on the last working group meeting as well as the RFC text.

So both Dockerfiles are applied automatically during pack build.

This is not yet clear to me. If a Build.Dockerfile exists within the app's project, will the image be built during the execution of the command pack build.

Yes. Having these files alongside an app allows the stack to be modified dynamically just for that particular application. So as pack build would run, it would apply these. build.Dockerfile such that it's done before the buildpacks run and to modify the build environment and run.Dockerfile to modify the run image for the launch environment.

Will next the container started using the root user and not as this is the case today using the CNB_UID - https://github.com/buildpacks/pack/blob/main/internal/build/lifecycle_execution.go#L210

Not sure what you mean exactly, but buildpacks will continue to run as normal. No changes to the user they run as. Only the commands that are placed into the *.Dockerfile files will be run as root. In short, you can literally do anything in the *.Dockerfiles , you have full and total access.

Question: What will happen if no build.Dockerfile is defined with the app's project when we execute the command pack build ?

Nothing. The buildpacks would use the default stack & run as normal.

[lance]

Yes. Having these files alongside an app allows the stack to be modified dynamically just for that particular application. So as pack build would run, it would apply these. build.Dockerfile such that it's done before the buildpacks run and to modify the build environment and run.Dockerfile to modify the run image for the launch environment.

I like this, but the big drawback I see compared to stackpacks is the fact that this is entirely user-driven, if I understand correctly. It's up to the application developer to provide these build.Dockerfile and run.Dockerfile files. So, while that does allow for more flexibility than the current model, it puts the onus on the application developer to understand how all of this works. And at this point, I fear they'll just say, "why bother with buildpacks if I have to write a Dockerfile anyway?".

It would be nice if builder providers/vendors could modify build images during detection so that system level dependencies discovered during the detect phase could be installed on build and run images for the build stage.

[sclevine]

It would be nice if builder providers/vendors could modify build images during detection so that system level dependencies discovered during the detect phase could be installed on build and run images for the build stage.

Considering the complexity and tradeoffs discussed in RFC0069 (which proposed to provide that functionality), I suspect that the buildpack API (consisting of isolated, order-independent, safe-to-swap layers) is not very helpful for solving this problem compared to the Dockerfile model. I think we should implement easy stack customizations and app-specified Dockerfiles first, and revisit buildpack-driven OS package installation in a future RFC. Implementing a simple, Dockerfile-based escape-hatch first will unblock users that need to install additional OS packages and give us more time rethink the buildpack-driven functionality.

[cmoulliard]

Nothing. The buildpacks would use the default stack & run as normal.

As the RFC172 is to remove the stack, how will then the image defined within the builder ?

Only the commands that are placed into the *.Dockerfile files will be run as root

Then, in this case it will not be possible to run a privileged command (yum install, ...) during buildpack build execution as the container will be created using the CNB_UID. From this point of view, this is a regression as it will not be possible to do something like https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md#example-ca-certificate-buildpack

[sclevine]

As the RFC172 is to remove the stack, how will then the image defined within the builder ?

RFC#172 doesn't remove the base images themselves -- it just removes the project-specific terms "stack" and "mixin" in favor of existing concepts in the ecosystem (base images, GOOS, GOARCH, etc.)

Then, in this case it will not be possible to run a privileged command (yum install, ...) during buildpack build execution as the container will be created using the CNB_UID. From this point of view, this is a regression as it will not be possible to do something like https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md#example-ca-certificate-buildpack

Correct, except RFC0069 is not implemented yet and will likely be reverted due to the complexity it adds. So it's not a regression for users of the project.

[sclevine]

Addressed feedback re: operator control. Now app-specified Dockerfiles are a special case of builder-specified Dockerfiles.

CC: @cmoulliard @samj1912 @jkutner @ekcasey

[sclevine]

Addressed additional feedback, ready for another round of review 🙂

[sclevine]

Idea I'd like feedback on:

What if we run buildpack detection first, and then pass the detected buildpack IDs to executable hooks as additional parameters? This would allow users to create small base images that support many buildpacks, with rebasing unavailable when buildpacks that require additional OS packages are used (unless a more specialized builder is created). Hooks would remain relatively decoupled from the buildpack API, and buildpacks authors still couldn't create buildpacks that users could never use with rebase.

This could be used to implement #174 without requiring PURLs (related comment: https://github.com/buildpacks/rfcs/pull/172#issuecomment-883800124). Instead of #174, we could introduce a /cnb/hooks.d/<name>.ref.out hook that outputs a run image tag/digest given a set of detected buildpack IDs. This would also allow apps to select custom run images, for builders with an app-specified-run-image hook.

[dmikusa]

What if we run buildpack detection first, and then pass the detected buildpack IDs to executable hooks as additional parameters?

This addresses a concern of mine. If I want to have the hook emit something that's specific for a buildpack, I would essentially need to have detect logic in both the hook and the buildpack. If the hook is provided the output of detect, it would then know what buildpacks were selected and could apply conditional logic more easily.

The obvious downside is that you cannot then use the hook/Docker file to satisfy a requirement of the detect script, like if it needs a special tool/cmd to operate. IMHO, detect is almost always a very simple process so this is probably not necessary but I suppose you could add a third state if this is a concern. You have run and build now, you could add a third like detect. The generated Dockerfile would then be applied prior to each corresponding buildpack stage.

This could be used to implement #174 without requiring PURLs (related comment: #172 (comment)). Instead of #174, we could introduce a /cnb/hooks.d/<name>.ref.out hook that outputs a run image tag/digest given a set of detected buildpack IDs. This would also allow apps to select custom run images, for builders with an app-specified-run-image hook.

👍

[sclevine]

Slight change to this idea, suggested by @ekcasey:

Instead of passing buildpack IDs to hooks, hooks could (optionally) be associated with buildpacks on the builder. This would reduce hook complexity, and make hooks more reusable. Buildpack IDs for a hook could be specified in builder.toml.

The buildpacks could be specified with the hook path:

/etc/hooks.d/com.example.buildpack/myhook.Dockerfile.out

But that would get messy (or require symlinks) for hooks that apply to multiple buildpacks.

Alternatively, we could use a list of IDs:

/etc/hooks.d/myhook.Dockerfile.out
/etc/hooks.d/myhook.Dockerfile.out.buildpacks # newline separated

(edited, thanks @dmikusa-pivotal)

[dmikusa]

Slight change to this idea, suggested by @ekcasey:

Instead of passing buildpack IDs to hooks, hooks could (optionally) be associated with buildpacks on the builder. This would reduce hook complexity, and make hooks more reusable. Buildpack IDs for a hook could be specified in builder.toml.

Would the hook still only execute once?

It seems like, with the previous idea, the hook would run once and be given a list of the buildpack ids that passed detection. The hook could execute once and use the full list to determine what it does.

It's not clear what would happen with this new format. If my hook is registered for two or more buildpacks would the hook run once and be given all the buildpacks ids? or would it run once per buildpack? My preference would be for running once.

Alternatively, we could use a list of IDs:

/etc/hooks.d/com.example.buildpack/myhook.Dockerfile.out
/etc/hooks.d/com.example.buildpack/myhook.Dockerfile.out.buildpacks # newline separated

Should this be:

/etc/hooks.d/myhook.Dockerfile.out
/etc/hooks.d/myhook.Dockerfile.out.buildpacks # newline separated

since the buildpacks would be in the file? or would it still be necessary to have them encoded as part of the path?

[sclevine]

Would the hook still only execute once?

No change, they still only run once. I like the .buildpacks option better, because it makes this more clear.

Should this be [...] since the buildpacks would be in the file

Yes, definitely -- that's a mistake. Will edit the original comment.

[cmoulliard]

What if we run buildpack detection first, and then pass the detected buildpack IDs to executable hooks as additional parameters? This would allow users to create small base images that support many buildpacks, with rebasing unavailable when buildpacks that require additional OS packages are used (unless a more specialized builder is created). Hooks would remain relatively decoupled from the buildpack API, and buildpacks authors still couldn't create buildpacks that users could never use with rebase.

In order to express my idea, I will provide you some examples which could help us to better understand what we should address as I dont understand this idea:

1) Paketo - no root needed If a user runs a build using a Paketo builder, then everything is working fine as the installation of the missing tools/libs is executed within the home folder of the user and no privileged root command should be executed

detect phase -> requires a tool/lib for version x.y.z --> build will install it using curl using METADATA provided within the buildpack toml definition. No dockerfiles will be needed

2) root needed A user using a RHEL/Centos/Fedora OS is using a builder where some the of the buildpacks require to execute some root privileged commands as the tool installing the package needs it (yum or microdnf or dnf, ...). In this case the hook idea should help us to resolve the existing limitation as the lifecycle creator to execute the build step uses the UID/GUI part of the build image and not the root privileged user.

detect phase --> requires a tool/lib for version x.y.z --> hook is executed (= build dockerfiles) using the METADATA provided --> additional layer is pushed top of the build image --> build (= doing by example a mvn compile will be able to take place as the maven tool has been installed).

Remark: Of course, if JDK or another require are needed, then the hook(s) able to install JDK like additional requires should be then executed.

• How then will it be possible to achieve this goal ?
• How finally should the hook be declared within the builder.toml file as it appears that there is a relation between the job done by the buildpack and requirements ?

[sclevine]

@cmoulliard

detect phase --> requires a tool/lib for version x.y.z --> hook is executed (= build dockerfiles) using the METADATA provided --> additional layer is pushed top of the build image --> build (= doing by example a mvn compile will be able to take place as the maven tool has been installed). Remark: Of course, if JDK or another require are needed, then the hook(s) able to install JDK like additional requires should be then executed.

This is close to the functionality currently proposed, but not exactly the same. Integrating hooks (or any commands that modify the base image) into the buildpack API would create extra complexity for buildpack authors and platforms (see: Stackpacks). I outlined an alternative that would work like this, but I'm not convinced that it's the right path forward -- it adds complexity, couples hooks to buildpacks, and prevents users from creating pre-built base images to improve performance.

Instead, as proposed now, hooks make it easy to automate creating an ephemeral language-specific and/or app-specific builder right before the buildpack build process.

How then will it be possible to achieve this goal ?

If your hook needs to install specific version of the JDK based on the contents of the app, it can determine this by examining the app directory directly. This will ensure that the correct version of the JDK is present at buildpack build-time, so buildpacks that require the JDK won't need to add a require to the build plan.

How finally should the hook be declared within the builder.toml file as it appears that there is a relation between the job done by the buildpack and requirements ?

The hook wouldn't need to be buildpack-specific in this case:

[[hooks]]
name = "jdk-hook"
path = "./my-jdk-hook" # determines JDK version from app dir, outputs Dockerfile to install that version

[cmoulliard]

Integrating hooks (or any commands that modify the base image) into the buildpack API would

This is not at all what I'm proposing. I just described how the process executed by the lifecycle could take place when the lifecycle creator will support to execute a hook between the phase detect and phase build. Is my assertion correct ?

[frenzymadness]

What if we run buildpack detection first, and then pass the detected buildpack IDs to executable hooks as additional parameters?

This addresses a concern of mine. If I want to have the hook emit something that's specific for a buildpack, I would essentially need to have detect logic in both the hook and the buildpack. If the hook is provided the output of detect, it would then know what buildpacks were selected and could apply conditional logic more easily.

I could not agree more :+1:

The way I see buildpacks as a replacement for s2i images (source-to-image) is that we would have one universal builder with some universal hooks and language-specific buildpacks. Then, if an application needs for example Python and NodeJS, both buildpacks are detected, and then the control is hand over to hooks. Hooks install RPM packages (root needed, that's the main problem we have with the current implementation), and hooks prepare an environment on OS level. Then buildpacks install dependencies from PyPI and NPM and we are done.

I like this idea more than the Dynamic runtime base image selection because we'd like to have only one universal image for all supported language stacks and then use hooks and buildpacks to adjust it according to the needs of an application.

[sclevine]

@ekcasey @frenzymadness @samj1912 @cmoulliard @jkutner @dmikusa-pivotal I think 4f73fe4721644103b53e098a68b2bc60240c0bf1 addresses all major feedback (at the expense of more complexity).

[buildpack-bot]

Maintainers,

As you review this RFC please queue up issues to be created using the following commands:

/queue-issue <repo> "<title>" [labels]...
/unqueue-issue <uid>

Issues

• ✅ f58b4c - buildpacks/lifecycle#890 "Detector should run hooks"
• ✅ 396cba - buildpacks/lifecycle#891 "Lifecycle should run hooks' bin/build"
• ✅ 4d5bdc - buildpacks/lifecycle#892 "Lifecycle apply Dockerfiles to build image"
• ✅ 7d2869 - buildpacks/lifecycle#893 "Lifecycle apply Dockerfiles to run image"
• ✅ df16be - buildpacks/lifecycle#894 "Lifecycle run /cnb/image/genpkgs on the extended run image"

[frenzymadness]

I think 4f73fe4 addresses all major feedback (at the expense of more complexity).

The commit you mentioned is 28 days old. I guess the correct one is the last one in this PR: https://github.com/buildpacks/rfcs/pull/173/commits/1e813b987ad6734169ed7c74ac60f51f943dd74f

[sambhav]

Can we also add https://www.gitmemory.com/issue/moby/buildkit/1091/515716668 as an example. I am more biased towards llb json than dockerfiles since we can control more things and it is easier to parse/create.

[sclevine]

As discussed, I'm adding support for Dockerfile build-args instead of LLB JSON (at least for this RFC).

[sclevine]

I believe all outstanding feedback has been addressed. Looking for additional feedback or approval @buildpacks/core-team

[sclevine]

All remaining feedback addressed again, ready for review.

[hone]

@hone to review.

[hone]

@natalieparellano will kick off a task force with interested parties to figure out what the spec changes should look like for this RFC.

[hone]

FCP blocking on changes to genpkg with suggestions from @samj1912.

[sclevine]

Also would like to block on a decision about which implementations are integrated into the lifecycle. (E.g., docker/podman vs. buildah vs. kaniko)

The default implementation must be suitable for multi-tenant environments and run on vanilla K8s.

[hone]

As part of this RFC, we'll also be review buildpacks/spec#276 that incorporates the spec changes from this RFC. @natalieparellano will lead a flow of the current PoC during this or next week's OH.

[hone]

Please look at buildpacks/spec#298 for hammering out further details.

[ekcasey]

@sclevine and/or @natalieparellano, can we get an update on the status of this given the progress made on the POC? What if any are the outstanding questions?

[natalieparellano]

Apologies, the context for this is now spread across multiple spec and lifecycle PRs.

The latest is that I've proposed a phased implementation of this feature and have submitted spec PRs (1, 2) covering the first phase. I am hoping that we could ship the first phase in experimental apis given the complexity involved.

If we're happy with this plan, support for phase 1 in the lifecycle could ship in the next minor release.

[natalieparellano]

The RFC text in its current state reflects the implementation proposed in https://github.com/buildpacks/spec/pull/307 and https://github.com/buildpacks/spec/pull/308. Could everyone please re-read and add your review comments and/or approval, so that this could (hopefully) be merged? cc @buildpacks/toc @buildpacks/platform-maintainers @buildpacks/implementation-maintainers @buildpacks/buildpack-authors-tooling-maintainers @buildpacks/distribution-maintainers

[natalieparellano]

I believe this RFC now has all the required approvals for FCP. Given @jromero is out, I'm going to put the closing date as Wednesday, 7/27.

[natalieparellano]

This is well past FCP close date, could it be merged? It's worth noting, due to the complexity involved and the phased implementation, there are still some ambiguities outstanding (in particular around run image extension and how rebase is going to work). But I think we have clearly decided to proceed here. If necessary I am happy to "amend" the RFC with information learned from the implementation.

[natalieparellano]

^^ cc @buildpacks/team-leads @hone (as the assignee)

[natalieparellano]

For anyone interested in following the implementation of this RFC, the status can be found here: https://github.com/buildpacks/rfcs/issues/224