search

shipwright-io / build

Shipwright - a framework for building container images on Kubernetes
https://shipwright.io
Apache License 2.0
672 stars 113 forks source link

(Cluster)BuildStrategy CR about buildpacks are outdated #1280

Closed cmoulliard closed 1 year ago

cmoulliard commented 1 year ago

Issue

The (Cluster)BuildStrategy CR about buildpacks is outdated

According to @BartWeller and as per https://github.com/buildpacks/spec/blob/main/platform.md#build A platform MUST execute these phases either by invoking the following phase-specific lifecycle binaries in order:

/cnb/lifecycle/analyzer
/cnb/lifecycle/detector
/cnb/lifecycle/restorer
/cnb/lifecycle/builder
/cnb/lifecycle/exporter

The version of the platform should also bumped to 0.11

Important: We should certainly find a way to align what Shipwright uses vs Tekton vs Openshift Pipelines as different tasks already exist:

Questions:

@otaviof

BarDweller commented 1 year ago

Adding comments here for future reference...

Anything that performs a build using buildpacks, counts as an implementation of the Buildpack platform spec, and should follow that spec. Only the most recent spec releases (0.10 onwards) include support for extensions.. Extensions are required for the upcoming Paketo UBI stacks, as the Extensions manage the dependency installs via dnf.

The current yaml implements the platform spec as per release 0.4, which is why the order of the phases disagrees with the current spec (the order of analyzer & detector inverted at spec 0.7).

Here's 10k ft high view of what that means from a platform implementation perspective.

Performing a build with buildpacks essentially boils down to 'spinning up a container based on the builder image, and invoking the lifecycle binaries inside with the appropriate volume mounts and arguments'. The lifecycle binaries then handle the coordination of detecting buildpacks & performing the build with them.

This still holds true for specs at 0.10 and beyond, although there are some differences to note.

In the past, you've had the choice of using the creator lifecycle binary, or invoking each of the phases 'analyze', 'detect', 'restore', 'build', 'export' sequentially in new containers, each based from the builder image. The creator binary does not support extensions, and so platforms implementing 0.10 onwards must use separate phase executions for builder images that include extensions. Additionally, when performing a phased build with extensions, you do not drive the build lifecycle binary, but instead drive the extender binary.

If you want to see the sequence of the phases being invoked, with mounts, and arguments, you can have a peek at https://github.com/BarDweller/buildpack-platform-bash/blob/main/bash_build.sh which assumes you are building with an image using extensions, and does the separate phase invocations. Afaik, no tekton pipeline exists today that demonstrates drives the extender path.

It's probably also worth saying that as a platform, you can check if the builder image contains extensions by seeing if io.buildpacks.extension.layers and io.buildpacks.buildpack.order-extensions are populated as labels.

I think tekton pipelines support setting variables and making tasks dependent on variable content, so it should be possible to determine if the image has those labels set, and then use multi-phase execution where the builder is using extensions.

Also, from the bash platform, pay attention to the userid/groupid the containers are invoked as, with most of them being CNB_USER_ID and CNB_GROUP_ID from the builder image environment, except the extender phase that runs as 0:0 with CNB_USER_ID and CNB_GROUP_ID passed via -uid and -gid

cmoulliard commented 7 months ago

I tested successfully this updated strategy https://github.com/redhat-buildpacks/testing/blob/a8cfe9664ac96652bd0e136ee4b708706c08e08a/k8s/shipwright/unsecured/clusterbuildstrategy.yml with the UBI Builder image (java, nodejs) top of a quarkus project on a kind cluster + tekton (v0.56) and shipwright 0.11

Job result: https://github.com/redhat-buildpacks/testing/actions/runs/8658401429/job/23742216328?pr=51

@adambkaplan @BarDweller

cmoulliard commented 7 months ago

Some todos:

@adambkaplan