Build images using the binary of my application

[sbose78]

As a developer, I want to build images using the binary of my application (e.g. app.jar) so that I can make use of artifacts produced in our existing CI process and stored on repositories like Nexus when building images.

Problem: Users have existing CI processes that often builds the binary of the application and store it in a repository. It's generally recommended to reuse the same app binary in the delivery phases after CI rather than rebuilding it during the image build phase. These binaries might be signed which prevent the customer from using build strategies that rebuild the binary of the application again.

Why is this important? To allow users to use Builds as the extension of their CI and reuse existing app binaries for building images via Builds.

[sbose78]

@zhangtbj thoughts?

[zhangtbj]

Hi @sbose78 ,

When you talking about the binary, do you mean the jar, war or tar of the source code, etc...?

Do you have this requirement before? I didn't see this type on the OpenShift console before.

I remember the Tekton can use it as a kind of resource, but I can not 100% make sure about that.

If Tekton doesn't support that, we should extract by ourselves at the beginning of the build.

At least, right now, we don't have this requirement. We can support that, but it is not on our 2Q delivery list now :)

[zhangtbj]

I remember wrong. The kaniko support to use tar file: https://github.com/GoogleContainerTools/kaniko#kaniko-build-contexts

[sbose78]

By Binary I mean, that the application code is already built into a jar/war in case of Java.

All that remains is that it needs to be put into a deployable image.

[zhangtbj]

Does that binary require any runtime support which inside of the deployable image?

I don't know if there is any build tool to support it for normal Dockerfile or CF(buildpacks) application.

But we tried the similar work before for Functions app, that for Java app, it comes from git repo first, but will be built as a jar file and inject it to a deployable image with the runtime.

I think jar/war is a special case just for Java. other language apps should base on tar file.

Is there any background/requirement detail behind that?

Or do you know what tool can support it?

[sbose78]

• A Jar/War should expect a runtime base image with Java
• A golang binary should expect a lean base image ( like a RHEL UBI image )
• A .Net binary should expect a runtime base image with the .Net framework binary in it.

[zhangtbj]

Is there a unify solution/binary to solve these three different languages as a whole?

Can RedHat s2i binary do that? Or other tool?

[sbose78]

Not really, the intention is to support a spec.output.base section where a user should be able to specify

• output base image
• location of the binary/binaries in the image in the previous stage
• ..

There would be runtime image specific optional buildSteps which would effectively execute a equivalent of a multi-stage build.

[sbose78]

Of course all of this is optional.

[zhangtbj]

For now, Dockerfile and buildpacks are high priority for us. I think if we have this kind of requirement. We can plan it after 2Q :)

[sbose78]

No worries, @otaviof is working on this, we'll discuss on a PR soon when it's up :)

[zhangtbj]

👍 :)

[rhuss]

For me, it's not so much important whether its an already compiled binary or the source code that it is compiled, it's about how the code (binary or source) is obtained. Typically in a CI/CD context you refer to the source via a Git or other SCM URL, which then is downloaded from there.

In contrast, in S2I a binary input (regardless of whether its an S2I, Docker or custom strategy) is uploaded from the local disc. Typically this is source (s2i/buildpack), or its a docker.tar container a Docker build context (e.g. for buildah/kaniko), but the important bit is that it's uploaded via an HTTP post.

For S2I the upload happens over a customer API endpoint ..../buildbinary to which that code is uploaded. This is not possible really for plain Kubernetes. There are several solutions to how this can be achieved here. One is to use an init-container to which you rsync/rcp the source before the application container does the build. I have some details for this mechanism with more details in some repo that I don't remember at the moment 😬 )

So I think you should rephrase your initial comment: It's really not about the binary aspect (i.e. some executable) but about a local upload (called "binary input" in S2I lingo)

[rhuss]

Ah, here's a POC with Buildah and Upload via an init-container: https://github.com/rhuss/buildah-poc

[rhuss]

The nice thing about using an init-container for the upload is that you have a nice synchronization point: The application container for the build will only start when the upload is finished (i.e. the init-container has been stopped). Of course, with Tekton you have probably many more possibilities to sync, this POC is just a poor men sync :)

[zhangtbj]

Hi @rhuss , Glad to meet you in another repo! And very cool PoC :)

And I am glad that we are discussing a real requirement. (I tagged it as release 0.2.x for the next release function.

Actually, I summarized almost ALL communities local upload/build support, and list in the doc, please also review, all: Local folder support for source-to-image https://docs.google.com/document/d/1Mxnbp-5a1I0ik_Jajb99AFNMnW0kCbvmO5KE1YaIvo0/edit?usp=sharing

kubectl cp is the easiest way but I think it is not the best way :), because:

• performance issue
• kubectl community may be refined or deprecated in future to use new style (I heard that from a CVE issue/doc)

So I prefer the rsync way or an upload service to help us to upload the image for build container/pod.

I also investigated and summarized the oc binary build in another doc:

I think it is a good way, and I think we can use the similar upload service/controller in this build repo to support the local file upload and pass it to the existing buildstrategy/build.

Also cc @sbose78

[sbose78]

Could we fork this into a separate issue so that this receives proper attention? :) We need two issues

• build from "jar" ( this one )
• build from local source ( new one )

[zhangtbj]

aha, yes, agree.

It mixes the binary build which make us confusing. :)

Let us discuss the local(binary) build in other issue: https://github.com/redhat-developer/build/issues/59

[adambkaplan]

Meant to follow up on this - I propose the following terminology so we don't confuse this capability from OCP Binary build sources:

1. "Remote artifact" - build source input from a file that is downloadable from a URL.
2. "Local artifact" - use the local content to upload a file or directory.

As @zhangtbj and @sbose78 suggested, we should treat these separately.

[qu1queee]

Closing this issue, local source code support will be prototyped while developing https://github.com/shipwright-io/cli and we can use https://github.com/shipwright-io/build/issues/59 for discussion. Remote artifacts already have a proposal in https://github.com/shipwright-io/build/blob/master/docs/proposals/remote-artifacts.md .