dashaun / java-native-builder-multiarch

A single `Cloud Native Buildpack` that can build native images for AMD64 and ARM64
Apache License 2.0
5 stars 1 forks source link

other buildpacks inheritance and multiarchitecture compatibility #3

Open cforce opened 1 year ago

cforce commented 1 year ago

Are in general buildpacks inherited from spring-boot-maven-plugin and "detect" by default as before? That would also mean if another dep "gcr.io/paketo-buildpacks/datadog" gets added to one being already registered as default like "gcr.io/paketo-buildpacks/java-native-image) it will be active as well. So i can expect takeover of the datadog support implicitly as soon as it gets part of the native-image buildpack with your builder as well? See https://github.com/paketo-buildpacks/datadog/pull/149 How to guarantee that those buildpacks available work well with your custom builder ?

dashaun commented 1 year ago

Just adding the java-native-image buildpack is not enough to make the datadog buildpack work with native images.

If the datadog buildpack isn't providing an ARM64 version, more work would need to be done.

I'm not opposed to making a multi-architecture buildpack for datadog though.

Am I understanding your issue properly?

cforce commented 1 year ago

The datadog builpack ist just adding a jar to the native image composite buildpack, so I assume it‘s out of the box multiarch. The jar is used as Java agent for instrumentation and the Graal support in DD buildpack is at the moment being pushed upstream. I really would like to native image musl compilation with datadog instrumentation on arm64 (edge device). Question is if you will inherit the child’s buildpack from native image composite as well

dashaun commented 1 year ago

My understanding is that native images don't support java agents, because there is no JVM to attach to.

Also, I've not been able to get musl compilations working with the buildpack yet. It's on my list, but further down the list, after the ARM64 support upstreamed to Paketo.

If the datadog is using the exposed actuators, it may work properly. Can you share a link to the datadog buildpack that you are using?

PerfectSlayer commented 1 year ago

Hi @dashaun, I am Bruce from @Datadog

My understanding is that native images don't support java agents, because there is no JVM to attach to.

GraalVM native-image does support java agent at build time, in a limited way but enough for datadog tracer agent to do its intrumentation.

I recently added the support for native build to the paketo datadog buildpack in this PR https://github.com/paketo-buildpacks/datadog/pull/149 if you would have a look to.

cforce commented 1 year ago

Hi Dashaun,

Regarding the question you asked earlier, Bruce mentioned that it should work as long as your buildpack inherits what he pushed upstream.

In addition, the musl compilation worked for me using the maven native profile settings, but I had to run my musl setup script before executing the Maven or Gradle build with a native profile. It might be helpful for you to add a "musl switch" based script execution to your builder image that applies the snippet in that case.

                    <plugin>
                        <groupId>org.graalvm.buildtools</groupId>
                        <artifactId>native-maven-plugin</artifactId>
                        <configuration>
                            <buildArgs>
                                <buildArg>--static</buildArg>
                                <buildArg>--libc=musl</buildArg>
                                <buildArg>-H:+ReportExceptionStackTraces</buildArg>
                            </buildArgs>
                            <metadataRepository>
                                <enabled>true</enabled>
                            </metadataRepository>
                        </configuration>
                        <executions>
                            <execution>
                                <id>add-reachability-metadata</id>
                                <goals>
                                    <goal>add-reachability-metadata</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>build-image</id>
                                <goals>
                                    <goal>compile-no-fork</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>                            

I am currently using a self-built approach to deliver a zip package for Spring Boot on an AWS Lambda custom x86 runtime compiled with musl. Now, I also want to use the AWS arm-based architecture stack and deliver to a serverless IoT device in the same way. I am also hoping that aws also will finally update their base image (eg used on lambda) to some kernel /image which provides current glibc versions. I don't have found out how i could base by cfg on e.g. on bionic even in your repo i found some bionic image hint.

Thank you for your contribution to arm so far. I was really happy to have found it, and I can soon start using it in buildpacks without all the obstacles on the build environment setup that my colleagues don't like either.

dashaun commented 1 year ago

@cforce @PerfectSlayer

That is great information!

Would either of you be available to "pair program" on this?

My day today (March 10) is pretty flexible. After today, I'll be traveling for 2 weeks, so I won't have much time to work in it.

dashaun commented 1 year ago

I read the other threads. As soon as your PR is merged, I'll create another buildpack, and we can test things out!

cforce commented 1 year ago

Hint: You can use Liberica as used by spring buildpacks but there is an issue with it when you want to create a musl based (instead of glibc) full static image- see https://github.com/oracle/graal/issues/4645. In that case rather use 22.3.r17-grl or 22.3.r19-grl related

PerfectSlayer commented 1 year ago

I read the other threads. As soon as your PR is merged, I'll create another buildpack, and we can test things out!

Hi 👋 A quick follow-up. As everything was merged on the packeto side, did you had the chance to create a buildpack and test it out?

cforce commented 1 year ago

@dashaun Any update on the buildpack ?

dashaun commented 1 year ago

@cforce @PerfectSlayer

I've pushed out a new version pulling all of the latest from upstream!

I don't have a way to validate the DataDog buildpack. Could one of you take it for a spin and report back?