Fix runtime support

[metacosm]

Currently, only Spring Boot runtime works correctly. This is due to the fact that we've been experimenting with maven build from source on cluster. This requires a specific image for several reasons:

1. we wanted to have a first build that would be as fast as possible so we experimented with bundling an initial maven repository with the image. Of course, this only works if you know what theses dependencies should be. In the case of Spring Boot, we could pre-load all the dependencies defined in the Snowdrop BOM. We could certainly adopt a similar strategy for the other java runtimes but we just haven't done so yet.
2. we need to be able to perform several maven builds without having to re-download the dependencies all the time. There are several ways to accomplish this: either use docker layering to bundle the mostly unchanging dependencies or try to somehow persist the maven repository across pod restart / re-creation. We have experimented with the second option and as a consequence, this requires a specific image layout and persistent volume setup.
3. as the linking process results in the component pod being dropped and re-created, we also needed to be able to persist any binary that might have already been built so that the user wouldn't lose their app and needed to do another build after creating a link. Similarly, we also needed to be able to automatically restart any existing binary for the same reason. This also requires a specific run command and set up.

Theses reasons led to the creation of a hal-mvn-jdk image that's currently only being used by the Spring Boot runtime (since it bundles Spring Boot dependencies) and associated supervisor commands. As a result, the default s2i java image is not compatible anymore.

We have 2 options to solve this issue: either create runtime-specific images with appropriate dependencies being bundled or make the hal-mvn-jdk image more generic by removing all the Spring Boot dependencies so that it's lighter (for a faster initial image pull) but with a slightly slower initial build.

Generifying the hal-mvn-jdk image is probably the best solution in the short term (and maybe longer term as well, since there are differing opinions on the value of bundling initial dependencies in the runtime image).

[metacosm]

See also #149

[jponge]

A generic image is probably easier.

[metacosm]

I've made it so that all the java runtimes use the same image as Spring Boot at the moment however, this still doesn't work for vert.x as several jars are generated and we use a blanket cp *.jar to identify the binary without knowing its name and vert.x generates several jars thus breaking that lame duck fix… 😄

We could use env variables in the runtime definition to specify the "shape" of the expected binary name so that we can step around this issue. Smarter build / run scripts are also possible options but I'd prefer explicit fixes for each runtime as opposed to heuristics aiming at identifying a possible binary to use…

[jponge]

What if we simply had a resource parameter to give the artefact?

This would work for self-contained Jars and also native images.

On 22 Oct 2019, at 16:55, Chris Laprun notifications@github.com wrote:

I've made it so that all the java runtimes use the same image as Spring Boot at the moment however, this still doesn't work for vert.x as several jars are generated and we use a blanket cp *.jar to identify the binary without knowing its name and vert.x generates several jars thus breaking that lame duck fix… 😄

We could use env variables in the runtime definition to specify the "shape" of the expected binary name so that we can step around this issue. Smarter build / run scripts are also possible options but I'd prefer explicit fixes for each runtime as opposed to heuristics aiming at identifying a possible binary to use…

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[cmoulliard]

1. . We could certainly adopt a similar strategy for the other java runtimes but we just haven't done so yet.

@Ladicek @jponge @cescoffier Do you consider to have a runtime image including GAVs to speed the build process within the pod and could then help us ?

[cmoulliard]

Generifying the hal-mvn-jdk image is probably the best solution in the short term

I agree upon this decision and I think that we should decide if we will extend the existing OpenJDK S2I image or have a UBI packaging maven + JDK or ... and including a list of needed commands :

• run: to start the java runtine
• test: to perform a maven or gradle test
• debug: to debug remotely
• hotreload: to support a hotreload using websocket
• build: to build
• ???

WDYT ? @metacosm @Ladicek @jponge @cescoffier

[metacosm]

What if we simply had a resource parameter to give the artefact? This would work for self-contained Jars and also native images.

This would create more work for the user, though… and in most instances, the name of the artifact is always the same for a given runtime and project GAV, i.e. these rules are known at the runtime level so it'd feel redundant to ask the user to provide that as part of the resources they're creating.

[metacosm]

Generifying the hal-mvn-jdk image is probably the best solution in the short term

I agree upon this decision and I think that we should decide if we will extend the existing OpenJDK S2I image or have a UBI packaging maven + JDK or ... and including a list of needed commands :

* run: to start the java runtine

* test: to perform a maven or gradle test

* debug: to debug remotely

* hotreload: to support a hotreload using websocket

* build: to build

* ???

WDYT ? @metacosm @Ladicek @jponge @cescoffier

I think the simpler the image, the better: fewer attach vectors, faster downloads, less maintenance… We can certainly think about future use cases but we need to have something that works for all runtimes now so I'd suggest focusing on fixing build and run for now and leave the rest for later.

[jponge]

I concur with @metacosm on build + run being priority 1.

Hot reload is interesting as well but that might be different from the experience you can get locally with Vert.x or Quarkus since upload time may be the main factor (both Vert.x and Quarkus are known to boot fast).

[cmoulliard]

I think the simpler the image, the better: fewer attach vectors, faster downloads, less maintenance…

We can rely on the UBI images as Quarkus did : https://github.com/quarkusio/quarkus-images

[Ladicek]

I'm trying to create a generic JDK image that doesn't have pre-downloaded artifacts and delegates build and run to the S2I scripts in /usr/local/s2i. It should be relatively easy to add another layer on top of that with preloaded local Maven repo; the S2I scripts have support for that.

So far, I'm struggling with the build script finishing way before Maven build finishes, but once I overcome that, I'll submit PRs to show you what I've got.

[metacosm]

Ok, I’ve updated our hal-maven-jdk image so that it doesn’t bundle the SB dependencies anymore and it should work for all runtimes now.

Le 24 oct. 2019 à 09:40, Ladislav Thon notifications@github.com a écrit :

 I'm trying to create a generic JDK image that doesn't have pre-downloaded artifacts and delegates build and run to the S2I scripts in /usr/local/s2i. It should be relatively easy to add another layer on top of that with preloaded local Maven repo; the S2I scripts have support for that.

So far, I'm struggling with the build script finishing way before Maven build finishes, but once I overcome that, I'll submit PRs to show you what I've got.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.