Springboot 3.3.1 breaks jkube openshift plugin

Farzad407 commented 1 month ago

Describe the bug

springboot 3.3.1 comes with a new layertool that breaks jkube openshift plugin. Projects upgrading to springboot 3.3.1 will get the following error when they run oc:build command:

[ERROR] Failed to execute goal org.eclipse.jkube:openshift-maven-plugin:1.16.2:build (default-cli) on project layered-jar-3.3.1-bug: Failed to execute the build: Unable to build the image using the OpenShift build service: Assemblies with more than one layer require a proper id for each layer -> [Help 1]

The core of the issue is the new layertool is that there is an extra message accompanied by a new line in the output when command java -Djarmode=layertools -jar xxx.jar list is run

❯ java -Djarmode=layertools -jar target/layered-jar-3.3.1-bug-0.0.1-SNAPSHOT.jar list
Warning: This command is deprecated. Use '-Djarmode=tools list-layers' instead.

dependencies
spring-boot-loader
snapshot-dependencies
application

Jkube identifies the empty line as an individual layer without an id and fails with error message provided above. Additionally, JKube identifies any line output in the layertool as a layer. In our case, layer tool has additional lines in the output that are picked as layers erroneously. Here is a sample layertool output on our project:

❯ java -Djarmode=layertools -jar payment-service-0.0.1-SNAPSHOT.jar list
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
SLF4J(W): Class path contains multiple SLF4J providers.
SLF4J(W): Found provider [ch.qos.logback.classic.spi.LogbackServiceProvider@52a86356]
SLF4J(W): Found provider [org.apache.logging.slf4j.SLF4JServiceProvider@5ce81285]
SLF4J(W): See https://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J(I): Actual provider is of type [ch.qos.logback.classic.spi.LogbackServiceProvider@52a86356]
Warning: This command is deprecated. Use '-Djarmode=tools list-layers' instead.

dependencies
spring-boot-loader
snapshot-dependencies
application

in the above case, JKube creates the following 8 layers where as only layer 4-7 are applicable.

Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
Warning: This command is deprecated. Use '-Djarmode=tools list-layers' instead.
dependencies
spring-boot-loader
snapshot-dependencies
application

Eclipse JKube version

1.16.2

Component

OpenShift Maven Plugin

Apache Maven version

other (please specify in additional context)

Gradle version

None

Steps to reproduce

run mvn package oc:build on the project in https://github.com/Farzad407/layered-jar-3.3.1-bug repository.

Expected behavior

image build task to succeed.

Runtime

OpenShift

Kubernetes API Server version

1.25.3

Environment

macOS

Eclipse JKube Logs

[INFO] --- oc:1.16.2:build (default-cli) @ layered-jar-3.3.1-bug ---
[INFO] oc: Using OpenShift build with strategy Docker
[INFO] oc: Running generator spring-boot
[INFO] oc: spring-boot: Using Docker image quay.io/jkube/jkube-java:0.0.23 as base / builder
[INFO] oc: Spring Boot layered jar detected
[ERROR] oc: Failed to execute the build [Unable to build the image using the OpenShift build service]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.364 s
[INFO] Finished at: 2024-07-15T15:40:45-04:00
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "prodprofile" could not be activated because it does not exist.
[ERROR] Failed to execute goal org.eclipse.jkube:openshift-maven-plugin:1.16.2:build (default-cli) on project layered-jar-3.3.1-bug: Failed to execute the build: Unable to build the image using the OpenShift build service: Assemblies with more than one layer require a proper id for each layer -> [Help 1]

Sample Reproducer Project

https://github.com/Farzad407/layered-jar-3.3.1-bug

Additional context

maven version 3.9.6 but it should be reproducible across all maven versions.

rohanKanojia commented 1 month ago

@Farzad407 : Based on your findings, I see this line is causing the issue (we're adding any output as a layer) . I think we should apply some kind of filtering here to only add in case there is only one word in the line. Or maybe add some flag to layertools command to not add any additional output apart from layer names.

https://github.com/eclipse-jkube/jkube/blob/3db16fd58999eff4144056b76e8107a3219e47ba/jkube-kit/jkube-kit-spring-boot/src/main/java/org/eclipse/jkube/springboot/SpringBootLayeredJar.java#L110-L113

manusa commented 1 month ago

Relates to:

Farzad407 commented 1 month ago

@Farzad407 : Based on your findings, I see this line is causing the issue (we're adding any output as a layer) . I think we should apply some kind of filtering here to only add in case there is only one word in the line. Or maybe add some flag to layertools command to not add any additional output apart from layer names.

https://github.com/eclipse-jkube/jkube/blob/3db16fd58999eff4144056b76e8107a3219e47ba/jkube-kit/jkube-kit-spring-boot/src/main/java/org/eclipse/jkube/springboot/SpringBootLayeredJar.java#L110-L113

that was my first inclination but then I realized that layertools prints out other informational lines such the line about the logger in the path and the line about the new syntax. All those lines are not empty and are being added as a layer in the assembly. I am not sure if those layers will cause any issues in the image.

I think one possibility is to read and parse the layers.idx file but I have not done the research to see if there are any issues with that. One other possibility is to test if the layer actually exists in the jar.

rohanKanojia commented 1 month ago

~reading layers.idx file won't directly give us list of layers. It's a bit more detailed~

Looking at IndexLayers.java this is parsed as Map<String, List<String>>

- "dependencies":
  - "BOOT-INF/lib/"
- "spring-boot-loader":
  - "org/"
- "snapshot-dependencies":
- "application":
  - "BOOT-INF/classes/"
  - "BOOT-INF/classpath.idx"
  - "BOOT-INF/layers.idx"
  - "META-INF/"

rohanKanojia commented 1 month ago

I think one possibility is to read and parse the layers.idx file but I have not done the research to see if there are any issues with that

I think you're right. Reading layers.idx file appears to be most consistent approach of getting layer information across current and previous versions of Spring Boot. I've checked this approach with Spring Boot 3.0, 3.1, 3.2 and 3.3 . It seems to be working okay.

rohanKanojia commented 1 month ago

I have checked the format of layers.idx file in Spring Boot 2.5.6, 3.0.11, 3.1.5, 3.2.7, 3.3.1 . It's format is the same. The javadoc defines it like this:

LayersIndex.java

Index describing the layer to which each entry in a jar belongs. Index files are simple text files that should be read from top to bottom. Each file defines the layers and their content. Layer names are written as quoted strings prefixed by a dash space "- " and with a colon ":" suffix. Layer content is either a file or directory name written as a quoted string prefixed by space dash space " - ". A directory name ends with/, a file name does not. When a directory name is used it means that all files inside that directory are in the same layer.

Index files are designed to be compatible with YAML and may be read into a list of Map<String, List<String>> instances.

Farzad407 commented 1 month ago

@rohanKanojia thanks for the quick turn around on this. when do you think the fix will be released?

rohanKanojia commented 1 month ago

@Farzad407 Could you please check if you can resolve issue by using 1.17-SNAPSHOT?

You can check how to use JKube Snapshot artifacts here

Farzad407 commented 1 month ago

@rohanKanojia Yes I just tested 1.17-SNAPSHOT and it works!

eclipse-jkube / jkube