GoogleContainerTools / jib

🏗 Build container images for your Java applications.
Apache License 2.0
13.58k stars 1.43k forks source link

Main class was not found for Spring Boot 3.1 and 3.2 #4154

Closed andytael closed 8 months ago

andytael commented 8 months ago

Environment:

Description of the issue: When building my application the build fails with [ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.0:build (default-cli) on project helloworld-31: Main class was not found, perhaps you should add amainClassconfiguration to jib-maven-plugi

Expected behavior: JIB should be able to introspect and find the right MainClass for Spring Boot applications.

If I force set the main class using

props.setProperty("jib.container.mainClass", "org.springframework.boot.loader.JarLauncher");
props.setProperty("jib.container.mainClass", "org.springframework.boot.loader.launch.JarLauncher");

I can get it to work however the MainClass for Spring Boot depends on the version. But that is not a very elegant way :-)

jib-maven-plugin Configuration:

            <build>
                <plugins>
                    <plugin>
                        <groupId>com.google.cloud.tools</groupId>
                        <artifactId>jib-maven-plugin</artifactId>
                        <version>3.4.0</version>
                        <configuration>
                            <containerizingMode>packaged</containerizingMode>
                            <allowInsecureRegistries>true</allowInsecureRegistries>
                        </configuration>
                    </plugin>
                </plugins>
            </build>

and I add this in Java as I build the image in an application and not from the command line.

        props.setProperty("interactiveMode", "false");
        props.setProperty("jib.from.image", "ghcr.io/graalvm/jdk:ol7-java17-22.2.0");
        props.setProperty("jib.to.image", registryUrl + "/" + imageName + ":" + imageVersion);
        props.setProperty("jib.to.auth.username", registryUsername);
        props.setProperty("jib.to.auth.password", registryPassword);
        props.setProperty("jib.container.format", "OCI");

Thanks Andy

andytael commented 8 months ago

Some more information, the MANIFEST.MF file contains the MainClass value (different for different version of SB).

Spring Boot 3.2

Manifest-Version: 1.0
Created-By: Maven JAR Plugin 3.3.0
Build-Jdk-Spec: 17
Implementation-Title: helloworld
Implementation-Version: 0.0.1-SNAPSHOT
Main-Class: org.springframework.boot.loader.launch.JarLauncher
Start-Class: com.example.helloworld.HelloworldApplication
Spring-Boot-Version: 3.2.0
Spring-Boot-Classes: BOOT-INF/classes/
Spring-Boot-Lib: BOOT-INF/lib/
Spring-Boot-Classpath-Index: BOOT-INF/classpath.idx
Spring-Boot-Layers-Index: BOOT-INF/layers.idx

Spring Boot 3.1

Manifest-Version: 1.0
Created-By: Maven JAR Plugin 3.3.0
Build-Jdk-Spec: 17
Implementation-Title: helloworld-31
Implementation-Version: 0.0.1-SNAPSHOT
Main-Class: org.springframework.boot.loader.JarLauncher
Start-Class: com.example.helloworld31.Helloworld31Application
Spring-Boot-Version: 3.1.6
Spring-Boot-Classes: BOOT-INF/classes/
Spring-Boot-Lib: BOOT-INF/lib/
Spring-Boot-Classpath-Index: BOOT-INF/classpath.idx
Spring-Boot-Layers-Index: BOOT-INF/layers.idx
andytael commented 8 months ago

Seems that GradleProjectProperties class has an implementation where it looks for Main-Class in the method getMainClassFromJarPlugin():

Object value = jarTask.getManifest().getAttributes().get("Main-Class");

vs the MavenProjectProperties class doesn't in the same method getMainClassFromJarPlugin(), well it looks for something I just don't know what?

Not sure if that is of any help?

chanseokoh commented 8 months ago

It's been a long while, so I don't remember the specifics, but given that you are talking about JarLauncher, I think you are looking at the Spring Boot fat jar created by the Spring Boot plugin? I believe Jib doesn't use the fat jar but the thin jar created by the maven-jar-plugin. Can't you just set the main class to com.example.helloworld.HelloworldApplication (in either <archive><manifest><mainClass> of maven-jar-plugin or jib.container.mainClass?

andytael commented 8 months ago

If I set the jib.container.mainClass to class com.example.helloworld.HelloworldApplication I get the following error when trying to run the JAR file:

Error: Could not find or load main class com.example.helloworld.HelloworldApplication
Caused by: java.lang.ClassNotFoundException: com.example.helloworld.HelloworldApplication

if I set jib.container.mainCLass to org.springframework.boot.loader.launch.JarLauncher or org.springframework.boot.loader.launch.JarLauncher (depending on the SB version) I can get it to work but setting this depending on the version of SB seems wrong?

The application is deploying a SB application and deploys it onto a k8s cluster. And the application doesn't know the SB version.

chanseokoh commented 8 months ago

I guess it should be com.example.helloworld31.Helloworld31Application? And what if you don't set <containerizingMode>packaged</containerizingMode>?

andytael commented 8 months ago

I got two different applications, one for SB 3.1 and one for SB 3.2. 3.2 starting class is com.example.helloworld.HelloworldApplication and the one for SB 3.1 is com.example.helloworld31.Helloworld31Application

<containerizingMode>packaged</containerizingMode> is set:

            <build>
                <plugins>
                    <plugin>
                        <groupId>com.google.cloud.tools</groupId>
                        <artifactId>jib-maven-plugin</artifactId>
                        <version>${jib-maven-plugin.version}</version>
                        <configuration>
                            <containerizingMode>packaged</containerizingMode>
                            <allowInsecureRegistries>true</allowInsecureRegistries>
                        </configuration>
                    </plugin>
                </plugins>
            </build>

I'm looking at the jib-gradle-plugin and the method getMainClassFromJarPlugin() in the GradleProjectProperties class and it seems to do a very different thing, maybe it is correct, but it looks for the value "Main-Class" in the JAR file:

Object value = jarTask.getManifest().getAttributes().get("Main-Class");

vs the same mermaid in the jib-maven-plugin where the method getMainClassFromJarPlugin() is doing something very different.

Is there a DEBUG flag that I can turn on to get more data?

chanseokoh commented 8 months ago

seems to do a very different thing

Basically it does the same thing; it retrieves the Main-Class attribute to be set inside the thin jar created by maven-jar-plugin or the Jar task (not by the Spring Boot plugin). In Maven, what it does is

  public String getMainClassFromJarPlugin() {
    Plugin mavenJarPlugin = project.getPlugin("org.apache.maven.plugins:maven-jar-plugin");
    if (mavenJarPlugin != null) {
      return getChildValue(
              (Xpp3Dom) mavenJarPlugin.getConfiguration(), "archive", "manifest", "mainClass")
          .orElse(null);

You seem to believe that the Gradle code opens up the built jar and inspects the manifest, but what it does is just to get the Main-Class information from the Jar task, much like Maven does (although the difference is that the info is retrieved from the Jar task API and hence very reliable, while in Maven, we just syntactically check the pom.xml configuration.

I think you should be able to see log messages by MainClassResolver where Jib got the main class info (e.g., here) when you increase the logging level to INFO or more.

andytael commented 8 months ago

I'll try to unset the <allowInsecureRegistries>true</allowInsecureRegistries>. Stay tuned

andytael commented 8 months ago

<allowInsecureRegistries>false </allowInsecureRegistries> gives the same error:

[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.0:build (default-cli) on project helloworld: Main class was not found, perhaps you should add amainClassconfiguration to jib-maven-plugin

Can I set the logging level using a parameter somehow?

chanseokoh commented 8 months ago

I believe mvn -X. And it's not allowInsecureRegistries but containerizingMode.

andytael commented 8 months ago

Ah, sorry about that!

Debugging gives me this:

[DEBUG]   (f) project = MavenProject: oracle.obaas:helloworld:0.0.1 @ /app/upload-dir/application/helloworld/containerPOM.xml
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@180b3819
[DEBUG] -- end configuration --
[DEBUG] Using JAR: /app/upload-dir/application/helloworld/target/helloworld-0.0.1.jar
[DEBUG] Searching for main class... Add a 'mainClass' configuration to 'jib-maven-plugin' to improve build speed.
[DEBUG] Could not find a valid main class from 'maven-jar-plugin'; looking into all class files to infer main class.
[DEBUG] MainClassFinder: /app/upload-dir/application/helloworld/target/classes is not a regular file; skipping

Not sure what that means?

andytael commented 8 months ago

If I don't use containerizingMode I still get this:

DEBUG]   (f) project = MavenProject: oracle.obaas:helloworld:0.0.1 @ /app/upload-dir/application/helloworld/containerPOM.xml
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@1e8ab90f
[DEBUG] -- end configuration --
[DEBUG] Searching for main class... Add a 'mainClass' configuration to 'jib-maven-plugin' to improve build speed.
[DEBUG] Could not find a valid main class from 'maven-jar-plugin'; looking into all class files to infer main class.
[DEBUG] MainClassFinder: /app/upload-dir/application/helloworld/target/classes is not a regular file; skipping
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.092 s
[INFO] Finished at: 2023-12-14T17:37:56Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.0:build (default-cli) on project helloworld: Main class was not found, perhaps you should add a `mainClass` configuration to jib-maven-plugin
chanseokoh commented 8 months ago

Do you have a .class file compiled from public static void main(...) under /app/upload-dir/application/helloworld/target/classes? Isn't it empty? It should output logs for all the sub-directories like this:

[DEBUG] MainClassFinder: /usr/local/google/home/chanseok/workspace4E/jib/examples/helloworld/target/classes is not a regular file; skipping                                                           
[DEBUG] MainClassFinder: /usr/local/google/home/chanseok/workspace4E/jib/examples/helloworld/target/classes/example is not a regular file; skipping                                                   
[DEBUG] MainClassFinder: /usr/local/google/home/chanseok/workspace4E/jib/examples/helloworld/target/classes/world is not a class file; skipping                                                       

MainClassFinder.find() is called from here and output these logs here.

andytael commented 8 months ago

I think so, I got this:

x BOOT-INF/
x BOOT-INF/classes/
x BOOT-INF/classes/com/
x BOOT-INF/classes/com/example/
x BOOT-INF/classes/com/example/helloworld/
x META-INF/maven/
x META-INF/maven/com.example/
x META-INF/maven/com.example/helloworld/
x BOOT-INF/classes/com/example/helloworld/HelloWorldController.class
x BOOT-INF/classes/com/example/helloworld/HelloworldApplication.class
x BOOT-INF/classes/application.properties
x META-INF/maven/com.example/helloworld/pom.xml
x META-INF/maven/com.example/helloworld/pom.properties

Attached is the JAR file that is used [DEBUG] Using JAR: /app/upload-dir/application/helloworld/target/helloworld-0.0.1.jar

helloworld-0.0.1-SNAPSHOT.jar.zip

chanseokoh commented 8 months ago

I am talking about /app/upload-dir/application/helloworld/target/classes (your log may be when not setting containerizingMode), not the contents inside the jar.

andytael commented 8 months ago

when not setting containerizingMode I get this error:

[DEBUG]   (f) project = MavenProject: oracle.obaas:helloworld:0.0.1 @ /app/upload-dir/application/helloworld/containerPOM.xml
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@1e8ab90f
[DEBUG] -- end configuration --
[DEBUG] Searching for main class... Add a 'mainClass' configuration to 'jib-maven-plugin' to improve build speed.
[DEBUG] Could not find a valid main class from 'maven-jar-plugin'; looking into all class files to infer main class.
[DEBUG] MainClassFinder: /app/upload-dir/application/helloworld/target/classes is not a regular file; skipping

And the /app/upload-dir/application/helloworld/target/classes is empty or at least I can't see them, maybe they get deleted before I can see them.

bash-4.4# ls -l
total 21628
drwxr-xr-x. 2 root root        6 Dec 14 19:02 classes
-rw-r--r--. 1 root root 22143943 Dec 14 19:02 helloworld-0.0.1.jar
bash-4.4# ls -l classes
total 0
andytael commented 8 months ago

Maybe a clarification on what we're trying to helps (or maybe not):

  1. We ask our customers to build a Spring Application and build a fat Spring JAR file of the application (spring-boot-maven-plugin)
  2. We then user JIB to package and containerize the JAR file. We are trying to avoid the (well known) security challenges of using a docker daemon during the build process.
  3. We then build the container and deploy it to an already installed k8s cluster (same for both build and deploy)
chanseokoh commented 8 months ago

Thanks. As you said, I believe /app/upload-dir/application/helloworld/target/classes is really empty, because otherwise, you should have seen some logs about its sub-directories as in my test project. And that's why Jib can't automatically find the main class. I think I see the picture now.

First of all, Jib doesn't use the fat jar produced by the spring-boot-maven-plugin. Whether or not someone really executes the Spring Boot plugin, it doesn't matter. Jib is designed to pick up the thin jar by the maven-jar-plugin.

Then, normally, you should see two jars under target/:

This may no longer be true if things might have changed. But this is what I remember from the past. And given the circumstances, this is my guess:

You just copied or left only the fat jar and are tricking Jib into believing that helloworld-0.0.1.jar, which is the fat jar by Spring Boot, is the only jar to be containerized. That may be why you set <containerizingMode>packaged, even though unpacked containerizing works just fine. And hopefully you are also pretending that your Maven project has no dependencies. If you are not, you'll be duplicating dependency jars (those put by Jib and those inside the far jar), which is super bad.

That's why manually setting the main class to com.example.helloworld31.Helloworld31Application doesn't work. The class is not accessible as a main class in the Spring Boot fat jar. It must be JarLauncher. And since JarLauncher is not a project class but some random class from one of the project dependencies, Jib can never infer it. And your Maven project tells Jib that there exist no classes, so nothing can be inferred from there either.

If all these are true, I'd say it's kind of a weird hack. If you have to use a Spring Boot fat jar (which I think you managed to achieve it with some tricks), then it's a fair game. You should set a correct main class yourself.

chanseokoh commented 8 months ago

But I discourage using a fat jar, because it completely destroys a lot of goodness about layering and reproducibility.

andytael commented 8 months ago

If I use helloworld-0.0.1-SNAPSHOT.jar.original I still get the same error:

[DEBUG]   (f) project = MavenProject: oracle.obaas:helloworld:0.0.1 @ /app/upload-dir/application/helloworld/containerPOM.xml
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@1e8ab90f
[DEBUG] -- end configuration --
[DEBUG] Searching for main class... Add a 'mainClass' configuration to 'jib-maven-plugin' to improve build speed.
[DEBUG] Could not find a valid main class from 'maven-jar-plugin'; looking into all class files to infer main class.
[DEBUG] MainClassFinder: /app/upload-dir/application/helloworld/target/classes is not a regular file; skipping
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.829 s
[INFO] Finished at: 2023-12-14T19:46:24Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.0:build (default-cli) on project helloworld: Main class was not found, perhaps you should add a `mainClass` configuration to jib-maven-plugin

I'll find some way to make it work

chanseokoh commented 8 months ago

Yeah, that's because /app/upload-dir/application/helloworld/target/classes is empty. However, when using the thin jar, I think manually setting the main class (either in maven-jar-plugin or in Jib) to HelloWorld31Application will probably work at least.

andytael commented 8 months ago

It doesn't work