beryx / badass-jlink-plugin

Create a custom runtime image of your modular application
https://badass-jlink-plugin.beryx.org
Apache License 2.0
386 stars 27 forks source link

Package com.fasterxml.jackson not exist... #106

Closed chqu1012 closed 4 years ago

chqu1012 commented 4 years ago

Hi,

I want to build my application OpenJFX13 with Spring Boot. The gradle Task jlink prints me this error log.

C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1370: error: package com.fasterxml.jackson does not exist
    uses com.fasterxml.jackson.core;
                              ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1371: error: package com.fasterxml does not exist
    uses com.fasterxml.classmate;
                      ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1372: error: package com.fasterxml.jackson does not exist
    uses com.fasterxml.jackson.databind;
                              ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1373: error: package com.fasterxml.jackson.module does not exist
    uses com.fasterxml.jackson.module.paramnames;
                                     ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1374: error: package com.fasterxml.jackson does not exist
    uses com.fasterxml.jackson.annotation;
                              ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1375: error: package javafx does not exist
    uses javafx.base;
               ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1423: error: the service implementation type must be a subtype of the service interface type, or have a public static no-args method named "provider" returning the service implementation
    provides javax.xml.bind.JAXBContext with com.sun.xml.bind.v2.ContextFactory;
                                                                ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1437: error: package javax.enterprise.inject.spi does not exist
    provides javax.enterprise.inject.spi.Extension with org.springframework.data.jpa.repository.cdi.JpaRepositoryExtension;
                                        ^
C:\Users\...\build\jlinkbase\tmpjars\de.dc.merged.module\module-info.java:1512: error: package org.apache.logging.log4j.spi is not visible
    provides org.apache.logging.log4j.spi.Provider with org.apache.logging.slf4j.SLF4JProvider;

After that I added this required dependencies to the build.gradle file like below, but it still won't work.

jlink {
    mergedModule {
        requires 'com.fasterxml.jackson.core';
        requires 'com.fasterxml.classmate';
        requires 'com.fasterxml.jackson.databind';
        requires 'com.fasterxml.jackson.module.paramnames';
        requires 'com.fasterxml.jackson.annotation';
        requires 'javafx.base';
    }
    launcher {
        name = 'exe_name'
    }
}

Does anyone have an idea?

Best regards chqu1012

siordache commented 4 years ago

Can you upload a simple project that reproduces this issue?

xp-vit commented 4 years ago

Hi @siordache,

First of all thanks a lot for all your contribution. This plugin is very much needed by Java community.

I have created a very simple project which illustrates a problem: https://github.com/xp-vit/spring-boot-jlink/

Just checkout it and run ./gradlew jlink

I also added this to GitHub actions, so you can take a look right away: https://github.com/xp-vit/spring-boot-jlink/runs/472072372?check_suite_focus=true#step:6:21

I also assume that this might be related to plugin: id "org.javamodularity.moduleplugin" version "1.6.0"

but without it I'm getting: Task :compileJava FAILED /Users/xp-vit/git/test-jlink/src/main/java/module-info.java:3: error: module not found: spring.boot requires spring.boot;

Let me know if more info needed and if there is anything I can help you with. I'm rather new to jlinl , but I have enough experience in JVM world.

siordache commented 4 years ago

I submitted the necessary changes as a pull request.

The arguments of forceMerge are prefixes of the JAR file names (not of the module names). So, you need to pass jackson instead of com.fasterxml.jackson:

forceMerge 'jackson', 'log4j'

Also, the plugin is not able to detect all services required by your application, so you need to add the missing ones explicitly:

mergedModule {
    additive = true
    uses 'ch.qos.logback.classic.spi.Configurator'
    uses 'javax.validation.valueextraction.ValueExtractor'
    uses 'javax.validation.ConstraintValidator'
}

You currently use Gradle 6.1.1. It works fine with this simple example, but you may run into this issue when trying to build your real application. If this happens, you need to downgrade to Gradle 6.0.

xp-vit commented 4 years ago

Yes, it seems made a trick! Thanks!!

Just severals more questions:

  1. How you build this list of forceMerge and more interesting "mergedModule {...}"? Any tools?
  2. I'm going to complicate my repo a little bit and add more dependencies. Do you want to have it added to the examples section? I can create PR. Or it's beetter to fix jlinking of petclinic spring project?

Once again: Thanks!!

siordache commented 4 years ago
  1. I have no tools. It's an iterative process, which can be quite time-consuming for complex applications. I usually run gradle with the --info flag to see some additional info. Initially, for this simple application, ./gradlew -i jlink displayed:
    
    modularJars: [
    log4j-api-2.12.1.jar, 
    jackson-datatype-jdk8-2.10.2.jar, 
    jackson-databind-2.10.2.jar, 
    jackson-annotations-2.10.2.jar, 
    jackson-core-2.10.2.jar, 
    jackson-datatype-jsr310-2.10.2.jar, 
    jackson-module-parameter-names-2.10.2.jar, 
    classmate-1.5.1.jar]

nonModularJars: [ spring-boot-autoconfigure-2.2.4.RELEASE.jar, ... ]

modularJarsRequiredByNonModularJars: [log4j-api-2.12.1.jar, classmate-1.5.1.jar]

artifactsHandledAsNonModular: []


Most of the times, errors occur because some of the `modularJars` should be treated as non-modular (i.e., they should be force-merged into the merged module).  In some situations, the plugin detects by itself that this is necessary (and in such cases, the corresponding JARs appear in the `artifactsHandledAsNonModular` list).

But this didn't happen in our case, so we got error messages of the type `module not found: com.fasterxml.jackson.xxx`.
After configuring `forceMerge 'jackson'`, I got error messages concerning `log4`, so I also added it to `forceMerge`.

After that, `./gradlew jlink` was successful, but I got errors when trying to execute the application:

Exception in thread "main" java.util.ServiceConfigurationError: ch.qos.logback.classic.spi.Configurator: module com.example.merged.module does not declare uses



The error message is pretty clear: we need to add `uses 'ch.qos.logback.classic.spi.Configurator'` to the mergedModule. The only problem is that we get only one error message at a time, so we need to add the missing `uses` clause, build the image again, and execute it. After that, we run into the next problem. We need to repeat this process of fixing, building, and executing the image until the application works without errors.

2. Yes, I would like to have your application in the list of examples!
xp-vit commented 4 years ago

Thanks for explanation!

I added 1 more dependency to project: implementation("org.springframework.boot:spring-boot-starter-data-jpa")

And: https://github.com/xp-vit/spring-boot-jlink/runs/473588027?check_suite_focus=true#step:6:29

How do I solve such issue?

siordache commented 4 years ago

I submitted a pull request again.

You need to use the excludeProvides methods to get rid of the unwanted provides caluses:

mergedModule {
    ...
    excludeProvides implementation: 'com.sun.xml.bind.v2.ContextFactory'
    excludeProvides servicePattern: 'javax.enterprise.*'
}

I also configured build.gradle to make spring.factories available to your application by including it into the man jar:

prepareMergedJarsDir.doLast {
    // extract META-INF/spring.factories from spring-boot-autoconfigure
    copy {
        from zipTree(configurations.springAutoConfig.singleFile).matching {
            include 'META-INF/spring.factories'
        }
        into jlinkBasePath
    }

    // insert META-INF/spring.factories into the main jar
    ant.zip(update: "true", destfile: jar.archivePath, keepcompression: true) {
        fileset(dir: "$jlinkBasePath", includes: 'META-INF/**')
    }
}
ThibautSF commented 4 years ago

Thanks for those tips, it helps me to solve an equivalent problem: Jackson and missing spring.factories :+1:

But now I get an error which doesn't occur with the bootRun Gradle command :

15:38:07.036 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy$SuffixingRandom$BaseNameResolver

EDIT: gradle build file

plugins {
    id 'java'
    id 'org.springframework.boot' version '2.2.4.RELEASE'
    id "org.beryx.jlink" version "2.17.2"
    id 'org.javamodularity.moduleplugin' version '1.6.0'
}

apply plugin: 'io.spring.dependency-management'

group = 'com.mocah'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
mainClassName = 'com.mocah.mindmath.server.ServerApplication'

configurations {
    compileOnly {
        extendsFrom annotationProcessor
    }
}

configurations {
    springAutoConfig { transitive = false }
}

repositories {
    mavenCentral()
}

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
    runtimeOnly 'org.apache.derby:derby'

    springAutoConfig 'org.springframework.boot:spring-boot-autoconfigure'
}

jar {
    enabled = true
}

jlink {
    options = ['--strip-debug', '--compress', '2', '--no-header-files', '--no-man-pages']
    launcher {
        name = 'LaunchMindMath'
        customImage {
            appModules = ['com.mocah.merged.module']
        }
        jvmArgs = [
            '--add-reads', 'com.mocah.merged.module=com.mocah.mindmath',
            '-cp', '../app/*'
        ]
    }
    mergedModule {
        additive = true
        uses ...
        uses 'org.hibernate.integrator.spi.Integrator'
        uses 'org.hibernate.boot.registry.selector.StrategyRegistrationProvider'
        excludeProvides ...
    }

    forceMerge 'jackson', 'log4j'
}

prepareMergedJarsDir.doLast {
    // extract META-INF/spring.factories from spring-boot-autoconfigure
    ...

    // insert META-INF/spring.factories into the main jar
    ...
}
siordache commented 4 years ago

@ThibautSF Try to also force-merge byte-buddy:

forceMerge 'jackson', 'log4j', 'byte-buddy'

Maybe it helps.

ThibautSF commented 4 years ago

The fatal error still occurs

ThibautSF commented 4 years ago

So, I also force merge hibernate: ̀forceMerge 'jackson', 'log4j', 'byte-buddy', 'hibernate'`

And also :

    mergedModule {
        additive = true
        uses 'ch.qos.logback.classic.spi.Configurator'
        uses 'javax.validation.valueextraction.ValueExtractor'
        uses 'javax.validation.ConstraintValidator'
        uses 'org.hibernate.integrator.spi.Integrator'
        uses 'org.hibernate.service.spi.ServiceContributor'

        uses 'org.hibernate.boot.registry.selector.StrategyRegistrationProvider'
        uses 'org.hibernate.boot.spi.MetadataSourcesContributor'
        uses 'org.hibernate.boot.spi.MetadataBuilderInitializer'
        uses 'org.hibernate.boot.spi.MetadataBuilderFactory'
        uses 'org.hibernate.boot.model.TypeContributor'
        uses 'org.hibernate.boot.spi.MetadataContributor'
        uses 'org.hibernate.boot.spi.AdditionalJaxbMappingProducer'
        uses 'org.hibernate.boot.spi.SessionFactoryBuilderFactory'
        uses 'org.hibernate.service.spi.SessionFactoryServiceContributor'

        excludeProvides implementation: 'com.sun.xml.bind.v2.ContextFactory'
        excludeProvides servicePattern: 'javax.enterprise.*'
    }

And now the server start! :+1:

xp-vit commented 4 years ago

I have updated my repository and added dependency: runtimeOnly 'org.apache.derby:derby'

this actually forced spring to start configuring JPA (Hibernate implementation when I start the app). And I currently see same error: Caused by: javax.persistence.PersistenceException: Unable to resolve persistence unit root URL at com.example.merged.module/org.springframework.orm.jpa.persistenceunit.DefaultPersistenceUnitManager.determineDefaultPersistenceUnitRootUrl(Unknown Source) at com.example.merged.module/org.springframework.orm.jpa.persistenceunit.DefaultPersistenceUnitManager.preparePersistenceUnitInfos(Unknown Source) at com.example.merged.module/org.springframework.orm.jpa.persistenceunit.DefaultPersistenceUnitManager.afterPropertiesSet(Unknown Source) at com.example.merged.module/org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(Unknown Source) at com.example.merged.module/org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(Unknown Source) ... 17 common frames omitted Caused by: java.io.FileNotFoundException: class path resource [] cannot be resolved to URL because it does not exist at com.example.merged.module/org.springframework.core.io.ClassPathResource.getURL(Unknown Source) ... 22 common frames omitted

Also I found out that "application.properties" file is not read by app. This makes me thinking that there are some problems with packaging resource files.

In order to reproduce:

  1. Code here: https://github.com/xp-vit/spring-boot-jlink/
  2. ./gradlew jlink
  3. build/image/bin/hello-spring-b
benson0217 commented 4 years ago

Hi, I want to build my application OpenJFX11.02 with Spring Boot. The gradle Task jlink prints me this error log. by the way this is my module-info.java and i user oracle java11 module com.maytry.ui { requires javafx.controls; requires javafx.fxml; requires javafx.swing; requires com.jfoenix; requires spring.boot.autoconfigure; requires spring.context; requires spring.boot; requires org.apache.logging.log4j; requires lombok; requires com.fazecast.jSerialComm;

opens com.maytry.ui to spring.core;

exports com.maytry.ui;

} and this is my build.gradle plugins { id 'java' id 'application' id 'org.openjfx.javafxplugin' version '0.0.8' id 'org.springframework.boot' version '2.2.4.RELEASE' id 'org.beryx.jlink' version '2.17.2' }

dependencies { testCompile group: 'junit', name: 'junit', version: '4.12' implementation 'org.springframework.boot:spring-boot-starter' implementation 'io.projectreactor:reactor-core:3.3.2.RELEASE' implementation 'com.fasterxml:classmate:1.5.1' implementation project(':data-logger-collector-service') compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.9' compile 'com.jfoenix:jfoenix:9.0.8' compile 'com.github.ben-manes.caffeine:caffeine:2.8.1' }

javafx { modules = [ 'javafx.base', 'javafx.controls', 'javafx.fxml', 'javafx.swing' ] version = '11.0.2' }

jar { enabled = true }

jlink { options = ['--strip-debug', '--compress', '2', '--no-header-files', '--no-man-pages'] launcher { name = 'uiAPP' customImage { appModules = ['com.maytry.ui.merged.module'] } jvmArgs = [ '--add-reads', 'com.maytry.ui.merged.module=com.maytry.ui', '-cp', '../app/*' ] } mergedModule { additive = true uses 'ch.qos.logback.classic.spi.Configurator' } jpackage { imageName = 'UiApp' skipInstaller = true installerName = 'UiApp' installerType = 'pkg' } forceMerge 'jackson','log4j' }

mainClassName = 'com.maytry.ui/com.maytry.ui.UiApplication'

The gradle Task jlink alwas get me this error log.

Task :data-logger-collector-ui:createMergedModule Cannot derive uses clause from service loader invocation in: ch/qos/logback/classic/util/EnvUtil.loadFromServiceLoader(). Cannot derive uses clause from service loader invocation in: io/r2dbc/spi/ConnectionFactories.lambda$loadProviders$0(). Cannot derive uses clause from service loader invocation in: org/hibernate/validator/internal/util/privilegedactions/GetInstancesFromServiceLoader.loadInstances(). Cannot derive uses clause from service loader invocation in: org/springframework/beans/factory/serviceloader/AbstractServiceLoaderBasedFactoryBean.createInstance(). D:\work\outsourcing_projects\data-logger-collector\data-logger-collector-ui\build\jlinkbase\tmpjars\com.maytry.merged.module\module-info.java:822: error: module not found: com.fasterxml.classmate requires com.fasterxml.classmate; ^ D:\work\outsourcing_projects\data-logger-collector\data-logger-collector-ui\build\jlinkbase\tmpjars\com.maytry.merged.module\module-info.java:831: error: module not found: javafx.base requires javafx.base; ^ 2 errors

Task :data-logger-collector-ui:createMergedModule FAILED

FAILURE: Build failed with an exception.

please help me

xp-vit commented 4 years ago

@benson0217 you can find working jlink + javafx + spring example here: https://github.com/mockbirds/javafx-springboot-badass-jlink

please take a look and probably you'll find what's wrong with your seetup.

benson0217 commented 4 years ago

i already look that, but still got the error, i don't know why and i still don't know what to modify, Can you please give me a direction??

xp-vit commented 4 years ago

@benson0217 Then I'd suggest you to upload your/or example project to Github, so people can easily reproduce your issue. Then ask for help in separate Issue in this repo.

xp-vit commented 4 years ago

@siordache If you'll get a chance, please take a look at this comment: https://github.com/beryx/badass-jlink-plugin/issues/106#issuecomment-593630358

siordache commented 4 years ago

@xp-vit Here the pull request.

To solve the "Unable to resolve persistence unit root URL" issue, I followed this suggestion and disabled the autoconfiguration of hibernate:

@SpringBootApplication(exclude = {HibernateJpaAutoConfiguration.class})

Concerning the application.properties issue, the first thing I did was to copy the file into the bin/config directory:

tasks.jlink.doLast {
    copy {
        from "src/main/resources"
        into "${imageDir.asFile}/bin/config"
    }
}

Having application.properties outside of the jar is a useful thing, but it didn't solve the problem. The app still didn't read it and it took me an embarrassing amount of time to figure out why. As always, it looks obvious in retrospect: the previous approach was to use the spring.factories from spring-boot-autoconfigure. But this file doesn't include the ConfigFileApplicationListener, which is the one that reads application.properties. ConfigFileApplicationListener is included in the spring.factories from spring-boot.

So, the solution is to merge the content of the spring.factories files from several sources:

springFactoriesHolder 'org.springframework.boot:spring-boot'
springFactoriesHolder 'org.springframework.boot:spring-boot-autoconfigure'
// springFactoriesHolder 'org.springframework.boot:spring-boot-actuator-autoconfigure'

I updated badass-jlink-spring-petclinic using the same approach.

siordache commented 4 years ago

@benson0217 Try to forceMerge the problematic modules:

jlink {
    forceMerge 'javafx', 'classmate'
    ...
}

However, I totally agree with @xp-vit:

@benson0217 Then I'd suggest you to upload your/or example project to Github, so people can easily reproduce your issue. Then ask for help in separate Issue in this repo.

goatfryed commented 3 years ago

I found this issue looking for answers to resolve some build issues. All encountered issues could be solved thanks to the provided information. Great FAQ source!

rnett commented 3 years ago

Sorry to necro this a bit, but I've been having similar issues with this plugin, but using the badass-runtime plugin works fine. Is there any reason to use this plugin over that one?

siordache commented 3 years ago

This plugin is for JPMS applications, while badass-runtime is for non-JPMS ones, so choosing one of the plugins means choosing whether your application uses the Java Module System or not.

JPMS gives you strong encapsulation and reliable configuration but setting up the build script for a JPMS application may be challenging in some cases. You need to weigh the pros and cons of your specific situation. When building Spring Boot applications I prefer to leave them non-modularized and use the badass-runtime plugin. But this is just my preference.