moditect / layrry

A Runner and API for Layered Java Applications
Apache License 2.0
335 stars 33 forks source link

ClassNotFoundException with shared modules #110

Open mortharia opened 3 years ago

mortharia commented 3 years ago

Hey,

We use Layrry to test a plugins based application. From the module layers process, we understand that when the core layer use a module, suppose spring.data.commons, the plugins should not include it in their layers. So for our application, we tag this shared module (spring.data.commons) as required in the module-info of the plugins but we include its maven dependency as provided to make sure they compile, .

The problem is when we run the application and request a plugin service, we get a ClassNotFoundException for classes invoked by the plugin from the shared module packages!!!

Have you had this problem and any idea to solve it?

Thank you.

aalmiray commented 3 years ago

Could you share the layer configuration file? Please obfuscate or change the names of any non-public dependencies/modules you may have in your application to retain privacy (if warranted).

If the code is available from a public repository even better.

If I understood correctly, the layers configuration could look like this

[layers.core]
    modules = [
        "com.acme:some-other-dependency:{{project_version}}",
        "com.acme:spring.data.commons:{{project_version}}",
        "org.moditect.layrry:layrry-platform:{{layrry_version}}"]
[layers.plugins]
    parents = ["core"]
    directory = "plugins"
[main]
  module = "com.acme.launcher"
  class = "com.acme.launcher.Main"

If this is the case then any plugins that are added/removed from the plugins layer will have access to classes defined in the core layer, regardless of how these plugin modules are configured for compilation.

mortharia commented 3 years ago

Yes, you can find a sample that reproduce the error in https://github.com/mortharia/layrry-demo

Appication details are provided in README.

aalmiray commented 3 years ago

Running the demo application with the specified instructions leads to an exception before plugins can be loaded

$ java -jar launcher-0.0.1-SNAPSHOT.jar
May 02, 2021 5:04:19 PM org.jboss.shrinkwrap.resolver.impl.maven.logging.LogTransferListener transferFailed
WARNING: Failed downloading org/jenkins-ci/jenkins/1.26/jenkins-1.26.pom from https://repo1.maven.org/maven2/. Reason: 
org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.jenkins-ci:jenkins:pom:1.26 in central (https://repo1.maven.org/maven2)
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:107)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: org.jboss.shrinkwrap.resolver.api.NoResolvedResultException: Unable to collect/resolve dependency tree for a resolution due to: Failed to collect dependencies at com.infradna.tool:bridge-method-annotation:jar:1.13 -> org.jenkins-ci:annotation-indexer:jar:1.4, caused by: Could not find artifact org.jenkins-ci:jenkins:pom:1.26 in central (https://repo1.maven.org/maven2)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.wrapException(MavenWorkingSessionImpl.java:503)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:242)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.using(MavenStrategyStageBaseImpl.java:70)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withoutTransitivity(MavenStrategyStageBaseImpl.java:57)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withoutTransitivity(MavenStrategyStageBaseImpl.java:39)
    at org.moditect.layrry.internal.resolver.ConfigurableRemoteArtifactResolverSystemImpl.resolve(ConfigurableRemoteArtifactResolverSystemImpl.java:80)
    at org.moditect.layrry.internal.resolver.ResolveImpl.resolve(ResolveImpl.java:58)
    at org.moditect.layrry.internal.LayersImpl.getModulePathEntries(LayersImpl.java:218)
    at org.moditect.layrry.internal.LayersImpl.handleLayerComponent(LayersImpl.java:147)
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:119)
    at layrry.demo.Launcher.main(Launcher.java:49)
    ... 8 more
Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Failed to collect dependencies at com.infradna.tool:bridge-method-annotation:jar:1.13 -> org.jenkins-ci:annotation-indexer:jar:1.4
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:353)
    at org.jboss.shrinkwrap.resolver.impl.maven.bootstrap.MavenRepositorySystem.resolveDependencies(MavenRepositorySystem.java:121)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:239)
    ... 17 more
Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at com.infradna.tool:bridge-method-annotation:jar:1.13 -> org.jenkins-ci:annotation-indexer:jar:1.4
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:291)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:309)
    ... 19 more
Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.jenkins-ci:annotation-indexer:jar:1.4
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:301)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor(DefaultDependencyCollector.java:541)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult(DefaultDependencyCollector.java:524)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:412)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:509)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:461)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process(DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:254)
    ... 20 more
Caused by: org.apache.maven.model.resolution.UnresolvableModelException: Could not find artifact org.jenkins-ci:jenkins:pom:1.26 in central (https://repo1.maven.org/maven2)
    at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:176)
    at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:222)
    at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:1070)
    at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:846)
    at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:337)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:292)
    ... 31 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.jenkins-ci:jenkins:pom:1.26 in central (https://repo1.maven.org/maven2)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:424)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:229)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:207)
    at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:172)
    ... 36 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.jenkins-ci:jenkins:pom:1.26 in central (https://repo1.maven.org/maven2)
    at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:48)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:369)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:262)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:499)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:401)
    ... 39 more

I suspect you may have additional configuration in your settings.xml or artifacts in your default Maven Local repository that were put there without being downloaded from Maven Central.

aalmiray commented 3 years ago

Commenting out "com.infradna.tool:bridge-method-annotation:1.13" from the layers.yml configuration file results in the following error

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:///Users/aalmiray/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/aalmiray/dev/github/layrry/layrry-launcher/target/distributions/layrry-launcher-1.0.0-SNAPSHOT-dist/layrry-launcher-1.0.0-SNAPSHOT/lib/slf4j-simple-1.7.29.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.1)

2021-05-02 17:09:55.212  INFO 36609 --- [           main] layrry.demo.rest.api.DemoApplication     : Starting DemoApplication using Java 11.0.11 on aalmiray2.local with PID 36609 (/Users/aalmiray/.m2/repository/demo/rest-api-module/0.0.1-SNAPSHOT/rest-api-module-0.0.1-SNAPSHOT.jar started by aalmiray in /private/tmp/layrry-demo/launcher/target)
2021-05-02 17:09:55.215  INFO 36609 --- [           main] layrry.demo.rest.api.DemoApplication     : No active profile set, falling back to default profiles: default
2021-05-02 17:09:55.520  INFO 36609 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-05-02 17:09:55.573  INFO 36609 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 47 ms. Found 1 JPA repository interfaces.
2021-05-02 17:09:55.853  INFO 36609 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-05-02 17:09:55.885  INFO 36609 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version [WORKING]
2021-05-02 17:09:55.975  INFO 36609 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-05-02 17:09:56.044  INFO 36609 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-05-02 17:09:56.212  INFO 36609 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-05-02 17:09:56.234  INFO 36609 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2021-05-02 17:09:56.651  INFO 36609 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-05-02 17:09:56.657  INFO 36609 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-05-02 17:09:57.043  INFO 36609 --- [           main] layrry.demo.rest.api.DemoApplication     : Started DemoApplication in 2.156 seconds (JVM running for 3.624)
2021-05-02 17:09:57.050  INFO 36609 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: JpaEntityManagerFactory]
2021-05-02 17:09:57.059  WARN 36609 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001002: Using Hibernate built-in connection pool (not for production use!)
2021-05-02 17:09:57.059  INFO 36609 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001005: using driver [org.h2.Driver] at URL [jdbc:h2:file:./h2Base/layrry-demo-db]
2021-05-02 17:09:57.059  INFO 36609 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001001: Connection properties: {password=****, user=sa}
2021-05-02 17:09:57.059  INFO 36609 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001003: Autocommit mode: false
2021-05-02 17:09:57.061  INFO 36609 --- [           main] .c.i.DriverManagerConnectionProviderImpl : HHH000115: Hibernate connection pool size: 20 (min=1)
2021-05-02 17:09:57.063  INFO 36609 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2021-05-02 17:09:57.073  INFO 36609 --- [           main] org.hibernate.orm.connections.access     : HHH10001501: Connection obtained from JdbcConnectionAccess [org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator$ConnectionProviderJdbcConnectionAccess@7683ea43] for (non-JTA) DDL execution was not in auto-commit mode; the Connection 'local transaction' will be committed and the Connection will be set into auto-commit mode.
2021-05-02 17:09:57.093  INFO 36609 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Exception in thread "main" java.lang.RuntimeException: Couldn't run module main class
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:139)
    at org.moditect.layrry.Layrry.launch(Layrry.java:81)
    at org.moditect.layrry.Layrry.run(Layrry.java:56)
    at org.moditect.layrry.launcher.LayrryLauncher.launch(LayrryLauncher.java:117)
    at org.moditect.layrry.launcher.LayrryLauncher.main(LayrryLauncher.java:46)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:136)
    ... 4 more
Caused by: layrry.demo.rest.api.error.PluginException: java.lang.IllegalArgumentException: Not a managed type: class layrry.demo.data.provider.model.Data
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.service.DataServiceImpl.getDataCounts(DataServiceImpl.java:37)
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.DemoApplication.main(DemoApplication.java:30)
    ... 9 more
Caused by: java.lang.IllegalArgumentException: Not a managed type: class layrry.demo.data.provider.model.Data
    at org.hibernate.orm.core@5.4.25.Final/org.hibernate.metamodel.internal.MetamodelImpl.managedType(MetamodelImpl.java:582)
    at org.hibernate.orm.core@5.4.25.Final/org.hibernate.metamodel.internal.MetamodelImpl.managedType(MetamodelImpl.java:85)
    at spring.data.jpa@2.4.2/org.springframework.data.jpa.repository.support.JpaMetamodelEntityInformation.<init>(JpaMetamodelEntityInformation.java:75)
    at spring.data.jpa@2.4.2/org.springframework.data.jpa.repository.support.JpaEntityInformationSupport.getEntityInformation(JpaEntityInformationSupport.java:66)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataQueryDslJpaRepository.getJpaEntityInformation(DataQueryDslJpaRepository.java:36)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataQueryDslJpaRepository.<init>(DataQueryDslJpaRepository.java:25)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataQueryDslJpaRepository.<init>(DataQueryDslJpaRepository.java:21)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataPluginDAO.initDataRepository(DataPluginDAO.java:39)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataPluginDAO.counts(DataPluginDAO.java:33)
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.service.DataServiceImpl.getDataCounts(DataServiceImpl.java:35)
    ... 10 more
^C2021-05-02 17:10:34.422  INFO 36609 --- [extShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2021-05-02 17:10:34.424  INFO 36609 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-05-02 17:10:34.427  INFO 36609 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

This means the plugin was loaded but something else failed likely due to missing behavior from the commented dependency. Now, the rest-api-module defines the following module descriptor

import layrry.demo.rest.api.internal.DemoPluginLifecycleListener;
import layrry.demo.provider.api.service.ApiDAOFactory;
import org.moditect.layrry.platform.PluginLifecycleListener;

module layrry.demo.rest.api {

    requires layrry.demo.provider.api;
    requires org.moditect.layrry.platform;

    requires java.annotation;
    requires java.persistence;
    requires java.validation;

    requires org.apache.commons.lang3;

    requires org.hibernate.validator;

    requires spring.beans;
    requires spring.boot;
    requires spring.boot.autoconfigure;
    requires spring.boot.starter.data.jpa;
    requires spring.context;
    requires spring.core;
    requires spring.data.commons;
    requires spring.data.jpa;
    requires spring.jcl;
    requires spring.jdbc;
    requires spring.orm;
    requires spring.tx;

    requires querydsl.core;

    exports layrry.demo.rest.api;
    exports layrry.demo.rest.api.model;
    exports layrry.demo.rest.api.model.dto;
    exports layrry.demo.rest.api.error;
    exports layrry.demo.rest.api.service;

    opens layrry.demo.rest.api;
    opens layrry.demo.rest.api.model;
    opens layrry.demo.rest.api.model.dto;
    opens layrry.demo.rest.api.error;
    opens layrry.demo.rest.api.service;

    provides PluginLifecycleListener with DemoPluginLifecycleListener;
    uses ApiDAOFactory;
}

It registers a PluginLifecycleListener, so far so good. Notice that is also consumes instances of ApiDAOFactory. The plugin module is defined as

import layrry.demo.provider.api.service.ApiDAOFactory;
import layrry.demo.data.provider.service.DataPluginDAOFactory;

module layrry.demo.data.provider {
    requires layrry.demo.provider.api;

    requires com.h2database;

    requires java.sql;
    requires java.annotation;
    requires java.persistence;
    requires java.validation;

    requires spring.data.commons;
    requires spring.data.jpa;
    requires spring.beans;
    requires spring.core;
    requires spring.context;
    requires spring.jdbc;
    requires spring.orm;
    requires spring.tx;

    requires querydsl.core;
    requires org.hibernate.orm.core;

    exports layrry.demo.data.provider.model;
    exports layrry.demo.data.provider.service;

    opens layrry.demo.data.provider.model;
    opens layrry.demo.data.provider.service;

    provides ApiDAOFactory with DataPluginDAOFactory;
}

It provides an instance of ApiDAOFactory. So far so good again.

However the PluginLifecycleListener does nothing more than keep track of plugins, it performs no initialization/registration/whatever

package layrry.demo.rest.api.internal;

import org.moditect.layrry.platform.PluginDescriptor;
import org.moditect.layrry.platform.PluginLifecycleListener;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

/**
 * Plugins listener to dynamically register installed plugins as ModuleLayers.
 */
public class DemoPluginLifecycleListener implements PluginLifecycleListener {

    /**
     * All registered plugins moduleLayers with one plugin per moduleLayer.
     * Each plugin is identified by its name and is managed in one ModuleLayer.
     */
    private static Map<String, ModuleLayer> moduleLayers = new ConcurrentHashMap<>();

    @Override
    public void pluginAdded(PluginDescriptor plugin) {
        moduleLayers.put(plugin.getName(), plugin.getModuleLayer());
    }

    @Override
    public void pluginRemoved(PluginDescriptor plugin) {
        moduleLayers.remove(plugin.getName());
    }

    /**
     * Get all installed plugins moduleLayers.
     *
     * @return : the registered moduleLayers
     */
    public static Map<String, ModuleLayer> getModuleLayers() {
        return moduleLayers;
    }
}

This tells me that instances of ApiDAOFactory are loaded in some way that's not compatible with how Layrry plugins work. This is how the modular-tiles application does it

package org.kordamp.tiles.core;

import org.kordamp.tiles.model.PluginRegistry;
import org.kordamp.tiles.model.TileContext;
import org.kordamp.tiles.model.TilePlugin;
import org.moditect.layrry.platform.PluginDescriptor;
import org.moditect.layrry.platform.PluginLifecycleListener;

import java.util.Collection;
import java.util.LinkedHashSet;
import java.util.ServiceLoader;

public class TilePluginLifecycleListener implements PluginLifecycleListener {
    @Override
    public void pluginAdded(PluginDescriptor plugin) {
        ModuleLayer layer = plugin.getModuleLayer();

        // Load all plugins
        ServiceLoader<TilePlugin> loader = ServiceLoader.load(layer, TilePlugin.class);

        // collect and filter plugins by _this_ layer
        Collection<TilePlugin> plugins = new LinkedHashSet<>();
        loader.forEach(tilePlugin -> {
            if (tilePlugin.getClass().getModule().getLayer() == layer) {
                plugins.add(tilePlugin);
            }
        });

        // register plugins of _this_ layer
        PluginRegistry.getInstance()
            .registerPlugins(layer, plugins);
    }

    @Override
    public void pluginRemoved(PluginDescriptor plugin) {
        // unregister plugins of _this_ layer
        PluginRegistry.getInstance()
            .unregisterPlugins(plugin.getModuleLayer());
    }
}

In this case TilePlugin has the same function as ApiDAOFactory.

aalmiray commented 3 years ago

Digging further. Removing the plugin tar.gz file leads to the following error:

$ layrry --layers-config layers.yml 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:///Users/aalmiray/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/aalmiray/dev/github/layrry/layrry-launcher/target/distributions/layrry-launcher-1.0.0-SNAPSHOT-dist/layrry-launcher-1.0.0-SNAPSHOT/lib/slf4j-simple-1.7.29.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.1)

2021-05-02 17:26:30.534  INFO 37089 --- [           main] layrry.demo.rest.api.DemoApplication     : Starting DemoApplication using Java 11.0.11 on aalmiray2.local with PID 37089 (/Users/aalmiray/.m2/repository/demo/rest-api-module/0.0.1-SNAPSHOT/rest-api-module-0.0.1-SNAPSHOT.jar started by aalmiray in /private/tmp/layrry-demo/launcher/target)
2021-05-02 17:26:30.537  INFO 37089 --- [           main] layrry.demo.rest.api.DemoApplication     : No active profile set, falling back to default profiles: default
2021-05-02 17:26:30.853  INFO 37089 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-05-02 17:26:30.896  INFO 37089 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 37 ms. Found 1 JPA repository interfaces.
2021-05-02 17:26:31.173  INFO 37089 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-05-02 17:26:31.204  INFO 37089 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version [WORKING]
2021-05-02 17:26:31.294  INFO 37089 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-05-02 17:26:31.363  INFO 37089 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-05-02 17:26:31.521  INFO 37089 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-05-02 17:26:31.541  INFO 37089 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2021-05-02 17:26:31.951  INFO 37089 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-05-02 17:26:31.958  INFO 37089 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-05-02 17:26:32.330  INFO 37089 --- [           main] layrry.demo.rest.api.DemoApplication     : Started DemoApplication in 2.036 seconds (JVM running for 3.414)
Exception in thread "main" java.lang.RuntimeException: Couldn't run module main class
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:139)
    at org.moditect.layrry.Layrry.launch(Layrry.java:81)
    at org.moditect.layrry.Layrry.run(Layrry.java:56)
    at org.moditect.layrry.launcher.LayrryLauncher.launch(LayrryLauncher.java:117)
    at org.moditect.layrry.launcher.LayrryLauncher.main(LayrryLauncher.java:46)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:136)
    ... 4 more
Caused by: java.lang.UnsupportedOperationException: No service DAO implementation found for this data source connector : H2
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.service.AbstractDataService.getServiceDaoFactory(AbstractDataService.java:42)
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.service.DataServiceImpl.getDataCounts(DataServiceImpl.java:32)
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.DemoApplication.main(DemoApplication.java:30)
    ... 9 more
^C2021-05-02 17:26:50.914  INFO 37089 --- [extShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2021-05-02 17:26:50.916  INFO 37089 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-05-02 17:26:50.920  INFO 37089 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

Because it seems the application expects to always find at least one instance of ApiDAOFactory which of course is no longer the case. The ApiPluginServiceLoader class closes the loop with the DemoPluginLifeycleListener

@Service
public class ApiPluginServiceLoader {

    /**
     * Get the ApiDAOFactory service subtype implementation for a data source plugin type.
     *
     * @param type    the data source plugin type.
     * @return List of ApiDAOFactory implementations.
     */
    public List<ApiDAOFactory> getDSPluginService(String type) {
        /**
         * List of DAO service implementations of T class.
         */
        List<ApiDAOFactory> apiDAOFactories = new ArrayList<>();

        for (Map.Entry<String, ModuleLayer> layer : DemoPluginLifecycleListener.getModuleLayers().entrySet()) {
            ServiceLoader.load(layer.getValue(), ApiDAOFactory.class)
                    .stream()
                    .map(ServiceLoader.Provider::get)
                    .filter(d -> d.getApiDAO().accept(type))
                    .forEach(apiDAOFactories::add);
        }

        return apiDAOFactories;
    }

}

Be aware that if the getDSPluginService() method is called more than once then you'd be loading multiple instances of ApiDAOFactory for the same type.

aalmiray commented 3 years ago

Alright, one more thing. Forcing Layrry to look into Maven Local by adding the following to layers.yml:

resolve:
  remote: true
  workOffline: true

Yields the same error when the com.infradna.tool:bridge-method-annotation:1.13 dependency was commented out. Take note that this run does include the dependency, thus the failure is with Hibernate configuration

$ layrry --layers-config layers.yml 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:///Users/aalmiray/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/aalmiray/dev/github/layrry/layrry-launcher/target/distributions/layrry-launcher-1.0.0-SNAPSHOT-dist/layrry-launcher-1.0.0-SNAPSHOT/lib/slf4j-simple-1.7.29.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.1)

2021-05-02 17:48:07.221  INFO 38745 --- [           main] layrry.demo.rest.api.DemoApplication     : Starting DemoApplication using Java 11.0.11 on aalmiray2.local with PID 38745 (/Users/aalmiray/.m2/repository/demo/rest-api-module/0.0.1-SNAPSHOT/rest-api-module-0.0.1-SNAPSHOT.jar started by aalmiray in /private/tmp/layrry-demo/launcher/target)
2021-05-02 17:48:07.229  INFO 38745 --- [           main] layrry.demo.rest.api.DemoApplication     : No active profile set, falling back to default profiles: default
2021-05-02 17:48:07.540  INFO 38745 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-05-02 17:48:07.586  INFO 38745 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 38 ms. Found 1 JPA repository interfaces.
2021-05-02 17:48:07.859  INFO 38745 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-05-02 17:48:07.894  INFO 38745 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version [WORKING]
2021-05-02 17:48:08.009  INFO 38745 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-05-02 17:48:08.083  INFO 38745 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-05-02 17:48:08.276  INFO 38745 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-05-02 17:48:08.306  INFO 38745 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2021-05-02 17:48:08.771  INFO 38745 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-05-02 17:48:08.778  INFO 38745 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-05-02 17:48:09.225  INFO 38745 --- [           main] layrry.demo.rest.api.DemoApplication     : Started DemoApplication in 2.277 seconds (JVM running for 3.875)
2021-05-02 17:48:09.233  INFO 38745 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: JpaEntityManagerFactory]
2021-05-02 17:48:09.243  WARN 38745 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001002: Using Hibernate built-in connection pool (not for production use!)
2021-05-02 17:48:09.243  INFO 38745 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001005: using driver [org.h2.Driver] at URL [jdbc:h2:file:./h2Base/layrry-demo-db]
2021-05-02 17:48:09.244  INFO 38745 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001001: Connection properties: {password=****, user=sa}
2021-05-02 17:48:09.244  INFO 38745 --- [           main] org.hibernate.orm.connections.pooling    : HHH10001003: Autocommit mode: false
2021-05-02 17:48:09.245  INFO 38745 --- [           main] .c.i.DriverManagerConnectionProviderImpl : HHH000115: Hibernate connection pool size: 20 (min=1)
2021-05-02 17:48:09.247  INFO 38745 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2021-05-02 17:48:09.256  INFO 38745 --- [           main] org.hibernate.orm.connections.access     : HHH10001501: Connection obtained from JdbcConnectionAccess [org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator$ConnectionProviderJdbcConnectionAccess@6166aac5] for (non-JTA) DDL execution was not in auto-commit mode; the Connection 'local transaction' will be committed and the Connection will be set into auto-commit mode.
2021-05-02 17:48:09.277  INFO 38745 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
Exception in thread "main" java.lang.RuntimeException: Couldn't run module main class
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:139)
    at org.moditect.layrry.Layrry.launch(Layrry.java:81)
    at org.moditect.layrry.Layrry.run(Layrry.java:56)
    at org.moditect.layrry.launcher.LayrryLauncher.launch(LayrryLauncher.java:117)
    at org.moditect.layrry.launcher.LayrryLauncher.main(LayrryLauncher.java:46)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.moditect.layrry.internal.LayersImpl.run(LayersImpl.java:136)
    ... 4 more
Caused by: layrry.demo.rest.api.error.PluginException: java.lang.IllegalArgumentException: Not a managed type: class layrry.demo.data.provider.model.Data
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.service.DataServiceImpl.getDataCounts(DataServiceImpl.java:37)
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.DemoApplication.main(DemoApplication.java:30)
    ... 9 more
Caused by: java.lang.IllegalArgumentException: Not a managed type: class layrry.demo.data.provider.model.Data
    at org.hibernate.orm.core@5.4.25.Final/org.hibernate.metamodel.internal.MetamodelImpl.managedType(MetamodelImpl.java:582)
    at org.hibernate.orm.core@5.4.25.Final/org.hibernate.metamodel.internal.MetamodelImpl.managedType(MetamodelImpl.java:85)
    at spring.data.jpa@2.4.2/org.springframework.data.jpa.repository.support.JpaMetamodelEntityInformation.<init>(JpaMetamodelEntityInformation.java:75)
    at spring.data.jpa@2.4.2/org.springframework.data.jpa.repository.support.JpaEntityInformationSupport.getEntityInformation(JpaEntityInformationSupport.java:66)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataQueryDslJpaRepository.getJpaEntityInformation(DataQueryDslJpaRepository.java:36)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataQueryDslJpaRepository.<init>(DataQueryDslJpaRepository.java:25)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataQueryDslJpaRepository.<init>(DataQueryDslJpaRepository.java:21)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataPluginDAO.initDataRepository(DataPluginDAO.java:39)
    at layrry.demo.data.provider@0.0.1-SNAPSHOT/layrry.demo.data.provider.service.DataPluginDAO.counts(DataPluginDAO.java:33)
    at layrry.demo.rest.api@0.0.1-SNAPSHOT/layrry.demo.rest.api.service.DataServiceImpl.getDataCounts(DataServiceImpl.java:35)
    ... 10 more
^C2021-05-02 17:48:11.522  INFO 38745 --- [extShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2021-05-02 17:48:11.525  INFO 38745 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-05-02 17:48:11.529  INFO 38745 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
aalmiray commented 3 years ago

@gunnarmorling two things of note here:

  1. The org.jenkins-ci:jenkins:pom:1.26 file is flagged as unreachable from central but that clearly is not the case when you query MC directly. This makes me think Layrry may be expecting to resolve only JARs and not parent POMs. The other odd thing is that Layrry is supposed to resolve dependencies without transitivity yet we get and error because of parent POM resolution of transitive dependencies (!!). See com.infradna.tool:bridge-method-annotation:jar:1.18 -> org.jenkins-ci:annotation-indexer:jar:1.4

    WARNING: Failed downloading org/jenkins-ci/jenkins/1.26/jenkins-1.26.pom from https://repo1.maven.org/maven2/. Reason: 
    org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.jenkins-ci:jenkins:pom:1.26 in central (https://repo1.maven.org/maven2)
  2. This application may need some additional Hibernate configuration to get the data classes working. I must confess that's beyond my expertise but you might have a clue.

mortharia commented 3 years ago

Hi @aalmiray Thank you for all your feedback! I created another branch with a simplified example (without spring jpa) where the core (rest module) and the plugin use the same dependency for logging (slf4j). This plugin is used just to display a log. I apply all yours suggestions but I still get the same error.

In the PluginLifecycleListener implementation, I register all available plugins and don't load instance of their services. This will be done when I need a plugin service.

Now I have more precision on why I get a ClassNotFoundException. When I use IDEA to launch the launcher with target as working directory, I get the root cause

java.lang.IllegalAccessError: class layrry.demo.data.provider.service.DataPluginDAO (in module layrry.demo.data.provider) cannot access class org.slf4j.LoggerFactory (in unnamed module @0x1c481ff2) because module layrry.demo.data.provider does not read unnamed module @0x1c481ff2

Do I need to configure the plugin module that it can read unamed module (can't figure out how)?

In your vertex example, vertex dependency are used by the core and the plugin and in your layrry configuration file, you define a vertex module to handle it. Do we need to extract all shared dependencies in separate module? This require to update the configuration file for each new plugin!

Mortharia.