Grails 5.0.0.M2: inconsistent classpath in multi-module project with transitive plugins?

[davidkron]

We are migrating and trying out and older Grails 3.0.x application with Grails 4.1.0.M1. We have a not so trivial setup in a mono-repository structure which includes multiple grails applications, which together form some kind of platform, and some grails plugins that are shared by these applications. The repository itself is a multi-module gradle build.

For example we have some project dependencies like this: pluginA --(depends)--> common-plugin pluginB --(depends)--> common-plugin myapp --(depends)--> [pluginA, pluginB]

We are developing with exploded-mode (each plugin has a gradle.properties file with exploded=true).

Some applications are working great in Grails 4.1.0.M1 but in one we are getting the following exception:

groovy.lang.MissingMethodException: No signature of method: xxx.web.upload.validation.UploaderOptionsRegistry.register() is applicable for argument types: (String, xxx.web.upload.validation.UploaderOptions) values: [default, xxx.web.upload.validation.UploaderOptions@72a26048]
Possible solutions: register(java.lang.String, xxx.web.upload.validation.UploaderOptions)
The following classes appear as argument class and as parameter class, but are defined by different class loader:
xxx.web.upload.validation.UploaderOptions (defined by 'jdk.internal.loader.ClassLoaders$AppClassLoader@7aec35a' and 'org.springframework.boot.devtools.restart.classloader.RestartClassLoader@64e844ce')
If one of the method suggestions matches the method you wanted to call, 
then check your class loader setup.

Unfortunately I am not very experienced in the class loader debugging domain and I can't reproduce this exception in newly created module-module grails build. Also our real applications are fairly complex, so the exception might occur only in our specific constellation.

What I did discover though is the following: I did a print of the bootRun task's classpath and it seems that event though the plugins are exploded, sometimes the .jar AND the exploded classes folder is added. From my observation this seems to happen only when there is a dependency chain like myapp -> plugin2 -> plugin1 and not in a simple case of myapp -> plugin.

To show this, I created the example application with a multi-module gradle build. When running the application, the following is printed during bootRun:

C:\Users\david\IdeaProjects\grails-multi-module-problem\myapp\src\main\resources
C:\Users\david\IdeaProjects\grails-multi-module-problem\myapp\grails-app\views
C:\Users\david\IdeaProjects\grails-multi-module-problem\myapp\grails-app\i18n
C:\Users\david\IdeaProjects\grails-multi-module-problem\myapp\grails-app\conf
C:\Users\david\IdeaProjects\grails-multi-module-problem\myapp\build\classes\java\main
C:\Users\david\IdeaProjects\grails-multi-module-problem\myapp\build\classes\groovy\main
C:\Users\david\IdeaProjects\grails-multi-module-problem\myapp\gsp-classes
C:\Users\david\IdeaProjects\grails-multi-module-problem\myplugin2\build\libs\myplugin2-0.1.jar
C:\Users\david\IdeaProjects\grails-multi-module-problem\myplugin2\build\classes\groovy\main
C:\Users\david\IdeaProjects\grails-multi-module-problem\myplugin\build\resources\main
C:\Users\david\IdeaProjects\grails-multi-module-problem\myplugin\build\classes\groovy\main

As you can see myplugin2 is added as a jar-file to the classpath and also the classes folder. What also seems suspicious is that in the case of myplugin the resources folder is added to the classpath, but not in the case of myplugin2.

Since some classes of the plugins are available multiple times (through jar file and classes folder), I suspect this might be the problem of my initial exception about the class loader. But I can't be sure, since I could never reproduce it in a simple project.

Edit: the problem doesn't seem to happen when using the bootRun task directly, as it seems the exploded-configuration doesn't affect bootRun. I've been running the application through the run configuration in IntelliJ.

Task List

• [x] Steps to reproduce provided
• [x] Stacktrace (if present) provided
• [x] Example that reproduces the problem uploaded to Github
• [x] Full description of the issue provided (see below)

Steps to Reproduce

1. checkout exaple application
2. run application
3. see gradle output (println) from build.gradle during bootRun task

Environment Information

• Operating System: Windows 10
• Grails Version: Grails 4.1.0.M1
• JDK Version: 11

Example Application

https://github.com/davidkron/grails-multi-module-problem

[niravassar]

I took at look at this example application and it comes up running with bootRun.

Parsing through the problem description i don't see the example producing the error. You mentioned some features work and some don't. I don't think we can pin down the problem area just based on the description you have given. I think it will require a working example of the error propping up based on a http invocation of some sort.

can you possibly add to the example repo and conjure up a call flow so that an error pops up and describe what should occur vs what is actually occurring? That would help more and give me a starting point

[davidkron]

I still didn't manage to reproduce the exception from my previous post in a newly created minimal example application.

But I discovered some weird behavior regarding the bootRun classpath example from above, which I can't comprehend: Under Windows I sometimes get a correct classpath (all plugins exploded) and sometimes not (output like above: plugin2 both as jar AND exploded classes folder), without changing anything in the code. I can't explain what is the reason for the different behavior and it seems random to me.

Then I did some testing in an Unix environment (WSL2) and with the given start.sh script in the example repository I managed to get the effect every time. I also placed an assertion in the build.gradle file to show what in my opinion is not correct.

I pushed some additional code to the example at https://github.com/davidkron/grails-multi-module-problem

[davidkron]

I did some additional debugging of the groovy exception message in my initial comment in the context of our affected application, and can now definitely confirm it is a problem with the exploded classpath.

What I did:

• I placed a debug breakpoint in groovy.lang.MethodMissingException.getMessage() method, which is exactly the method which generates the groovy exception
• "Evaluate expression..." in IntelliJ with the following snippet:

  
  Class c1 = arguments[1].getClass()
  Class c2 = type.getMethods().find { it.name == method }.getParameterTypes()[1]

println c1.getCanonicalName() == c2.getCanonicalName() println c1.getProtectionDomain().getCodeSource().getLocation().getPath() println c2.getProtectionDomain().getCodeSource().getLocation().getPath() println c1.getClassLoader() println c2.getClassLoader()

This prints the following output:

true /C:/Users/david/IdeaProjects/bund/sbfi-grails4/sbfi-base/build/libs/sbfi-base.jar /C:/Users/david/IdeaProjects/bund/sbfi-grails4/sbfi-base/build/classes/groovy/main/ jdk.internal.loader.ClassLoaders$AppClassLoader@368239c8 org.springframework.boot.devtools.restart.classloader.RestartClassLoader@2164bebb

For me this confirms clearly that there is something wrong with the application classpath in a multi-module grails project using `exploded=true`.

[neilabdev]

While it may not appear related, try commenting out 'spring-boot-devtools' from your build.grade. I likewise had a complex multi project build, though my exception is reported differently, it was a result of dev-tools loading two seperate classpaths and comparing/using the same object from each. If commenting out 'spring-boot-devtools' works, thats the issue, otherwise not sure. There was no solution in my case, I simply reverted to 3.1.12 which doesn't require it. While thats not really a fix IMHO, I'm nolonger dead in the water.

This seems to be a problem with Spring devtools. The check here fails because the same class is loaded by 2 different class loaders. One if the system class loader and the other is devtools' RestartClassLoader.

https://github.com/grails/grails-data-mapping/blob/master/grails-datastore-gorm-validation/src/main/groovy/grails/gorm/validation/PersistentEntityValidator.groovy#L59

It is not clear to me why Spring Boot DevTools is loading the same classes in multiple class loaders. Seems like a bug in Boot.

Removing the devtools dependency from build.gradle works around the issue.

https://github.com/grails/grails-data-mapping/issues/970#issuecomment-317443202

https://github.com/grails/grails-core/issues/11654

[davidkron]

Thank you for the tip, but not really an option for me, since the alternative is to use spring-loaded, which doesn't work on Java 11. And if I am doing a big framework and infrastructure upgrade, I certainly won't use a 6 year old Java version.

[neilabdev]

Agreed. But if it can be identified & verfied it may be helpful in discovering a solution at a latter date, even though solution alludes us for now. In the interim , you may be able to extract your modules & install them locally, via ./gradlew install for each project . Not sure if that would solve the issue, IF devtools is the problem, but if it loads multi-module projects differently, it might. Nevertheless, good luck.

[davidkron]

I tried without devtools but it still happens. As far as I understand, devtools should be a runtime-only thing and not do anything in the build. See the following lines from my example project, where I already check the classpath at the bootRun task:

https://github.com/davidkron/grails-multi-module-problem/blob/da01c636cf206f074fd01d02ce829fc943460ec1/myapp/build.gradle#L85-L93

To me this says, that the culprit is already active in the build and has to be either Gradle, the Grails Gradle Plugin or the Spring Boot Gradle Plugin.

But I think I have to try later with Grails 5, since a lot of dependencies are likely newer than when I created this issue.

[davidkron]

I had another discovery recently:

We had another project migrated to Grails 4 recently and we wanted to use the Gradle composite build feature, to keep some plugins separated in their own repository, but include them in the same IntelliJ workspace during development. This also doesn't work smoothly with the Grails exploded configuration, as we have to include the plugins inside the dependencies-block and not the grails.plugins block. As can be seen in the following code snippet, the exploded configuration is only referenced when using the grails.plugins block: https://github.com/grails/grails-core/blob/77843bd857d3718d439e264b013eb566fc3afd32/grails-gradle-plugin/src/main/groovy/org/grails/gradle/plugin/core/PluginDefiner.groovy#L58

For the record, we are currently using the following workaround to solve the problem with the jar-artifact described in this issue:

configurations {
    // Workaround to make 'exploded' configuration consistent in multi-module gradle builds, so that
    // the runtime classpath only contains the 'exploded' classes folder and not the built jar-artifact aswell.
    // see https://github.com/grails/grails-core/issues/11566
    runtimeElements.artifacts.clear()
}

The following workaround makes Gradle composite builds work with exploded configuration:

configurations {
    // Workaround to make to Grails 'exploded' configuration work correctly in Gradle composite builds
    Configuration exploded = configurations.findByName('exploded')
    if (exploded) {
        // in a Gradle composite build, the 'exploded' configuration doesn't work correctly, as the plugins are
        // included in the regular dependencies and not inside the grails.plugins block (which explicitly references
        // the 'exploded' configuration, see https://github.com/grails/grails-core/blob/77843bd857d3718d439e264b013eb566fc3afd32/grails-gradle-plugin/src/main/groovy/org/grails/gradle/plugin/core/PluginDefiner.groovy#L58)
        runtime.artifacts.addAll(exploded.artifacts)
    }
}

These workarounds have to be placed inside the build.gradle file of those plugin projects, which also contains a gradle.properties file with exploded=true

[puneetbehl]

I am curious about what is the value for c2.getClassloader() when you are not using spring-boot-devtools. Also, please update the example to Grails 5.0.0-SNAPSHOT to verify if it is still a problem.

[xqliu]

This is still a problem on version 5.1.5, here is the detail info as below

One class loader is org.springframework.boot.devtools.restart.classloader.RestartClassLoader, which use to load current project class definition, another is jdk.internal.loader.ClassLoader$AppClassLoader, which use to load the classes inside jar files.

One possible workaround we have verified works on version 4 is to add the jar file you would like to use RestartClassLoader is

1. Add a file spring-devtools.properties to grails-app/conf/META-INFO/ folder
2. Content of the file to be

   restart.include.thirdparty=/<the_jar_file_you_would_like_to_use_restart_classloader>

By the way: the file name in the setting supports wildcard match

Related springboot documentations:

https://docs.spring.io/spring-boot/docs/2.1.2.RELEASE/reference/html/using-boot-devtools.html