Closed mmoayyed closed 1 year ago
I can certainly share a reproducer as a github repository if you find that to be helpful.
Hey there, I haven't had a good look at this yet, but yes a small project that reproduces the bug, (plus some basic instructions for how to build it and reproduce the bug as I've never made a GraalVM native image for a project before!), would really help. Thank you!
[EDIT]: Possibly related: #229, though that mentions a different exception
Here is the repository that you should be able to use to reproduce the issue:
https://github.com/mmoayyed/demo-spring-boot-native/tree/master
Instructions to build are here: https://github.com/mmoayyed/demo-spring-boot-native/tree/master#readme
Please note that you need the following to build:
java version "17.0.5" 2022-10-18 LTS
Java(TM) SE Runtime Environment GraalVM EE 22.3.0 (build 17.0.5+9-LTS-jvmci-22.3-b07)
Java HotSpot(TM) 64-Bit Server VM GraalVM EE 22.3.0 (build 17.0.5+9-LTS-jvmci-22.3-b07, mixed mode, sharing)
Once you run the native app, then you should be able to go to http://localhost:8080/cas and see the error.
Thanks for the demo project. However, it looks like it's relying on a lot of snapshot artifacts which I don't have access to, namely a lot of org.opensaml:opensaml-*:5.0.0-SNAPSHOT
artifacts, plus a net.shibboleth:shib-support:9.0.0-SNAPSHOT
, so I'm unable to run it just yet.
My apologies. I must have had those cached. I pushed a change that adds repositories into the build so you should be able to resolve those now. Could I ask you to pull again and resume? Thank you.
Thanks, I was able get a bit further with the updated code and then with the help of some online guides able to get passed some other setup issues (was initially missing a keystore at /etc/cas/thekeystore
, plus it looks like the server is hosting at https://localhost:8443/cas
instead), and finally was able to reproduce the bug.
Your idea of revisiting the problem area sounds like a good idea - git blame says this method hasn't been touched in 3 years! If I remember what this was for, it's that dialect instances aren't always present from the places that needed to use this, so a class lookup felt like a good way to go about it, but maybe there's a better way to do that now.
Anyway, the bug repro project is working and should let me get closer to figuring something out 👍
You're right, of course. With the addition of the keystore, the server switches to https. Thanks much for looking into this. If I can help with anything, please let me know.
There is a related issue reported at https://github.com/spring-projects/spring-boot/issues/33548
I've been playing around with this and trying to learn how GraalVM and native images work, and the main thing that I've found to get around the original issue is that the Class.getName
method needs to be exposed. So adding this somewhere in the MyRuntimeHints.java
allowed me to get a bit further:
try {
hints.reflection().registerMethod(Class.class.getMethod("getName"), ExecutableMode.INVOKE);
}
catch (NoSuchMethodException e) {
throw new RuntimeException(e);
}
Then I'd run into issues about more Default Groovy Method classes missing (eg: "org.codehaus.groovy.runtime.dgm$27"
) which I could add, then followed by the closure classes in the layout dialect (eg: nz.net.ultraq.thymeleaf.layoutdialect.context.extensions.IContextExtensions$_getPrefixForDialect_closure1
), which I could also add. Adding these manually was super tedious, so I started looking for whether there were better ways to go about these.
I see you've also gone part-way down the 'add Default Groovy Methods automatically' route with that GroovyDgmClassesRegistrationFeature
class, which looks to be inspired by this documentation: https://github.com/croz-ltd/klokwrk-project/blob/master/support/documentation/article/groovy-graalvm-native-image/groovy-graalvm-native-image.md#default-groovy-methods I hope that helps save you some trouble there?
I also learned that you could generate all this metadata by running Graal's native image agent (https://www.graalvm.org/22.0/reference-manual/native-image/Agent/), which I used and it managed to generate a lot of the JSON metadata that I was manually entering into MyRuntimeHints.java
. However, I couldn't figure out how to get that metadata into the build process as the demo project looks like it has its own step to generate files which I can see created in the build/generated/aotResources
folder?
Would it not be easier to convert the project to Java, given there is a fork that already has done the work?
@ultraq Thank you very much for the analysis. Based on your note, I was able to make a bit more progress forward and pushed a few more changes upstream here: https://github.com/mmoayyed/demo-spring-boot-native. Unfortunately, I have hit yet another blocker which appears to be a more general issue with Groovy 4: https://github.com/oracle/graal/issues/4484#issuecomment-1416795209
To your comments:
For the time being, it might be good for this project to provide runtime hints on its own for the stuff that is part of the nz.net.ultraq.thymeleaf
package. Perhaps at a minimum, stuff that is in this file would be good candidates for inclusion: https://github.com/mmoayyed/demo-spring-boot-native/blob/master/src/main/java/org/apereo/cas/MyRuntimeHints.java
Thanks for all your help thus far.
I've been looking at the ways to package the metadata needed to get this going with native builds, and I initially thought of adding it to https://github.com/oracle/graalvm-reachability-metadata as per the linked Spring issue, but that feels like a big undertaking, whereas adding RuntimeHints as per what your project is doing and what I've read from https://docs.spring.io/spring-framework/docs/current/reference/html/core.html#core.aot seems a lot easier to do 😅 It will be a Spring-specific solution, but given my lack of familiarity with GraalVM I think this is a good start for now.
However, Spring AOT was only introduced with Spring 6, and a requirement for Spring 6 is Java 17. Trying to include Spring 6 core/AOT given this project targets Java 8 results in Gradle build errors:
> No matching variant of org.springframework:spring-core:6.0.4 was found. The consumer was configured to find an API of a library compatible with Java 8, preferably not packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally but:
- Variant 'apiElements' capability org.springframework:spring-core:6.0.4 declares an API of a library, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally:
- Incompatible because this component declares a component compatible with Java 17 and the consumer needed a component compatible with Java 8
So that's where I'm at right now. I do like the idea of including the native/reflection metadata in the project like what's happening in the demo project and I've seen Spring do for other projects, but I'm trying to figure out how to package things given the Java 17 requirement of Spring AOT.
As for the question about converting this project to Java: the short-ish answer is no as I personally enjoy coding in Groovy over Java when I need to be on the JVM. The Java port of this project exists for those who need it and is a good option when Groovy is an issue in some way.
If I may help:
I initially thought of adding it to oracle/graalvm-reachability-metadata
This is only relevant when a project, for whatever reason, is unable or unwilling to host metadata itself. Since that is not the case here, you do not have to worry about that.
whereas adding RuntimeHints as per what your project is doing and what I've read from seems a lot easier to do
If you'd like to provide a Spring-specific solution, that is perfectly fine. However, aside from a dependency on Spring, this is not without other caveats. You also need to instruct other projects to have a compile-time dependency on this/your project, so they can import the hints and have them included as part of their build and their build process will generate the ultimate metadata (as does the demo project). While this is fine, I think there are easier ways to do this, namely:
src/main/resources/META-INF/native-image
We recommend that you provide the configuration for the native-image tool by embedding a native-image.properties file into a project JAR file. The native-image tool will also automatically pick up all configuration options provided in the META-INF/native-image/ directory (or any of its subdirectories) and use it to construct native-image command line arguments.
Then:
The generated configuration files can be supplied to the native-image tool by placing them in a META-INF/native-image/ directory on the class path, for example, in a JAR file used in the image build. This directory (or any of its subdirectories) is searched for files with the names jni-config.json, reflect-config.json, proxy-config.json and resource-config.json, which are then automatically included in the build. Not all of those files must be present. When multiple files with the same name are found, all of them are included.
This might also be a good resource: https://github.com/graalvm/graalvm-demos
Ah, I didn't realize you could put those into library JARs to share, some other docs I read gave me the impression it was only for the final application. That path sounds far better - I'll work towards that 👍
Hey there, I've had a go at this over the last week and a bit but I found the demo project doesn't seem to be working at the moment. I get errors like: Code generation is not supported for bean definitions declaring an instance supplier callback : Root bean: class [org.springframework.integration.config.MessagingAnnotationPostProcessor]; scope=singleton; abstract=false; lazyInit=null; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodNames=null; destroyMethodNames=null
. I'm not too sure what that's about, but now that I know just a little bit more about the native image build process I thought to use the benchmark project in this repo as a way to test including the native image metadata.
I did have to update another dependency of the layout dialect to get around some Groovy-specific issue (it's one of mine, the thymeleaf-expression-processor
), but after that I was successful in getting that benchmark project to compile and run as a native image 🥳 I've published a 3.2.1-SNAPSHOT
version of the layout dialect which includes the native image metadata and references an updated snapshot of the expression processor that also includes native image metadata.
This cut includes a native-image.properties
file which loads up the Groovy classes so that people shouldn't have to:
Args = --initialize-at-build-time=org.codehaus.groovy \
--initialize-at-build-time=org.apache.groovy \
--initialize-at-build-time=groovyjarjarantlr4.v4.runtime \
--initialize-at-build-time=groovy.lang
However, I'm not entirely sure if that's a responsibility of the layout dialect to include. I'm hoping that once better Groovy support in GraalVM comes along I can remove that.
Anyway, I hope this helps you make progress on creating a native image for your own project - let me know if you've got any more input or advice. I hope to get a proper release of all these updates out maybe next weekend.
Thanks much. I have reported the failure here: https://github.com/spring-projects/spring-integration/issues/8571 Once I get past this somehow, I will give the new SNAPSHOT a try.
I've gone and released 3.2.1 which contains the native image metadata, as it seems to have at least helped with the test case I came up with so maybe it'll be of use to someone else out there as well. I'll close this issue for now, but if you give that a try and find some more problems, feel free to let me know, either with a new issue or a comment on this one, whichever works for you 🙂
@mmoayyed did you manage?
Cross-referencing with a related issue - https://github.com/ultraq/thymeleaf-layout-dialect/issues/235
I have been able to make progress on this issue quite a bit, but it always feels like one step forward two steps back. The problems I face are largely Groovy-related, the latest of which happens to be something like this:
Caused by: org.thymeleaf.exceptions.TemplateProcessingException: Error during execution of processor 'nz.net.ultraq.thymeleaf.layoutdialect.decorators.DecorateProcessor' (template: "error" - line 2, col 64)
at org.thymeleaf.processor.element.AbstractAttributeModelProcessor.doProcess(AbstractAttributeModelProcessor.java:134) ~[cas:3.1.1.RELEASE]
at org.thymeleaf.processor.element.AbstractElementModelProcessor.process(AbstractElementModelProcessor.java:98) ~[cas:3.1.1.RELEASE]
at org.thymeleaf.util.ProcessorConfigurationUtils$ElementModelProcessorWrapper.process(ProcessorConfigurationUtils.java:649) ~[na:na]
at org.thymeleaf.engine.ProcessorTemplateHandler.handleOpenElement(ProcessorTemplateHandler.java:1510) ~[na:na]
at org.thymeleaf.engine.OpenElementTag.beHandled(OpenElementTag.java:205) ~[cas:3.1.1.RELEASE]
at org.thymeleaf.engine.Model.process(Model.java:282) ~[cas:3.1.1.RELEASE]
at org.thymeleaf.engine.Model.process(Model.java:290) ~[cas:3.1.1.RELEASE]
at org.thymeleaf.engine.GatheringModelProcessable.process(GatheringModelProcessable.java:78) ~[na:na]
at org.thymeleaf.engine.ProcessorTemplateHandler.handleCloseElement(ProcessorTemplateHandler.java:1640) ~[na:na]
at org.thymeleaf.engine.CloseElementTag.beHandled(CloseElementTag.java:139) ~[cas:3.1.1.RELEASE]
at org.thymeleaf.engine.TemplateModel.process(TemplateModel.java:136) ~[cas:3.1.1.RELEASE]
at org.thymeleaf.engine.TemplateManager.parseAndProcess(TemplateManager.java:661) ~[cas:3.1.1.RELEASE]
Which is then followed by:
Caused by: groovy.lang.MissingPropertyException: No such property: key for class: java.util.LinkedHashMap$Entry
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:67) ~[cas:4.0.12]
at org.codehaus.groovy.vmplugin.v8.IndyGuardsFiltersAndSignatures.unwrap(IndyGuardsFiltersAndSignatures.java:161) ~[na:na]
at java.base@17.0.6/java.lang.reflect.Method.invoke(Method.java:568) ~[cas:na]
at org.graalvm.nativeimage.builder/com.oracle.svm.core.methodhandles.Util_java_lang_invoke_MethodHandle.invokeInternal(Target_java_lang_invoke_MethodHandle.java:212) ~[na:na]
at java.base@17.0.6/java.lang.invoke.MethodHandle.invokeBasic(MethodHandle.java:76) ~[cas:na]
at org.graalvm.nativeimage.builder/com.oracle.svm.core.methodhandles.MethodHandleIntrinsicImpl.execute(MethodHandleIntrinsicImpl.java:181) ~[na:na]
at org.graalvm.nativeimage.builder/com.oracle.svm.core.methodhandles.Util_java_lang_invoke_MethodHandle.invokeInternal(Target_java_lang_invoke_MethodHandle.java:142) ~[na:na]
at java.base@17.0.6/java.lang.invoke.MethodHandle.invokeBasic(MethodHandle.java:76) ~[cas:na]
at java.base@17.0.6/java.lang.invoke.LambdaForm$NamedFunction.invokeWithArguments(LambdaForm.java:96) ~[na:na]
The problem appears to be here: https://github.com/ultraq/thymeleaf-layout-dialect/blob/main/thymeleaf-layout-dialect/source/nz/net/ultraq/thymeleaf/layoutdialect/models/ModelBuilder.groovy#L150
if (attributes) {
attributes.entrySet().each { entry ->
attributes[(entry.key)] = attributes[(entry.key)].toString()
}
}
It looks as though the native-agent is unclear where key
comes from. Of course there is a key
field, and there is of course a getKey()
method. So far, I have completely failed to come up with some sort of metadata that would teach it about key
.
I am using the latest version of the dialect, 3.2.0, with GraalVM 22.3 and JDK 17. When building a native image via the GraalVM gradle plugin, I ultimately see the following failure when I run the native application:
The key here seems to be:
...which does this:
I am a complete n00b when it comes to Groovy, but should the use of
dialectClass.name
be revisited? TheStandardDialect
class has a NAME constant, aname
property and here we are invokinggetName()
on aClass
. I think the native-image builder is getting confused here.My reflection configuration is as follows:
I can certainly share a reproducer as a github repository if you find that to be helpful.