Closed kristofdho closed 2 months ago
In addition to the above here is a list of -H:
options used by Quarkus: https://github.com/quarkusio/quarkus/issues/27784
kindly provided by @zakkak
@kristofdho we discussed your suggestions:
com.oracle.svm.core.SubstrateOptions
- Used to get the value of
SubstrateOptions.Class
, as we have multiple entrypoints within a module, and need to switch config based on which one is used.
As mentioned before use e.g. -Dmy.main.class=<MainClass>
to get entrypoint information into the Feature
com.oracle.svm.core.jdk.NativeLibrarySupport
,com.oracle.svm.hosted.c.NativeLibraries
andcom.oracle.svm.core.jdk.JNIRegistrationUtil
- Used to write library support in a similar way to
JNIRegistrationJava
.- The JNIRegistrationUtil superclass can be substituted by our own implementation, but without the other classes we have no way to do the static linking ourselves.
Please copy whatever you need from there into your source base. We have no intention making theses helper methods public API.
com.oracle.svm.core.jdk.Resources#registerResource(java.lang.String, java.lang.String, java.io.InputStream)
- Used in our custom support for the fontmanager library, to add the
fontconfig.bfc
file to the resources, to be used from the substitution.- We can possibly work around this by loading the resource in a static byte array instead in our substitutions.
This is an interesting usecase.
org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResource(Module module, String resourcePath, byte[] resource)
that will allow you to do that. Thanks for bringing this up.
com.oracle.svm.hosted.FeatureImpl.DuringSetupAccessImpl#registerClassReachabilityListener
- Used to add reachability handlers based on conditions not currently available. For us specifically a listener that fires for all classes with a specific annotation.
There are
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerClassInitializerReachabilityHandler
and
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerReachabilityHandler
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerMethodOverrideReachabilityHandler
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerSubtypeReachabilityHandler
in the API. The only thing left is to add API that allows you to query annotations without triggering JDK internal annotation handling. We have that internally since com.oracle.svm.util.GuardedAnnotationAccess
was added in June.
com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization
- Hack to conditionally add resource bundles during analysis, used in the same way as here.
- Any workaround suggestions to add simple resource-bundles on the fly would be much appreciated.
This needs to be fixed as well:
org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResourceBundle
API methods are usable during analysis (GR-40907)@zakkak from the reported -H:
options we have identified the following to be added as proper @APIOption
s:
-H:IncludeLocales
-H:BuildOutputJSONFile
-H:+AddAllCharsets
-H:NativeLinkerOption
They will be added soon (GR-40904).
For the other option-use you mentioned there are the following remarks:
-H:-ParseOnce
We recommend not using this option anymore.-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime
What is the reasoning behind that? Did you measure that this is actually beneficial?-H:+AllowFoldMethods
This should not be necessary. Please try to remove the option and report back if it causes problems.-H:FallbackThreshold=5
This is equivalent to API option --auto-fallback
-H:FallbackThreshold=0
This is equivalent to API option --no-fallback
-H:EnableURLProtocols
This is equivalent to API option --enable-url-protocols
-H:-InlineBeforeAnalysis
This option should not be used anymore.-H:-SpawnIsolates
Except for native-debugging on Windows this option should not be used anymore.-H:-StackTrace
What is the reasoning behind that? Did you measure that this is actually beneficial?They will be added soon (GR-40904).
Thank you @olpaw
For the other option-use you mentioned there are the following remarks:
* `-H:-ParseOnce` We recommend not using this option anymore.
This is only used with GraalVM < 22.2 so there is no issue moving forward.
* `-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime` What is the reasoning behind that? Did you measure that this is actually beneficial?
It seems like this was based on some past evaluation of the then options and needs to be re-evaluated. There are people working on it.
* `-H:+AllowFoldMethods` This should not be necessary. Please try to remove the option and report back if it causes problems.
@galderz it looks like this was introduced by you in https://github.com/quarkusio/quarkus/pull/13376. Can you please have a look?
* `-H:FallbackThreshold=5` This is equivalent to API option `--auto-fallback` * `-H:FallbackThreshold=0` This is equivalent to API option `--no-fallback` * `-H:EnableURLProtocols` This is equivalent to API option `--enable-url-protocols`
The above have been replaced in https://github.com/quarkusio/quarkus/pull/27783
* `-H:-InlineBeforeAnalysis` This option should not be used anymore.
The option adding this flag has been deprecated in https://github.com/quarkusio/quarkus/pull/27783
* `-H:-SpawnIsolates` Except for native-debugging on Windows this option should not be used anymore.
This was introduced back in Dec 2018, when isolates where spawned by default(?). We should mark this as deprecated and eventually remove it. Users (and testers) wil still disable full stack traces using quarkus.native.additional-build-args
.
* `-H:-StackTrace` What is the reasoning behind that? Did you measure that this is actually beneficial?
@Sanne this was introduced as an option in https://github.com/quarkusio/quarkus/commit/8d16a4214b14ccbc35eb29240435ab32866190ea do you remember why it was needed for?
In any case I believe we should deprecate the corresponding option and eventually delete it. Users will still be able to disable full stack traces using quarkus.native.additional-build-args
.
[Regarding -H:-StackTrace I've commented on https://github.com/quarkusio/quarkus/issues/27784#issuecomment-1241806173 ]
Unrelated to the -H options, I'm more concerned about the move of the annotations.
I'm understanding that many annotations are being moved from the svm
artifact to the new API artifact - this is nice.
However it would seem that some of them have a different package; as we're currently building Quarkus with the old annotations (we have to until GraalVM releases this), this is compiling fine (no errors, no warnings), but then the new GraalVM version isn't going to apply these in practice. Examples: @NeverInline
, @AlwaysInline
, @Uninterruptible
, @UnknownObjectField
, ...
This concerns me as we can only observe the problem via obscure side-effects; could we maintain the existing annotations in the current package in addition to the new one, deprecate the old ones, and have the compiler take into account both flavours?
Alternatively, may I suggest to only keep them in the original package, mark them as deprecated (so to express they are still not meant as stable API) and address this problem in the future. Essentially postpone the move to a new package and treat that as a separate task to be addressed in a different version, that would keep the current migration simpler and give some more time to consider these implications.
@olpaw
com.oracle.svm.core.jdk.NativeLibrarySupport
,com.oracle.svm.hosted.c.NativeLibraries
andcom.oracle.svm.core.jdk.JNIRegistrationUtil
- Used to write library support in a similar way to JNIRegistrationJava.
- The JNIRegistrationUtil superclass can be substituted by our own implementation, but without the other classes we have no way to do the static linking ourselves.
Please copy whatever you need from there into your source base. We have no intention making theses helper methods public API.
I agree that we should just copy whatever util function that we use, but that only covers the usages of JNIRegistrationUtil
.
We're still doing calls like:
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("some_prefix");
NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("library");
NativeLibraries nativeLibraries = ((FeatureImpl.BeforeAnalysisAccessImpl) a).getNativeLibraries();
nativeLibraries.addStaticJniLibrary("library");
nativeLibraries.addStaticNonJniLibrary("otherLibrary");
nativeLibraries.addDynamicNonJniLibrary("anotherLibrary");
com.oracle.svm.hosted.FeatureImpl.DuringSetupAccessImpl#registerClassReachabilityListener
There are
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerClassInitializerReachabilityHandler
and
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerReachabilityHandler org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerMethodOverrideReachabilityHandler org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerSubtypeReachabilityHandler
I'm probably missing something here, but are you suggesting the functionality I need is already available here? These methods always have some sort of matching criteria, what I'm requesting is a method that returns classes that have a certain annotation on them, eg:
void registerAnnotatedClassReachabilityHandler(BiConsumer<DuringAnalysisAccess, Class<?>> callback,
Class<? extends Annotation> annotation);
So it can be used like:
registerAnnotatedClassReachabilityHandler(MyFeature::annotatedClassReachable, javax.inject.Singleton.class);
Possibly extend it to allow searching for any annotated element:
void registerAnnotatedElementReachabilityHandler(BiConsumer<DuringAnalysisAccess, java.lang.reflect.AnnotatedElement> callback,
Class<? extends Annotation> annotation,
java.lang.annotation.ElementType elementType); // elementType could be optional
I don't follow exactly how GuardedAnnotationAccess
fits into that. Just to return the instance of the annotation, instead of calling Class#getAnnotation
? I guess that could still be kept internal, if the callback includes the annotation instance.
Side question - could we get/generate concise API doc/html with versions and since/deprecated notion in them assuming that list of those -H
options continues to grow/mutate?
@kristofdho regarding:
Ensure org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResourceBundle API methods are usable during analysis (GR-40907)
Are you sure this does not work with the new RuntimeResourceAccess#addResourceBundle
API method?
I tested the following:
Implemented DemoFeature
that provides org.graalvm.nativeimage.hosted.Feature#duringAnalysis
that uses
if (addBundle) {
Module module = ModuleLayer.boot().findModule("jdk.compiler").orElseThrow();
String bundleName = "com.sun.tools.javac.resources.javac";
System.out.printf("Adding ResourceBundle %s:%s%n", module.getName(), bundleName);
RuntimeResourceAccess.addResourceBundle(module, bundleName);
System.out.printf("ResourceBundle added%n");
addBundle = false;
}
Then in the Demo.main
method I use:
Module compilerModule = ModuleLayer.boot().findModule("jdk.compiler").orElseThrow();
String bundleName = "com.sun.tools.javac.resources.javac";
ResourceBundle bundle = ResourceBundle.getBundle(bundleName, compilerModule);
System.out.println(bundle);
bundle.getKeys().asIterator().forEachRemaining(System.out::println);
This works as expected and gives me
com.sun.tools.javac.resources.javac@2d2802e4
javac.opt.Xlint.desc.synchronization
javac.opt.arg.key.equals.value
javac.opt.Xdoclint.subopts
javac.opt.g
javac.opt.Xlint.desc.unchecked
javac.msg.usage.header
javac.opt.d
javac.opt.X
javac.opt.sourcepath
....
javac.opt.Xlint.all
javac.opt.Xlint.desc.module
javac.opt.release
javac.opt.modulepath
at image runtime.
Implemented DemoFeature that provides org.graalvm.nativeimage.hosted.Feature#duringAnalysis that uses
Also works if I use ModuleLayer.boot().findModule("java.xml") and
String bundleName = "com.sun.org.apache.xml.internal.serializer.utils.SerializerMessages";
or
String bundleName = "com.sun.org.apache.xalan.internal.xsltc.compiler.util.ErrorMessages";
@kristofdho, can you provide me with a sample program where you cannot use RuntimeResourceAccess.addResourceBundle
in org.graalvm.nativeimage.hosted.Feature#duringAnalysis
?
@olpaw I was merely reporting our usages of internal API and why they are used. I tried removing the setConfigurationSealed
call (with GraalVM 22.2) and it indeed seems to compile fine. This was added in our codebase back when we were using GraalVM 21.0 and hasn't been updated since. It definitely wasn't allowed to add resource bundles during analysis back then, as evident from this internal feature as well. It must have been updated along the way.
So we can probably assume this is no longer an issue. (The internal usage I refer to can probably also be cleaned up a bit, and no longer call setConfigurationSealed
).
So we can probably assume this is no longer an issue.
Thanks for confirming!
-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime
What is the reasoning behind that? Did you measure that this is actually beneficial?
This goes back to https://github.com/quarkusio/quarkus/commit/223288aa3d482a7617f0a72c09558dc3c850e584 in 2018. At the time the default GC policy (adaptive) was deemed to generate full GCs constantly compared to say the space/time one. I'm exploring if this is still the case today, and see what tuning can be done, if any, to improve things while staying with the default adaptive GC policy.
@christianwimmer please could you check https://github.com/oracle/graal/issues/4862#issuecomment-1241826003 as well - I think it was missed.
I don't follow exactly how GuardedAnnotationAccess fits into that. Just to return the instance of the annotation, instead of calling Class#getAnnotation? I guess that could still be kept internal, if the callback includes the annotation instance.
@kristofdho if you use
org.graalvm.nativeimage.hosted.Feature.BeforeAnalysisAccess#registerSubtypeReachabilityHandler
with java.lang.Object
as the subtype and you combine it with the recently merged
https://github.com/oracle/graal/pull/4989/files#diff-c30464fae56f58c5775804672939d2a4e75ab85a73420ba286099f2ceac5efea
you can do the same as what you would expect from registerAnnotatedElementReachabilityHandler
This is an interesting usecase.
We need to add a new variant to the API (GR-40906):
org.graalvm.nativeimage.hosted.RuntimeResourceAccess#addResource(Module module, String resourcePath, byte[] resource)
that will allow you to do that. Thanks for bringing this up.
@kristofdho this implemented and merged (will be in 22.3): https://github.com/oracle/graal/pull/4945
@christianwimmer please could you check #4862 (comment) as well - I think it was missed.
For the record, I've discussed this particular case at the face to face in Zurich with @christianwimmer and I think our conclusion was that it's not a big problem; if something could be done to catch people relying on the old annotations (now silently ignored) that would be a "nice to have", but not a blocker.
@olpaw We have identified an additional internal API usage, which sort-of fits in the bucket of using the options directly:
File imagePath = NativeImageGenerator.generatedFiles(HostedOptionValues.singleton()).toFile();
// This ultimately boils down to the value of SubstrateOptions.Path
We use this to find where the image will be writen.
I am aware that org.graalvm.nativeimage.hosted.Feature.AfterImageWriteAccess#getImagePath
exists. However, as we're building through Gluon Substrate, afterImageWrite
never gets called because they run with the -H:+ExitAfterRelocatableImageWrite
option:
https://github.com/gluonhq/substrate/blob/6b447be1317a51f3aecb68ba8a6ecada24aeac1a/src/main/java/com/gluonhq/substrate/target/AbstractTargetConfiguration.java#L144
Would it be possible to expose the parent directly where these files will get writen?
PS: from a library dev standpoint, it may be usefull to also expose com.oracle.svm.core.BuildArtefacts#add
so these type of additions are also included in the artefacts file, but that's obviously more a nice-to-have kind of thing.
I also don't think the PlatformNativeLibrarySupport
and NativeLibraries
class access has been adresses yet.
I also don't think the PlatformNativeLibrarySupport and NativeLibraries class access has been adresses yet.
@kristofdho we need to fully understand your need to use those low-level classes.
There is
-H:NativeLinkerOption=<string>* Pass the provided raw option that will be appended to the linker command to produce the final binary. The possible
options are platform specific and passed through without any validation.
If you need more than that please explain your specific usecase in detail on a separate issue.
@olpaw I'm honestly not sure if I need more than that or not.
We're using it to statically link in jni libraries, one example is the fontmanager library:
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_font");
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_java2d_loops_DrawGlyphList");
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_awt_Win32FontManager_getFontPath");
PlatformNativeLibrarySupport.singleton().addBuiltinPkgNativePrefix("sun_awt_Win32FontManager_populateFontFileNameMap0");
NativeLibraries nativeLibraries = ((FeatureImpl.BeforeAnalysisAccessImpl) a).getNativeLibraries();
NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("fontmanager");
NativeLibrarySupport.singleton().preregisterUninitializedBuiltinLibrary("freetype");
nativeLibraries.addStaticJniLibrary("fontmanager", "awt");
nativeLibraries.addStaticNonJniLibrary("freetype");
nativeLibraries.addDynamicNonJniLibrary("gdi32");
This works in the same way as other JNI libraries are configured in GraalVM already (sunmscapi
, sunec
, prefs
, mlib_image
, lcms
, ...)
I don't think that's possible entirely through -H:NativeLinkerOption
?
@olpaw I have discovered a few more API usage cases that were hidden by configuration to allow the previously fixed internal usages:
ImageSingletons.lookup(ResourcesRegistry.class).addResources(pattern);
RuntimeResourceAccess
only allows registering resources one by one, suggestion to add addResourcePattern(Module module, String pattern)
to allow the old behavior.
Context: We have a resource jar with static content to serve in a webapp, but this is only conditionally included. The resources itself are randomized as part of the build, but we just need to include the entire folder anyway, so adding them without wildcard is currently not possible.ImageSingletons.lookup(SystemPropertiesSupport.class).initializeProperty("property.name", "value")
initializeProperty(String property, String value)
and initializePropertyLazy(String property, Supplier<String> value)
BuildArtifacts.singleton().add(ArtifactType.JDK_LIB_SHIM, dll)
build_artifacts.txt
file contains everything correctly.
Edit: I realise this is a repeat of this comment https://github.com/oracle/graal/issues/4862#issuecomment-1257011390Current API in RuntimeResourceAccess only allows registering resources one by one, suggestion to add addResourcePattern(Module module, String pattern) to allow the old behavior.
At some point we want to get rid of pattern-based resource specification even in the json-file approach. The problem is that allowing patterns forces the builder to scan through the entirety of classpath and modulepath to determine for each individual entry in there if it matched the given pattern. It is a scalability issue.
But feel free to implemented that scanning through your resources on your side (e.g in your feature-code) and then call addResources for each element that you found.
ImageSingletons.lookup(SystemPropertiesSupport.class).initializeProperty("property.name", "value")
Would a META-INF/native-image/.../native-image.properties
with
Args = -Dproperty.name=value ...
also do the trick for your usecase?
Nice to have: BuildArtifacts.singleton().add(ArtifactType.JDK_LIB_SHIM, dll)
Are you relying on the output in file build_artifacts.txt
? @fniephaus is currently working on this and plans to get rid of it (by replacing it with equivalent information in some json file).
Allowing users to refer to arbitrary files in feature code and make them part of what constitutes the build-output is problematic and will conflict with replay-bundles. See https://github.com/oracle/graal/issues/5473
Re: https://github.com/oracle/graal/issues/4862#issuecomment-1260797988
This works in the same way as other JNI libraries are configured in GraalVM already (sunmscapi, sunec, prefs, mlib_image, lcms, ...) I don't think that's possible entirely through -H:NativeLinkerOption?
It's not. But I don't think we want to make this official API as this goes way beyond what we want our users to do with public API (remember, we have to support public API). In your case, I guess, it makes more sense to either, 1) keep using our internals or 2) contribute your features to our codebase so that using internals is fine. cc @christianwimmer @wirthi
But feel free to implemented that scanning through your resources on your side (e.g in your feature-code) and then call addResources for each element that you found.
That was the obvious alternative, but we didn't do that for now as there was an internal API for that. We'll implement this on our side instead.
Would a META-INF/native-image/.../native-image.properties with
Args = -Dproperty.name=value ...
also do the trick for your usecase?
Unless I'm missing something, that would be a build-time only argument. What I'm looking for (and how it is used in the Micronaut NettyFeature) is a way to configure the default value of the property for the runtime of the native executable. This also doesn't allow setting any lazilly computed values.
Are you relying on the output in file
build_artifacts.txt
?
Currently no. At this time we are blindly copying over all *.dll
and *.exe
files from the target folder (Maven). But since we have a custom feature that adds a few dlls to this output, it would've made sense to declare them as artifacts to native-image as well. But this was more as a general feature towards library developers which require native libraries, so they would have a proper official way of including these.
I don't think this would be problematic for replay-bundles, I would even argue that it would be necessary.
The replay bundle will contain all sources used for the original build anyway, so this includes the custom features that would add additional artifacts. So the replay would once again include these artifacts, as expected. The described image-building-as-a-service use-case would need to create an archive of all created artifacts for the user to download. So if native-image is not made aware of these additional artifacts, those would not be included.
What you're maybe referring to as problematic is if a user would try to include an arbitrary file from the host, which is not originating from one of the source bundles. On that I agree, that would be abuse of that specific API, but there is always room to misuse anything..
2) contribute your features to our codebase so that using internals is fine.
I'm certainly willing to do so, I however currently only have a working setup for Windows, and I remember from previous interactions (years ago at this point) that cross-platform support is preferred over single-platform support. And specifically for the fontmanager
and freetype
libraries I have no knowledge how to test them on linux/macos.
And then there's the problem that this requires some substitutions that are in our current setup very minimalistic and remove conficuration flexibility as we don't use it, but would probably still need to be configurable in an official supporting solution. However that's nothing that a good round of reviews shouldn't be able to fix.
There are other issues that discuss or relate to such a potential API: https://github.com/oracle/graal/issues/4796 https://github.com/oracle/graal/issues/4028 https://github.com/oracle/graal/issues/3359
Edit: I do believe a first version of any native library support will likely start as a custom feature before it can reach a point where it can be made into a PR to include support for everyone. So without official API it makes the road towards general support ever so slightly more cumbersome.
Unless I'm missing something, that would be a build-time only argument. What I'm looking for (and how it is used in the Micronaut NettyFeature) is a way to configure the default value of the property for the runtime of the native executable.
I see. It makes sense to add an API for that.
Unless I'm missing something, that would be a build-time only argument. What I'm looking for (and how it is used in the Micronaut NettyFeature) is a way to configure the default value of the property for the runtime of the native executable.
I see. It makes sense to add an API for that.
Created https://github.com/oracle/graal/issues/5557 for that.
Additional issue with contributing our fontmanager
feature for Windows. It works on standard native-image, but there are linker issues on Gluon substrate: https://github.com/gluonhq/substrate/issues/945
So for now we need to be able to switch between static and dynamic linking depending on if GluonFX is used or not.
In general anything awt related is a mess with javafx because we need the shim dlls, but substrate doesn't create them. So we need a second dummy project just to create the shims through the official build tools and provide a boatload of additional linker flags to substrate to make it work.
I don't see any obvious open issue here. #5557 that Paul created has been closed already.
Closing this ticket; if there is something concrete left to be done, please open a new ticket specifically for those parts.
Copied from https://github.com/oracle/graal/pull/4783#issuecomment-1225802940:
We have some usages that I think are not yet covered by https://github.com/oracle/graal/pull/4783 (workarounds are very welcome):
com.oracle.svm.core.SubstrateOptions
SubstrateOptions.Class
, as we have multiple entrypoints within a module, and need to switch config based on which one is used.com.oracle.svm.core.jdk.NativeLibrarySupport
,com.oracle.svm.hosted.c.NativeLibraries
andcom.oracle.svm.core.jdk.JNIRegistrationUtil
JNIRegistrationJava
.com.oracle.svm.core.jdk.Resources#registerResource(java.lang.String, java.lang.String, java.io.InputStream)
fontconfig.bfc
file to the resources, to be used from the substitution.com.oracle.svm.hosted.FeatureImpl.DuringSetupAccessImpl#registerClassReachabilityListener
com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization
/cc @olpaw