Simplify JNI configuration for polyglot use cases with Native Image

[nirvdrum]

Feature request

The preferred mechanism for using Native Image in embedded uses cases is to use JNI. A common use case is to make use of Truffle languages via GraalVM Polyglot. In order to use the API, it must be mapped as JNI config for the build. It would be very helpful if Native Image handled this out of the box. It already knows when a Truffle language is being included in the built image, so it could provide the mappings at that point. I'd also be happy with a flag to the native-image process.

Please include the following information:

Is your feature request related to a problem? Please describe.

When using JNI for polyglot embedded cases, I need to manually build the configuration the GraalVM Polyglot API. Currently I do it by running the application with libjvm and the tracing agent. It would be helpful to eliminate this step.

Describe the solution you'd like.

Native Image already has special handling for Truffle languages. I'd like to see that extended to supporting automatic JNI configuration for the GraalVM Polyglot API. I'm indifferent as to whether that happens automatically or with an extra flag.

Describe who do you think will benefit the most.

Anyone attempting to embed a Truffle language using Native Image.

Describe alternatives you've considered.

We could provide the configuration as a downloadable JSON file somewhere or perhaps included with the distribution. Currently, I run my application with libjvm andthe GraalVM tracing agent so it intercept any JNI calls and dump out the necessary JNI configuration.

Additional context.

This is a usability issue I ran into when building the Native Image Playground project.

[oubidar-Abderrahim]

Hi, Thank you for your suggestion, I'll make sure it reaches the appropriate team

[wirthi]

Hi @nirvdrum

thanks for your suggestion. I recently ran into a similar problem, so understand the pain.

You are limiting your request to JNI specifically; any reason for that? I think if we tackle that, any reasonable Polyglot usecase should be covered by default, right?

However, I think that most of the necessary configuration will actually stem from your application code. In the case of the bug I investigated, only 8 lines of ~40 lines in the necessary reflection config came from Polyglot itself, the rest was required for the few lines of usercode in the JavaScript application. My point is: yes, we can probably provide a default configuration for a reasonable Polyglot application, but that won't save you from providing additional configuration (via tracing agent, etc.) in all but the most trivial cases.

Christian

[nirvdrum]

I've only limited to JNI because that's the recommended approach to embedding Truffle languages. When using @CEntryPoint, everything was reachable. And when using the GraalVM Polyglot C API, I think everything necessary is mapped in libpolyglot already. I could see there being cases where mapping it for reflection would be helpful as well; I just didn't run into that. For my sample application, which only used a small subset of the GraalVM Polyglot functionality, I needed:

  {
    "name":"org.graalvm.polyglot.Context",
    "methods":[
      {"name":"eval","parameterTypes":["java.lang.String","java.lang.CharSequence"] },
      {"name":"newBuilder","parameterTypes":["java.lang.String[]"] }
    ]
  },
  {
    "name":"org.graalvm.polyglot.Context$Builder",
    "methods":[
      {"name":"allowExperimentalOptions","parameterTypes":["boolean"] },
      {"name":"build","parameterTypes":[] },
      {"name":"option","parameterTypes":["java.lang.String","java.lang.String"] }
    ]
  },
  {
    "name":"org.graalvm.polyglot.Value",
    "methods":[
      {"name":"asDouble","parameterTypes":[] },
      {"name":"execute","parameterTypes":["java.lang.Object[]"] }
    ]
  },

There was some additional JNI config needed for the app as well, mostly needed for argument processing (e.g., converting strings from argv to doubles), but I found that configuration fairly straightforward.

[chumer]

Yes, I also think this is a simple approach to this problem and this would make our polyglot C API obsolete, which is very maintenance heavy. @nirvdrum can you share what you built already? I think this could be a good starting point for the default exports.

[nirvdrum]

I'm sorry, I missed the notification for this. https://github.com/oracle/graal/issues/5036#issuecomment-1259953257 was the entirety of what I had for my limited example. I've just looked at embedding again and really would have like to just point at the Truffle libraries that ship with GraalVM (e.g., librubyvm.dylib). However, since they don't include JNI mappings for the GraalVM Polyglot API, I have to build my own library with a custom mapping of the GraalVM Polyglot API. I think it'd be nice if we just supported everything in Context and Value out of the box.

[eregon]

For maintenance maybe we should register at image build time all Context & Value & Source methods? Unless the NI config allows to register everything for a class? Not sure if we also need to specify the arguments types so they can be looked by class name or so.

[eregon]

Something like:

[
  {
    "name" : "org.graalvm.polyglot.Context",
    "allPublicMethods" : true,
  },
  {
    "name" : "org.graalvm.polyglot.Context$Builder",
    "allPublicMethods" : true,
  },
  {
    "name" : "org.graalvm.polyglot.Value",
    "allPublicMethods" : true,
  },
  {
    "name" : "org.graalvm.polyglot.Source",
    "allPublicMethods" : true,
  }
]

https://www.graalvm.org/latest/reference-manual/native-image/dynamic-features/Reflection/#manual-configuration could work, but we'd probably want all public classes directly in the org.graalvm.polyglot package (e.g. Engine & SourceSection are likely also useful).

[nirvdrum]

Ahh, I didn't know about allPublicMethods. That would be very helpful. I'd love to be able to use any of the Truffle images installed with gu in an embedded context without having to rebuild them.

[eregon]

@wirthi For the case of embedding a language just to expose eval, etc, and just reusing the libLANGvm.so existing library, I don't think anything besides embedder APIs need to be exposed, so org.graalvm.polyglot.* should be enough. E.g. @nirvdrum is trying to use librubyvm from a Rust webserver.

[chumer]

We could have a default reflection config like this for -macro:truffle-language-library We would need to extend it to all classes in polyglot, though. Then we could build a small example that shows how to use it from C using JNI.

What do you think @tzezula ?