search

oracle / graal

GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀
https://www.graalvm.org
Other
20.36k stars 1.63k forks source link

Image generation fails asking for runtime-initialization for class already added #6452

Open carlosedp opened 1 year ago

carlosedp commented 1 year ago

Describe the issue

My Scala project based on ZIO and Temporal uses multiple libraries that depend on Netty, GRPC and many more (dep tree at the end of the issue).

I'm trying to generate the GraalVM native image but I get an error about classes that should be initialized at runtime and although I added it to the native-image.properties file, I still get initialization errors.

Some of the parameters were obtained from TemporalIO Java SDK repo at https://github.com/temporalio/sdk-java/blob/master/temporal-serviceclient/src/main/resources/META-INF/native-image/io/temporal/temporal-serviceclient/native-image.properties.

There is a repro in the repository https://github.com/carlosedp/temporal-samples.

To run, use: scala-cli activity-retry/zio-temporal-activity-retry.scala to get the error on native-image, use scala-cli package --native-image -f activity-retry/zio-temporal-activity-retry.scala -o zio-temporal-retry.

The native-image params are at https://github.com/carlosedp/temporal-samples/blob/main/activity-retry/resources/META-INF/native-image/native-image.properties

When I comment-out the import //> using lib "dev.zio::zio-logging-slf4j2-bridge:2.1.12" which provides the SLF4J -> ZIO-logging bridge, the native-image build succeeds. I added a resolve tree for this module at the end of the issue.

Describe GraalVM and your environment:

openjdk version "17.0.6" 2023-01-17
OpenJDK Runtime Environment GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13, mixed mode, sharing)

MacOS Ventura, 13.3.1 (22E261) (Intel Mac)

Full run log (with trace for the class):

❯ scli package --native-image -f activity-retry/zio-temporal-activity-retry.scala -o zio-temporal-retry
========================================================================================================================
GraalVM Native Image: Generating 'zio-temporal-retry' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                   (16.2s @ 0.58GB)
 Version info: 'GraalVM 22.3.0 Java 17 CE'
 Java version info: '17.0.5+8-jvmci-22.3-b08'
 C compiler: cc (apple, x86_64, 14.0.3)
 Garbage collector: Serial GC
 2 user-specific feature(s)
 - com.oracle.svm.polyglot.scala.ScalaFeature
 - com.oracle.svm.thirdparty.gson.GsonFeature
[2/7] Performing analysis...  []                                                                        (33.3s @ 3.80GB)
  13,600 (94.60%) of 14,376 classes reachable
  23,000 (66.35%) of 34,664 fields reachable
  83,230 (85.10%) of 97,798 methods reachable
     509 classes,     0 fields, and     0 methods registered for reflection
       1 native library: -framework CoreServices

Error: Classes that should be initialized at run time got initialized during image building:
 io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder the class was requested to be initialized at run time (from 'META-INF/native-image/native-image.properties' in 'file:///private/var/folders/xq/m67gn0295cj54j6nlg3xndnr0000gn/T/zio-temporal-retry15774137957627939999.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder' and from 'META-INF/native-image/native-image.properties' in 'file:///Users/cdepaula/repos/scala-playground/zio/temporal-samples/activity-retry/resources/' with 'io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder' and from 'META-INF/native-image/io/temporal/temporal-serviceclient/native-image.properties' in 'file:///Users/cdepaula/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/temporal/temporal-serviceclient/1.19.1/temporal-serviceclient-1.19.1.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder'). io.grpc.netty.shaded.io.netty.util.internal.StringUtil caused initialization of this class with the following trace:
    at io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder.<clinit>(Slf4JLoggerFactory.java:63)
    at io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory.getInstanceWithNopCheck(Slf4JLoggerFactory.java:59)
    at io.grpc.netty.shaded.io.netty.util.internal.logging.InternalLoggerFactory.useSlf4JLoggerFactory(InternalLoggerFactory.java:62)
    at io.grpc.netty.shaded.io.netty.util.internal.logging.InternalLoggerFactory.newDefaultFactory(InternalLoggerFactory.java:42)
    at io.grpc.netty.shaded.io.netty.util.internal.logging.InternalLoggerFactory.getDefaultFactory(InternalLoggerFactory.java:111)
    at io.grpc.netty.shaded.io.netty.util.internal.logging.InternalLoggerFactory.getInstance(InternalLoggerFactory.java:134)
    at io.grpc.netty.shaded.io.netty.util.internal.logging.InternalLoggerFactory.getInstance(InternalLoggerFactory.java:127)
    at io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil.<clinit>(SystemPropertyUtil.java:31)
    at io.grpc.netty.shaded.io.netty.util.internal.StringUtil.<clinit>(StringUtil.java:32)

Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
------------------------------------------------------------------------------------------------------------------------
                        2.3s (4.3% of total time) in 21 GCs | Peak RSS: 5.41GB | CPU load: 6.23
========================================================================================================================
Failed generating 'zio-temporal-retry' after 49.8s.
Error: Image build request failed with exit status 1
Error: scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
For more details, please see '/Users/cdepaula/repos/scala-playground/zio/temporal-samples/activity-retry/.scala-build/stacktraces/1681938592-2771805065232649891.log'

The log contains:

❯ cat /Users/cdepaula/repos/scala-playground/zio/temporal-samples/activity-retry/.scala-build/stacktraces/1681938592-2771805065232649891.log
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
File: /Users/cdepaula/repos/scala-playground/zio/temporal-samples/activity-retry/.scala-build/stacktraces/1681938592-2771805065232649891.log
Size: 1.9 KB
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
  scala.cli.packaging.NativeImage$.buildNativeImage$$anonfun$1$$anonfun$1$$anonfun$1(NativeImage.scala:264)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.cli.packaging.NativeImage$.maybeWithShorterGraalvmHome(NativeImage.scala:158)
  scala.cli.packaging.NativeImage$.buildNativeImage$$anonfun$1$$anonfun$1(NativeImage.scala:264)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.build.internal.ManifestJar$.maybeWithManifestClassPath(ManifestJar.scala:82)
  scala.cli.packaging.NativeImage$.buildNativeImage$$anonfun$1(NativeImage.scala:267)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.cli.packaging.Library$.withLibraryJar(Library.scala:26)
  scala.cli.packaging.NativeImage$.buildNativeImage(NativeImage.scala:268)
  scala.cli.commands.package0.Package$.doPackage$$anonfun$1(Package.scala:408)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.cli.commands.package0.Package$.doPackage(Package.scala:521)
  scala.cli.commands.package0.Package$.runCommand(Package.scala:154)
  scala.cli.commands.package0.Package$.runCommand(Package.scala:70)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:365)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:352)
  caseapp.core.app.CaseApp.main(CaseApp.scala:150)
  scala.cli.commands.ScalaCommand.main(ScalaCommand.scala:337)
  caseapp.core.app.CommandsEntryPoint.main(CommandsEntryPoint.scala:120)
  scala.cli.ScalaCliCommands.main(ScalaCliCommands.scala:124)
  scala.cli.ScalaCli$.main0(ScalaCli.scala:203)
  scala.cli.ScalaCli$.main(ScalaCli.scala:107)
  scala.cli.ScalaCli.main(ScalaCli.scala)

Resolve tree for "dev.zio::zio-logging-slf4j2-bridge:2.1.12":

❯ cs resolve -t dev.zio::zio-logging-slf4j2-bridge:2.1.12
  Result:
└─ dev.zio:zio-logging-slf4j2-bridge_3:2.1.12
   ├─ dev.zio:zio-logging_3:2.1.12
   │  ├─ dev.zio:zio-parser_3:0.1.9
   │  │  ├─ dev.zio:zio-streams_3:2.0.10 -> 2.0.11
   │  │  │  ├─ dev.zio:zio_3:2.0.11
   │  │  │  │  ├─ dev.zio:izumi-reflect_3:2.3.1
   │  │  │  │  │  ├─ dev.zio:izumi-reflect-thirdparty-boopickle-shaded_3:2.3.1
   │  │  │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  │  ├─ dev.zio:zio-internal-macros_3:2.0.11
   │  │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  │  ├─ dev.zio:zio-stacktracer_3:2.0.11
   │  │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  ├─ dev.zio:zio_3:2.0.10 -> 2.0.11
   │  │  │  ├─ dev.zio:izumi-reflect_3:2.3.1
   │  │  │  │  ├─ dev.zio:izumi-reflect-thirdparty-boopickle-shaded_3:2.3.1
   │  │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  ├─ dev.zio:zio-internal-macros_3:2.0.11
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  ├─ dev.zio:zio-stacktracer_3:2.0.11
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │     └─ org.scala-lang:scala-library:2.13.10
   │  ├─ dev.zio:zio-streams_3:2.0.11
   │  │  ├─ dev.zio:zio_3:2.0.11
   │  │  │  ├─ dev.zio:izumi-reflect_3:2.3.1
   │  │  │  │  ├─ dev.zio:izumi-reflect-thirdparty-boopickle-shaded_3:2.3.1
   │  │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  ├─ dev.zio:zio-internal-macros_3:2.0.11
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  ├─ dev.zio:zio-stacktracer_3:2.0.11
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │     └─ org.scala-lang:scala-library:2.13.10
   │  ├─ dev.zio:zio_3:2.0.11
   │  │  ├─ dev.zio:izumi-reflect_3:2.3.1
   │  │  │  ├─ dev.zio:izumi-reflect-thirdparty-boopickle-shaded_3:2.3.1
   │  │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  ├─ dev.zio:zio-internal-macros_3:2.0.11
   │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  ├─ dev.zio:zio-stacktracer_3:2.0.11
   │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │     └─ org.scala-lang:scala-library:2.13.10
   │  └─ org.scala-lang:scala3-library_3:3.2.2
   │     └─ org.scala-lang:scala-library:2.13.10
   ├─ dev.zio:zio_3:2.0.11
   │  ├─ dev.zio:izumi-reflect_3:2.3.1
   │  │  ├─ dev.zio:izumi-reflect-thirdparty-boopickle-shaded_3:2.3.1
   │  │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │  │     └─ org.scala-lang:scala-library:2.13.10
   │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │     └─ org.scala-lang:scala-library:2.13.10
   │  ├─ dev.zio:zio-internal-macros_3:2.0.11
   │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │     └─ org.scala-lang:scala-library:2.13.10
   │  ├─ dev.zio:zio-stacktracer_3:2.0.11
   │  │  └─ org.scala-lang:scala3-library_3:3.2.2
   │  │     └─ org.scala-lang:scala-library:2.13.10
   │  └─ org.scala-lang:scala3-library_3:3.2.2
   │     └─ org.scala-lang:scala-library:2.13.10
   ├─ org.scala-lang:scala3-library_3:3.2.2
   │  └─ org.scala-lang:scala-library:2.13.10
   ├─ org.scala-lang.modules:scala-collection-compat_3:2.8.1
   │  └─ org.scala-lang:scala3-library_3:3.1.3 -> 3.2.2 (possible incompatibility)
   │     └─ org.scala-lang:scala-library:2.13.10
   └─ org.slf4j:slf4j-api:2.0.7
zakkak commented 1 year ago

@carlosedp what the error says is that although you are asking GraalVM to initialize io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder at runtime some classes not registered for runtime initialization depend on it and thus try to initialize it at build time.

Registering io.grpc.netty.shaded.io.netty.util.internal.StringUtil for runtime initialization as well should work around this.

carlosedp commented 1 year ago

Well, it worked progressing thru it... after adding:

--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.logging.Log4JLogger \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.internal.tcnative \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.handler.ssl \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.AsciiString \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.AbstractChannel \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.StringUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.MultithreadEventLoopGroup \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.Unpooled.wrappedBuffer \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.Unpooled \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.socket.nio.SelectorProviderUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.CleanerJava6 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetectorFactory \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer \
--trace-class-initialization=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector

I got to the point of the error:

❯ scli package --native-image -f activity-retry/zio-temporal-activity-retry.scala -o zio-temporal-retry
========================================================================================================================
GraalVM Native Image: Generating 'zio-temporal-retry' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                   (13.8s @ 0.61GB)
 Version info: 'GraalVM 22.3.0 Java 17 CE'
 Java version info: '17.0.5+8-jvmci-22.3-b08'
 C compiler: cc (apple, x86_64, 14.0.3)
 Garbage collector: Serial GC
 2 user-specific feature(s)
 - com.oracle.svm.polyglot.scala.ScalaFeature
 - com.oracle.svm.thirdparty.gson.GsonFeature
[2/7] Performing analysis...  []                                                                        (28.3s @ 3.76GB)
  11,485 (93.55%) of 12,277 classes reachable
  18,973 (65.51%) of 28,964 fields reachable
  65,511 (89.01%) of 73,602 methods reachable
     493 classes,     0 fields, and     0 methods registered for reflection
       1 native library: -framework CoreServices

Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(int, int)
Parsing context:
   at io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:391)
   at io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
   at io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
   at io.grpc.netty.shaded.io.netty.handler.ssl.PemX509Certificate.newBuffer(PemX509Certificate.java:152)
   at io.grpc.netty.shaded.io.netty.handler.ssl.PemX509Certificate.append(PemX509Certificate.java:112)
   at io.grpc.netty.shaded.io.netty.handler.ssl.PemX509Certificate.toPEM(PemX509Certificate.java:84)
   at io.grpc.netty.shaded.io.netty.handler.ssl.ReferenceCountedOpenSslContext.toBIO(ReferenceCountedOpenSslContext.java:922)
   at io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:276)

    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:153)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:104)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:83)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraph(MethodTypeFlow.java:65)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultVirtualInvokeTypeFlow.java:109)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:562)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:488)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector are allowed in the image heap as this class should be initialized at image runtime. To see how this object got instantiated use --trace-object-instantiation=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector.
    at parsing io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:51)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2518)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:110)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3393)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3345)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3190)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1138)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1030)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:97)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:84)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:446)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:135)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:685)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.lookupEncodedGraph(InlineBeforeAnalysis.java:180)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.doInline(PEGraphDecoder.java:1162)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.tryInline(PEGraphDecoder.java:1145)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.trySimplifyInvoke(PEGraphDecoder.java:1003)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.handleInvoke(PEGraphDecoder.java:957)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:817)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:240)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:548)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:833)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:98)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:179)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:349)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:93)
    ... 13 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector are allowed in the image heap as this class should be initialized at image runtime. To see how this object got instantiated use --trace-object-instantiation=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector.
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.checkImageHeapInstance(ClassInitializationFeature.java:132)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:595)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:177)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:148)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readValue(AnalysisConstantReflectionProvider.java:100)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readFieldValue(AnalysisConstantReflectionProvider.java:79)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.readValue(ConstantFoldUtil.java:55)
    at jdk.internal.vm.compiler/org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:78)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:72)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:51)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.tryConstantFold(ConstantFoldLoadFieldPlugin.java:53)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.handleLoadStaticField(ConstantFoldLoadFieldPlugin.java:49)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4801)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4772)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5282)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3385)
    ... 38 more
------------------------------------------------------------------------------------------------------------------------
                        1.3s (3.0% of total time) in 19 GCs | Peak RSS: 4.02GB | CPU load: 6.40
========================================================================================================================
Failed generating 'zio-temporal-retry' after 42.6s.
Error: Image build request failed with exit status 1
Error: scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
For more details, please see '/Users/cdepaula/repos/scala-playground/zio/temporal-samples/activity-retry/.scala-build/stacktraces/1682345185-4811752098193188954.log'

This is very painful to do since you have to go error by error, adding classes to the native-image.properties file, re-run and get the new error for a long time (never know exactly when it will be done).

Any idea on how to progress?

zakkak commented 1 year ago

AFAIK there is no other way.

Out of curiosity what led you to add the --initialize-at-build-time parameters? Does it compile without any --initialize-at-build-time and --initialize-at-run-time parameters?

carlosedp commented 1 year ago

I got it from Temporal's own SDK at https://github.com/temporalio/sdk-java/blob/master/temporal-serviceclient/src/main/resources/META-INF/native-image/io/temporal/temporal-serviceclient/native-image.properties

It doesn't compile with the basic config I posted below... I get the Slf4JLoggerFactory$NopInstanceHolder error like below. I'll try to start from scratch, removing all initialize params and go one by one... problem is that it's very, very time consuming.

I generated the reflection config with:

❯ scli activity-retry/zio-temporal-activity-retry.scala -J -agentlib:native-image-agent=config-output-dir=resources/META-INF/native-image

And configured the native-image.properties as:

Args = --no-fallback \
-H:DynamicProxyConfigurationResources=${.}/proxy-config.json \
-H:JNIConfigurationResources=${.}/jni-config.json \
-H:ReflectionConfigurationResources=${.}/reflect-config.json \
-H:ResourceConfigurationResources=${.}/resource-config.json \
-H:SerializationConfigurationResources=${.}/serialization-config.json \
--enable-http \
--enable-url-protocols=http,https \
--install-exit-handlers \
-Djdk.http.auth.tunneling.disabledSchemes="" \

This is the full execution:

❯ scli package --native-image -f activity-retry/zio-temporal-activity-retry.scala -o zio-temporal-retry
Compiling project (Scala 3.3.0-RC4, JVM)
Compiled project (Scala 3.3.0-RC4, JVM)

========================================================================================================================
GraalVM Native Image: Generating 'zio-temporal-retry' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                   (12.8s @ 0.57GB)
 Version info: 'GraalVM 22.3.0 Java 17 CE'
 Java version info: '17.0.5+8-jvmci-22.3-b08'
 C compiler: cc (apple, x86_64, 14.0.3)
 Garbage collector: Serial GC
 2 user-specific feature(s)
 - com.oracle.svm.polyglot.scala.ScalaFeature
 - com.oracle.svm.thirdparty.gson.GsonFeature
[2/7] Performing analysis...  []                                                                        (40.5s @ 3.55GB)
  13,572 (94.57%) of 14,351 classes reachable
  22,947 (66.31%) of 34,605 fields reachable
  83,082 (85.08%) of 97,651 methods reachable
     509 classes,     0 fields, and     0 methods registered for reflection
       1 native library: -framework CoreServices

Error: Classes that should be initialized at run time got initialized during image building:
 io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder the class was requested to be initialized at run time (from 'META-INF/native-image/io/temporal/temporal-serviceclient/native-image.properties' in 'file:///Users/cdepaula/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/temporal/temporal-serviceclient/1.19.1/temporal-serviceclient-1.19.1.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder'). To see why io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder
To see how the classes got initialized, use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder
Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
------------------------------------------------------------------------------------------------------------------------
                        3.7s (6.7% of total time) in 21 GCs | Peak RSS: 5.04GB | CPU load: 6.09
========================================================================================================================
Failed generating 'zio-temporal-retry' after 53.7s.
Error: Image build request failed with exit status 1
Error: scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
For more details, please see '/Users/cdepaula/repos/scala/temporal-samples/activity-retry/.scala-build/stacktraces/1682432422-11290098331082673205.log'
carlosedp commented 1 year ago

BTW, without the SLF4J lib it builds without any --initialize... param.

carlosedp commented 1 year ago

Funny enough @zakkak , I went on adding the required initialization params... when I got to:

Args = --no-fallback \
-H:DynamicProxyConfigurationResources=${.}/proxy-config.json \
-H:JNIConfigurationResources=${.}/jni-config.json \
-H:ReflectionConfigurationResources=${.}/reflect-config.json \
-H:ResourceConfigurationResources=${.}/resource-config.json \
-H:SerializationConfigurationResources=${.}/serialization-config.json \
--enable-http \
--enable-url-protocols=http,https \
--install-exit-handlers \
-Djdk.http.auth.tunneling.disabledSchemes="" \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.StringUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.AsciiString \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioByteChannel \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.AbstractChannel \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.MultithreadEventLoopGroup \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator \
--trace-object-instantiation=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator

It required to add a --initialize-at-build-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.

With the full trace:

❯ scli package --native-image -f activity-retry/zio-temporal-activity-retry.scala -o zio-temporal-retry
========================================================================================================================
GraalVM Native Image: Generating 'zio-temporal-retry' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                   (12.2s @ 0.60GB)
 Version info: 'GraalVM 22.3.0 Java 17 CE'
 Java version info: '17.0.5+8-jvmci-22.3-b08'
 C compiler: cc (apple, x86_64, 14.0.3)
 Garbage collector: Serial GC
 2 user-specific feature(s)
 - com.oracle.svm.polyglot.scala.ScalaFeature
 - com.oracle.svm.thirdparty.gson.GsonFeature
[2/7] Performing analysis...  []                                                                        (24.1s @ 2.76GB)
  10,431 (92.21%) of 11,312 classes reachable
  14,926 (59.77%) of 24,973 fields reachable
  45,577 (83.15%) of 54,812 methods reachable
     491 classes,     0 fields, and     0 methods registered for reflection
       1 native library: -framework CoreServices

Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>()
Parsing context: <no parsing context available>

    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:153)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:104)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:83)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraph(MethodTypeFlow.java:65)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultAnalysisPolicy.staticRootMethodGraph(DefaultAnalysisPolicy.java:182)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis.lambda$addRootMethod$0(PointsToAnalysis.java:320)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$2.run(PointsToAnalysis.java:507)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the io.grpc.netty.shaded.io.netty.buffer.Unpooled class initializer with a trace:
    at io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.<init>(UnpooledByteBufAllocator.java:46)
    at io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.<clinit>(UnpooledByteBufAllocator.java:37)
    at io.grpc.netty.shaded.io.netty.buffer.Unpooled.<clinit>(Unpooled.java:75)
.  To fix the issue mark io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator for build-time initialization with --initialize-at-build-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.

    at parsing io.grpc.netty.shaded.io.netty.buffer.Unpooled.wrappedBuffer(Unpooled.java:160)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2518)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:110)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3393)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3345)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3190)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1138)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1030)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:97)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:84)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:446)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
    at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:135)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:685)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.lookupEncodedGraph(InlineBeforeAnalysis.java:180)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.doInline(PEGraphDecoder.java:1162)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.tryInline(PEGraphDecoder.java:1145)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.trySimplifyInvoke(PEGraphDecoder.java:1003)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.handleInvoke(PEGraphDecoder.java:957)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:817)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:240)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:548)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:833)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:98)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:179)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:349)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:93)
    ... 13 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the io.grpc.netty.shaded.io.netty.buffer.Unpooled class initializer with a trace:
    at io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.<init>(UnpooledByteBufAllocator.java:46)
    at io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.<clinit>(UnpooledByteBufAllocator.java:37)
    at io.grpc.netty.shaded.io.netty.buffer.Unpooled.<clinit>(Unpooled.java:75)
.  To fix the issue mark io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator for build-time initialization with --initialize-at-build-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.

    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.checkImageHeapInstance(ClassInitializationFeature.java:132)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:595)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:177)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:148)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readValue(AnalysisConstantReflectionProvider.java:100)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readFieldValue(AnalysisConstantReflectionProvider.java:79)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.readValue(ConstantFoldUtil.java:55)
    at jdk.internal.vm.compiler/org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:78)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:72)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:51)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.tryConstantFold(ConstantFoldLoadFieldPlugin.java:53)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.ConstantFoldLoadFieldPlugin.handleLoadStaticField(ConstantFoldLoadFieldPlugin.java:49)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4801)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genGetStatic(BytecodeParser.java:4772)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5282)
    at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3385)
    ... 38 more
------------------------------------------------------------------------------------------------------------------------
                        1.2s (3.1% of total time) in 18 GCs | Peak RSS: 3.54GB | CPU load: 5.98
========================================================================================================================
Failed generating 'zio-temporal-retry' after 36.6s.
Error: Image build request failed with exit status 1
Error: scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
For more details, please see '/Users/cdepaula/repos/scala/temporal-samples/activity-retry/.scala-build/stacktraces/1682434154-3017510009471933864.log'
carlosedp commented 1 year ago

I've continued adding the UnpooledByteBufAllocator at runtime since adding at build-time didn't work... got to:

Args = --no-fallback \
-H:DynamicProxyConfigurationResources=${.}/proxy-config.json \
-H:JNIConfigurationResources=${.}/jni-config.json \
-H:ReflectionConfigurationResources=${.}/reflect-config.json \
-H:ResourceConfigurationResources=${.}/resource-config.json \
-H:SerializationConfigurationResources=${.}/serialization-config.json \
--enable-http \
--enable-url-protocols=http,https \
--install-exit-handlers \
-Djdk.http.auth.tunneling.disabledSchemes="" \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.StringUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.AsciiString \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioByteChannel \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.AbstractChannel \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.MultithreadEventLoopGroup \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.Unpooled \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.AbstractByteBuf \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetectorFactory \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.CleanerJava6 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.Recycler$2 \
--trace-object-instantiation=io.grpc.netty.shaded.io.netty.util.Recycler$2

And now I get the error:

❯ scli package --native-image -f activity-retry/zio-temporal-activity-retry.scala -o zio-temporal-retry
========================================================================================================================
GraalVM Native Image: Generating 'zio-temporal-retry' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                   (11.4s @ 0.42GB)
 Version info: 'GraalVM 22.3.0 Java 17 CE'
 Java version info: '17.0.5+8-jvmci-22.3-b08'
 C compiler: cc (apple, x86_64, 14.0.3)
 Garbage collector: Serial GC
 2 user-specific feature(s)
 - com.oracle.svm.polyglot.scala.ScalaFeature
 - com.oracle.svm.thirdparty.gson.GsonFeature
[2/7] Performing analysis...  []                                                                        (29.8s @ 1.16GB)
  12,449 (93.83%) of 13,268 classes reachable
  21,066 (64.82%) of 32,500 fields reachable
  73,054 (89.43%) of 81,693 methods reachable
     507 classes,     0 fields, and     0 methods registered for reflection
       1 native library: -framework CoreServices

Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache.add(io.grpc.netty.shaded.io.netty.buffer.PoolArena, io.grpc.netty.shaded.io.netty.buffer.PoolChunk, java.nio.ByteBuffer, long, int, io.grpc.netty.shaded.io.netty.buffer.PoolArena$SizeClass)
Parsing context:
   at io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache.add(PoolThreadCache.java:182)
   at io.grpc.netty.shaded.io.netty.buffer.PoolArena.free(PoolArena.java:228)
   at io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf.deallocate(PooledByteBuf.java:176)
   at io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf.handleRelease(AbstractReferenceCountedByteBuf.java:111)
   at io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:101)
   at io.grpc.netty.shaded.io.netty.handler.ssl.ReferenceCountedOpenSslContext.toBIO(ReferenceCountedOpenSslContext.java:952)
   at io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:279)

    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:153)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:104)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:83)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraph(MethodTypeFlow.java:65)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultSpecialInvokeTypeFlow.onObservedUpdate(DefaultSpecialInvokeTypeFlow.java:61)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:562)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:488)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of io.grpc.netty.shaded.io.netty.util.Recycler$2 are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache$MemoryRegionCache class initializer with a trace:
    at io.grpc.netty.shaded.io.netty.util.Recycler$2.<init>(Recycler.java:96)
    at io.grpc.netty.shaded.io.netty.util.Recycler.<init>(Recycler.java:96)
    at io.grpc.netty.shaded.io.netty.util.Recycler.<init>(Recycler.java:116)
    at io.grpc.netty.shaded.io.netty.util.Recycler.<init>(Recycler.java:112)
    at io.grpc.netty.shaded.io.netty.util.internal.ObjectPool$RecyclerObjectPool$1.<init>(ObjectPool.java:74)
    at io.grpc.netty.shaded.io.netty.util.internal.ObjectPool$RecyclerObjectPool.<init>(ObjectPool.java:74)
    at io.grpc.netty.shaded.io.netty.util.internal.ObjectPool.newPool(ObjectPool.java:67)
    at io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache$MemoryRegionCache.<clinit>(PoolThreadCache.java:465)
.  To fix the issue mark io.grpc.netty.shaded.io.netty.util.Recycler$2 for build-time initialization with --initialize-at-build-time=io.grpc.netty.shaded.io.netty.util.Recycler$2 or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.

    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.checkImageHeapInstance(ClassInitializationFeature.java:132)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:595)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:177)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:148)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readValue(AnalysisConstantReflectionProvider.java:100)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.readFieldValue(AnalysisConstantReflectionProvider.java:79)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil$1.readValue(ConstantFoldUtil.java:55)
    at jdk.internal.vm.compiler/org.graalvm.compiler.core.common.spi.JavaConstantFieldProvider.readConstantField(JavaConstantFieldProvider.java:78)
    at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ameta.AnalysisConstantFieldProvider.readConstantField(AnalysisConstantFieldProvider.java:72)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.ConstantFoldUtil.tryConstantFold(ConstantFoldUtil.java:51)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.asConstant(LoadFieldNode.java:178)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:144)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:135)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.java.LoadFieldNode.canonical(LoadFieldNode.java:72)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.spi.Canonicalizable$Unary.canonical(Canonicalizable.java:101)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.SimplifyingGraphDecoder.canonicalizeFixedNode(SimplifyingGraphDecoder.java:214)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.canonicalizeFixedNode(PEGraphDecoder.java:1572)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.canonicalizeFixedNode(InlineBeforeAnalysis.java:192)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.SimplifyingGraphDecoder.handleFixedNode(SimplifyingGraphDecoder.java:193)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:821)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:240)
    at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:548)
    at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:833)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:98)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:179)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:349)
    at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:93)
    ... 13 more
------------------------------------------------------------------------------------------------------------------------
                        2.5s (5.9% of total time) in 24 GCs | Peak RSS: 4.61GB | CPU load: 6.96
========================================================================================================================
Failed generating 'zio-temporal-retry' after 41.4s.
Error: Image build request failed with exit status 1
Error: scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
For more details, please see '/Users/cdepaula/repos/scala/temporal-samples/activity-retry/.scala-build/stacktraces/1682437397-13338046594007349972.log'
carlosedp commented 1 year ago

And after all I got stuck with no apparent solution going forward... my config is:

Args = --no-fallback \
-H:DynamicProxyConfigurationResources=${.}/proxy-config.json \
-H:JNIConfigurationResources=${.}/jni-config.json \
-H:ReflectionConfigurationResources=${.}/reflect-config.json \
-H:ResourceConfigurationResources=${.}/resource-config.json \
-H:SerializationConfigurationResources=${.}/serialization-config.json \
--enable-http \
--enable-url-protocols=http,https \
--install-exit-handlers \
-Djdk.http.auth.tunneling.disabledSchemes="" \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.StringUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.AsciiString \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioByteChannel \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.AbstractChannel \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.channel.MultithreadEventLoopGroup \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.Unpooled \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.AbstractByteBuf \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetectorFactory \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.CleanerJava6 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.Recycler$2 \
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache$MemoryRegionCache
--initialize-at-run-time=io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder \
--trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent,io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector,io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil,io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0,io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9,io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder,io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal

and I get:

❯ scli package --native-image -f activity-retry/zio-temporal-activity-retry.scala -o zio-temporal-retry
========================================================================================================================
GraalVM Native Image: Generating 'zio-temporal-retry' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                   (12.0s @ 0.59GB)
 Version info: 'GraalVM 22.3.0 Java 17 CE'
 Java version info: '17.0.5+8-jvmci-22.3-b08'
 C compiler: cc (apple, x86_64, 14.0.3)
 Garbage collector: Serial GC
 2 user-specific feature(s)
 - com.oracle.svm.polyglot.scala.ScalaFeature
 - com.oracle.svm.thirdparty.gson.GsonFeature
[2/7] Performing analysis...  []                                                                        (34.7s @ 3.43GB)
  13,622 (94.64%) of 14,394 classes reachable
  23,099 (66.57%) of 34,698 fields reachable
  83,490 (85.16%) of 98,043 methods reachable
     511 classes,     0 fields, and     0 methods registered for reflection
       1 native library: -framework CoreServices

Error: Classes that should be initialized at run time got initialized during image building:
 io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent the class was requested to be initialized at run time (from 'META-INF/native-image/native-image.properties' in 'file:///private/var/folders/xq/m67gn0295cj54j6nlg3xndnr0000gn/T/zio-temporal-retry15112916426091332441.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent' and from 'META-INF/native-image/native-image.properties' in 'file:///Users/cdepaula/repos/scala/temporal-samples/resources/' with 'io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent'). To see why io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent
io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0 the class was requested to be initialized at run time (from 'META-INF/native-image/native-image.properties' in 'file:///private/var/folders/xq/m67gn0295cj54j6nlg3xndnr0000gn/T/zio-temporal-retry15112916426091332441.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0' and from 'META-INF/native-image/native-image.properties' in 'file:///Users/cdepaula/repos/scala/temporal-samples/resources/' with 'io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0'). To see why io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0 got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0
io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9 the class was requested to be initialized at run time (from 'META-INF/native-image/native-image.properties' in 'file:///private/var/folders/xq/m67gn0295cj54j6nlg3xndnr0000gn/T/zio-temporal-retry15112916426091332441.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9' and from 'META-INF/native-image/native-image.properties' in 'file:///Users/cdepaula/repos/scala/temporal-samples/resources/' with 'io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9'). To see why io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9 got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9
io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector the class was requested to be initialized at run time (from 'META-INF/native-image/native-image.properties' in 'file:///private/var/folders/xq/m67gn0295cj54j6nlg3xndnr0000gn/T/zio-temporal-retry15112916426091332441.jar' with 'io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector' and from 'META-INF/native-image/native-image.properties' in 'file:///Users/cdepaula/repos/scala/temporal-samples/resources/' with 'io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector'). To see why io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector
io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder the class was requested to be initialized at run time (from 'META-INF/native-image/io/temporal/temporal-serviceclient/native-image.properties' in 'file:///Users/cdepaula/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/temporal/temporal-serviceclient/1.19.1/temporal-serviceclient-1.19.1.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder'). To see why io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder
io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil the class was requested to be initialized at run time (from 'META-INF/native-image/native-image.properties' in 'file:///private/var/folders/xq/m67gn0295cj54j6nlg3xndnr0000gn/T/zio-temporal-retry15112916426091332441.jar' with 'io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil' and from 'META-INF/native-image/native-image.properties' in 'file:///Users/cdepaula/repos/scala/temporal-samples/resources/' with 'io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil'). To see why io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil
io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal the class was requested to be initialized at run time (from 'META-INF/native-image/native-image.properties' in 'file:///private/var/folders/xq/m67gn0295cj54j6nlg3xndnr0000gn/T/zio-temporal-retry15112916426091332441.jar' with 'io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal' and from 'META-INF/native-image/native-image.properties' in 'file:///Users/cdepaula/repos/scala/temporal-samples/resources/' with 'io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal'). To see why io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal
To see how the classes got initialized, use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent,io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0,io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9,io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector,io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder,io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil,io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal
Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
------------------------------------------------------------------------------------------------------------------------
                        2.9s (6.0% of total time) in 21 GCs | Peak RSS: 5.29GB | CPU load: 6.66
========================================================================================================================
Failed generating 'zio-temporal-retry' after 47.0s.
Error: Image build request failed with exit status 1
Error: scala.cli.errors.GraalVMNativeImageError: Error building native image with GraalVM
For more details, please see '/Users/cdepaula/repos/scala/temporal-samples/activity-retry/.scala-build/stacktraces/1682439272-12004488857185520744.log'

Sorry about the lenght of this thread but I couldn't find any information somewhere else to help with this.

zakkak commented 1 year ago

I got it from Temporal's own SDK at https://github.com/temporalio/sdk-java/blob/master/temporal-serviceclient/src/main/resources/META-INF/native-image/io/temporal/temporal-serviceclient/native-image.properties

OK, I see. So temporal comes with some kind of native-image support but it looks like this configuration is not complete. I suggest reporting this issue at https://github.com/temporalio/sdk-java/issues

Further notes

I've continued adding the UnpooledByteBufAllocator at runtime since adding at build-time didn't work...

And actually that's the right thing to do IMO.

Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the io.grpc.netty.shaded.io.netty.buffer.Unpooled class initializer with a trace:
    at io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.<init>(UnpooledByteBufAllocator.java:46)
    at io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.<clinit>(UnpooledByteBufAllocator.java:37)
    at io.grpc.netty.shaded.io.netty.buffer.Unpooled.<clinit>(Unpooled.java:75)
.  To fix the issue mark io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator for build-time initialization with --initialize-at-build-time=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.

Is misleading. Note that the error message says that "No instances of io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator are allowed in the image heap as this class should be initialized at image runtime"

That said the suggested by GraalVM solution to mark UnpooledByteBufAllocator for build time initialization is not right IMO. Instead you should try to avoid getting an instance of it instantiated at build time. Based on the trace above to achieve this you would need to mark io.grpc.netty.shaded.io.netty.buffer.Unpooled for runtime initialization.

And after all I got stuck with no apparent solution going forward...

At this point I guess you tried --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent,io.grpc.netty.shaded.io.netty.util.internal.PlatformDependent0,io.grpc.netty.shaded.io.netty.util.internal.CleanerJava9,io.grpc.netty.shaded.io.netty.util.ResourceLeakDetector,io.grpc.netty.shaded.io.netty.util.internal.logging.Slf4JLoggerFactory$NopInstanceHolder,io.grpc.netty.shaded.io.netty.util.internal.SystemPropertyUtil,io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal but adding further initialize-at-run-time and initialize-at-build-time parameters didn't help.

Note that this could mean that netty is not native-image friendly and it might also require some "substitutions" to make it work. For instance Quarkus apart from the configuration it does regarding initialization etc in https://github.com/quarkusio/quarkus/blob/main/extensions/netty/deployment/src/main/java/io/quarkus/netty/deployment/NettyProcessor.java it also performs a number of substitutions in https://github.com/quarkusio/quarkus/blob/main/extensions/netty/runtime/src/main/java/io/quarkus/netty/runtime/graal/NettySubstitutions.java

Disclaimer: The Quarkus configuration and substitutions are not necessarily required outside of Quarkus since Quarkus in contrast to the default GraalVM CE behavior registers all classes for build-time initialization.

These kind of changes and parameterization are not expected to be performed by applications developers. Instead they are expected to be provided by the libraries themselves or from frameworks (like Quarkus) supporting native compilation. As a result, I suggest opening an issue on netty or temporalio, requesting better native-image support or using a framework that already supports the libraries you need.

carlosedp commented 1 year ago

Finally I took some time, started over and got it working. Enabled the SLF4J lib and after a lot of iteration I got the native-image built. Here's the commit I added it:

https://github.com/carlosedp/zio-temporal-hello/commit/b5611757acded3219618510d443d24060fd51b16

Now I don't know where to report this to. zio-logging (which is the lib I use but it's mostly a wrapper), slf4j or netty since it was the most required init settings I needed to add.