micronaut-projects / micronaut-core

Micronaut Application Framework
http://micronaut.io
Apache License 2.0
6k stars 1.04k forks source link

Unable to run a native image on GCP (Cloud Run) #3042

Open gbhojak opened 4 years ago

gbhojak commented 4 years ago

Hi everyone,

I am trying to compile a simple hello-world application to a native image on GCP. I have added firebase-admin sdk provided by GCP to access their cloud firestore database.

Would really appreciate any help here. We are trying to use micronaut to create at least 4-5 microservices to run on Cloud Run, hence would like to leverage the fast startup times and reduced memory footprint.

Steps to Reproduce

  1. mn create-app hello-world --features graal-native-image

  2. Added the dependency for firebase-admin sdk to pom.xml

    <dependency>
      <groupId>com.google.firebase</groupId>
      <artifactId>firebase-admin</artifactId>
      <version>6.12.2</version>
    </dependency>
  1. Submitted a build to GCP using docker file created by 'mn' command

I tried to follow the recommendations mentioned here and here

Here are the contents of native-image.properties:

Args = -H:IncludeResources=logback.xml|application.yml|bootstrap.yml|reflect-config.json \
       -H:Name=hello-world \
       -H:Class=com.company.samplepackage.Application \
       -H:+ReportExceptionStackTraces \
       --report-unsupported-elements-at-runtime \
       -H:ReflectionConfigurationFiles=target/classes/reflect-config.json \
       --initialize-at-run-time=io.netty.handler.codec.http2.Http2ClientUpgradeCodec \
       --initialize-at-run-time=com.google.protobuf.ExtensionRegistryFactory

and reflect-config.json


[
  {
    "name": "io.grpc.netty.shaded.io.netty.channel.socket.nio.NioSocketChannel",
    "methods": [
      {
        "name": "<init>",
        "parameterTypes": []
      }
    ]
  }
]

Expected Behaviour

Should be able to invoke the 'hello world' api

Actual Behaviour


A 2020-04-09T23:44:28.888369Z   ... 41 common frames omitted 
A 2020-04-09T23:44:28.888362Z   at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1693) 
A 2020-04-09T23:44:28.888352Z   at com.company.samplepackage.$FirebaseBeanDefinition.build(Unknown Source) 
A 2020-04-09T23:44:28.888330Z   at com.company.samplepackage.FirebaseBean.<init>(FirebaseBean.java:23) 
A 2020-04-09T23:44:28.888323Z   at com.google.cloud.ServiceOptions.getService(ServiceOptions.java:518) 
A 2020-04-09T23:44:28.888316Z   at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreFactory.create(FirestoreOptions.java:66) 
A 2020-04-09T23:44:28.888305Z   at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreFactory.create(FirestoreOptions.java:73) 
A 2020-04-09T23:44:28.888273Z   at com.google.cloud.firestore.FirestoreImpl.<init>(FirestoreImpl.java:77) 
A 2020-04-09T23:44:28.888266Z   at com.google.cloud.firestore.FirestoreOptions.getFirestoreRpc(FirestoreOptions.java:385) 
A 2020-04-09T23:44:28.888259Z   at com.google.cloud.ServiceOptions.getRpc(ServiceOptions.java:538) 
A 2020-04-09T23:44:28.888247Z   at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreRpcFactory.create(FirestoreOptions.java:82) 
A 2020-04-09T23:44:28.888239Z   at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreRpcFactory.create(FirestoreOptions.java:90) 
A 2020-04-09T23:44:28.888227Z   at com.google.cloud.firestore.spi.v1.GrpcFirestoreRpc.<init>(GrpcFirestoreRpc.java:122) 
A 2020-04-09T23:44:28.888219Z   at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:122) 
A 2020-04-09T23:44:28.888203Z   at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:155) 
A 2020-04-09T23:44:28.888166Z   at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:186) 
A 2020-04-09T23:44:28.888155Z   at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:194) 
A 2020-04-09T23:44:28.888144Z   at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:296) 
A 2020-04-09T23:44:28.888132Z   at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:507) 
A 2020-04-09T23:44:28.888121Z   at io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.buildTransportFactory(NettyChannelBuilder.java:406) 
A 2020-04-09T23:44:28.888109Z   at io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts.forClient(GrpcSslContexts.java:120) 
A 2020-04-09T23:44:28.888098Z   at io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171) 
A 2020-04-09T23:44:28.888089Z   at io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:244) 
A 2020-04-09T23:44:28.888075Z   at java.lang.Class.ensureInitialized(DynamicHub.java:496) 
A 2020-04-09T23:44:28.888068Z   at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270) 
A 2020-04-09T23:44:28.888061Z   at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350) 
A 2020-04-09T23:44:28.888054Z   at io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:121) 
A 2020-04-09T23:44:28.888047Z   at java.lang.Class.ensureInitialized(DynamicHub.java:496) 
A 2020-04-09T23:44:28.888040Z   at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270) 
A 2020-04-09T23:44:28.888031Z   at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350) 
A 2020-04-09T23:44:28.888023Z   at io.grpc.netty.shaded.io.netty.internal.tcnative.SSL.<clinit>(SSL.java:67) 
A 2020-04-09T23:44:28.888014Z   at io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference(NativeStaticallyReferencedJniMethods.java) 
A 2020-04-09T23:44:28.888Z  at com.oracle.svm.jni.JNIGeneratedMethodSupport.nativeCallAddress(JNIGeneratedMethodSupport.java:57) 
A 2020-04-09T23:44:28.887990Z   at com.oracle.svm.jni.access.JNINativeLinkage.getOrFindEntryPoint(JNINativeLinkage.java:145) 
A 2020-04-09T23:44:28.887982Z Caused by: java.lang.UnsatisfiedLinkError: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__] 
A 2020-04-09T23:44:28.887964Z   at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193) 
A 2020-04-09T23:44:28.887931Z   at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:527) 
A 2020-04-09T23:44:28.887923Z   at java.lang.Thread.run(Thread.java:748) 
A 2020-04-09T23:44:28.887915Z   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
A 2020-04-09T23:44:28.887908Z   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
A 2020-04-09T23:44:28.887900Z   at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:253) 
A 2020-04-09T23:44:28.887892Z   at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:288) 
A 2020-04-09T23:44:28.887884Z   at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.run(FlowableSubscribeOn.java:82) 
A 2020-04-09T23:44:28.887876Z   at io.reactivex.Flowable.subscribe(Flowable.java:14865) 
A 2020-04-09T23:44:28.887869Z   at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887861Z   at io.reactivex.internal.operators.flowable.FlowableFromPublisher.subscribeActual(FlowableFromPublisher.java:29) 
A 2020-04-09T23:44:28.887853Z   at io.micronaut.http.context.ServerRequestTracingPublisher.subscribe(ServerRequestTracingPublisher.java:52) 
A 2020-04-09T23:44:28.887845Z   at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:68) 
A 2020-04-09T23:44:28.887837Z   at io.micronaut.http.context.ServerRequestTracingPublisher.lambda$subscribe$0(ServerRequestTracingPublisher.java:52) 
A 2020-04-09T23:44:28.887828Z   at io.reactivex.Flowable.subscribe(Flowable.java:14868) 
A 2020-04-09T23:44:28.887821Z   at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887814Z   at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty.subscribeActual(FlowableSwitchIfEmpty.java:32) 
A 2020-04-09T23:44:28.887805Z   at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887796Z   at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:37) 
A 2020-04-09T23:44:28.887785Z   at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887777Z   at io.reactivex.internal.operators.flowable.FlowableCreate.subscribeActual(FlowableCreate.java:71) 
A 2020-04-09T23:44:28.887692Z   at io.micronaut.http.server.netty.RoutingInBoundHandler.lambda$buildResultEmitter$16(RoutingInBoundHandler.java:1474) 
A 2020-04-09T23:44:28.887683Z   at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:122) 
A 2020-04-09T23:44:28.887674Z   at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:228) 
A 2020-04-09T23:44:28.887667Z   at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:2993) 
A 2020-04-09T23:44:28.887659Z   at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.getTarget(DefaultBeanContext.java:2972) 
A 2020-04-09T23:44:28.887651Z   at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:618) 
A 2020-04-09T23:44:28.887640Z   at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2058) 
A 2020-04-09T23:44:28.887594Z   at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2084) 
A 2020-04-09T23:44:28.887584Z   at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2393) 
A 2020-04-09T23:44:28.887575Z   at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2407) 
A 2020-04-09T23:44:28.887567Z   at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1693) 
A 2020-04-09T23:44:28.887560Z   at com.company.samplepackage.$HelloControllerDefinition.build(Unknown Source) 
A 2020-04-09T23:44:28.887552Z   at com.company.samplepackage.$HelloControllerDefinition.injectBean(Unknown Source) 
A 2020-04-09T23:44:28.887543Z   at io.micronaut.context.AbstractBeanDefinition.getBeanForField(AbstractBeanDefinition.java:1184) 
A 2020-04-09T23:44:28.887535Z   at io.micronaut.context.AbstractBeanDefinition.getBeanForField(AbstractBeanDefinition.java:1411) 
A 2020-04-09T23:44:28.887526Z   at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1095) 
A 2020-04-09T23:44:28.887501Z   at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2058) 
A 2020-04-09T23:44:28.887192Z   at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2084) 
A 2020-04-09T23:44:28.887180Z   at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2393) 
A 2020-04-09T23:44:28.887167Z   at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2407) 
A 2020-04-09T23:44:28.887113Z   at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1719) 
A 2020-04-09T23:44:28.887104Z Path Taken: HelloController.fb 
A 2020-04-09T23:44:28.887096Z Message: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__] 
A 2020-04-09T23:44:28.887078Z 
A 2020-04-09T23:44:28.887071Z io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type  [com.company.mypackage.HelloController] 
A 2020-04-09T23:44:28.887054Z Path Taken: HelloController.fb 
A 2020-04-09T23:44:28.887044Z Message: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__] 
A 2020-04-09T23:44:28.887027Z 
A 2020-04-09T23:44:28.886998Z 23:44:28.886 [pool-2-thread-2] ERROR i.m.h.s.netty.RoutingInBoundHandler - Unexpected error occurred: Error instantiating bean of type  [com.company.mypackage.HelloController] 
A 2020-04-09T23:44:28.776407Z 23:44:28.775 [pool-2-thread-2] INFO  i.g.n.s.i.n.u.i.PlatformDependent - Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability. 
A 2020-04-09T23:44:28.738912Z Initialized the Firebase App. [DEFAULT] 

Environment Information

renannprado commented 4 years ago

@gbhojak to narrow down a little bit: looking at your stacktrace it seems this error is related to gRPC (which is what the library uses underneath).

gbhojak commented 4 years ago

@gbhojak to narrow down a little bit: looking at your stacktrace it seems this error is related to gRPC (which is what the library uses underneath).

Hi @renannprado , thank you for responding back. It is related to use of gRPC. Looking at 2516, it does look like micronaut app should compile in native mode while using Cloud Firestore sdk. I am not sure if I am missing anything. Would appreciate any help with it. Thank you.

dzou commented 4 years ago

@gbhojak - Hey did you ever figure this out? I am encountering the same error.

gbhojak commented 4 years ago

Hi @dzou , I ended up not using micronaut coz of this issue. I developed the microservice in Go language instead.

emilvberglind commented 3 years ago

I ran into the same issue, and it is related to the included version of grpc-netty-shaded. What I wound up doing, at least for now, was to exclude that transitive artifact and instead use grpc-okhttp. So, my pom.xml looks like this:

    <dependency>
      <groupId>com.google.firebase</groupId>
      <artifactId>firebase-admin</artifactId>
      <version>7.0.0</version>
            <exclusions>
              <exclusion>
                <groupId>io.grpc</groupId>
                <artifactId>grpc-netty-shaded</artifactId>
              </exclusion>
            </exclusions>
    </dependency>

    <!--Using OkHttp over netty because the shaded netty included in grpc does not work with native image-->
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-okhttp</artifactId>
      <version>1.30.2</version>
    </dependency>
TMSantos commented 9 months ago

@emilvberglind does it work for you with exclusion? It compiles, but FCM always returns 400 at cloud Run, running in JVM works fine.

I'm having same error as here https://github.com/GoogleCloudPlatform/native-image-support-java/issues/116

 <!-- FCM -->
    <dependency>
      <groupId>com.google.firebase</groupId>
      <artifactId>firebase-admin</artifactId>
      <version>9.2.0</version>
      <exclusions>
        <exclusion>
          <groupId>io.grpc</groupId>
          <artifactId>grpc-netty-shaded</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

    <!--Using OkHttp over netty because the shaded netty included in grpc does not work with native image-->
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-okhttp</artifactId>
      <version>1.51.1</version>
    </dependency>