search

googleapis / google-cloud-java

Google Cloud Client Library for Java
https://cloud.google.com/java/docs/reference
Apache License 2.0
1.9k stars 1.06k forks source link

Java VerifyError occurs when using com.google.cloud libraries-bom > 26.31.0 #10698

Open rnaval opened 6 months ago

rnaval commented 6 months ago

Environment details

  1. API tested: CloudFilestoreManagerClient.create(), SubscriptionAdminClient.create()
  2. OS type and version: CentOS 7 docker
  3. Java version: Java 17
  4. Version(s): 
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>libraries-bom</artifactId>
        <version>26.37.0</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-monitoring</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-compute</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-storage</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-filestore</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java-util</artifactId>
      <version>4.26.1</version>
    </dependency>
  </dependencies>

Code example

  private CloudFilestoreManagerClient createClient() {
    try {
      return CloudFilestoreManagerClient.create();
    } catch (Exception e) {
      logger.get().error("{} Error when creating client for CloudFilestoreManagerClient : {}",
          this.getClass().getSimpleName(), e.getMessage());
    }
    return null;
  }

Stack trace

java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    com/google/cloud/filestore/v1/Instance.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @24: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @24
    flags: { }
    locals: { 'com/google/cloud/filestore/v1/Instance', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0000 0000 0018 0000 0001 0000 0009
    0000010: 0000 0013 2ab7 0004 b0bb 001d 59bb 001e
    0000020: 59b7 001f 1220 b600 211b b600 22b6 0023
    0000030: b700 24bf                              
  Stackmap Table:
    same_frame(@20)
    same_frame(@25)

    at com.google.cloud.filestore.v1.stub.GrpcCloudFilestoreManagerStub.<clinit>(GrpcCloudFilestoreManagerStub.java:88)
    at com.google.cloud.filestore.v1.stub.CloudFilestoreManagerStubSettings.createStub(CloudFilestoreManagerStubSettings.java:494)
    at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.<init>(CloudFilestoreManagerClient.java:517)
    at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:499)
    at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:490)

Another one:

java.lang.VerifyError: Bad return type
Exception Details:
Location:
com/google/pubsub/v1/Subscription.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @24: areturn
Reason:
Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
Current Frame:
bci: @24
flags: { }
locals: { 'com/google/pubsub/v1/Subscription', integer }
stack: { 'com/google/protobuf/MapField' }
Bytecode:
0000000: 1bab 0000 0000 0018 0000 0001 0000 0009
0000010: 0000 0013 2ab7 0007 b0bb 0069 59bb 006b
0000020: 59b7 006d 126e b600 701b b600 74b6 0077
0000030: b700 7bbf
Stackmap Table:
same_frame(@20)
same_frame(@25)

    at com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.<clinit>(GrpcSubscriberStub.java:78)
    at com.google.cloud.pubsub.v1.stub.SubscriberStubSettings.createStub(SubscriberStubSettings.java:376)
    at com.google.cloud.pubsub.v1.SubscriptionAdminClient.<init>(SubscriptionAdminClient.java:544)
    at com.google.cloud.pubsub.v1.SubscriptionAdminClient.create(SubscriptionAdminClient.java:526)
    at com.google.cloud.pubsub.v1.SubscriptionAdminClient.create(SubscriptionAdminClient.java:517)

Any additional information below

Issue only occurs when we use 26.32.0 onwards. 26.31.0 works fine.

Unit tests seem to be working fine for all versions, but not sure what is happening in the docker image.

diegomarquezp commented 6 months ago

Hi @rnaval, thanks for reporting this. Would you mind sharing the tag of the docker image you are using? Has this occurred when running your java application on your host machine? Thanks!

diegomarquezp commented 6 months ago

It looks like protobuf-java-util may be involved (thanks @suztomo).

diegomarquezp commented 6 months ago

@rnaval our libraries are not using protobuf 4.26 just yet. Would you mind trying to modify your dependency on java-protobuf-util to ~3.25.3~ 3.25.2 (the one our libraries use)?

From https://github.com/googleapis/java-cloud-bom/releases/tag/v26.37.0 image

rnaval commented 6 months ago

Thanks for the response.

Tried it out with the recommended protobuf library, issue still occurs.

java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    com/google/cloud/filestore/v1/Instance.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @24: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @24
    flags: { }
    locals: { 'com/google/cloud/filestore/v1/Instance', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0000 0000 0018 0000 0001 0000 0009
    0000010: 0000 0013 2ab7 0004 b0bb 001d 59bb 001e
    0000020: 59b7 001f 1220 b600 211b b600 22b6 0023
    0000030: b700 24bf                              
  Stackmap Table:
    same_frame(@20)
    same_frame(@25)

    at com.google.cloud.filestore.v1.stub.GrpcCloudFilestoreManagerStub.<clinit>(GrpcCloudFilestoreManagerStub.java:88)
    at com.google.cloud.filestore.v1.stub.CloudFilestoreManagerStubSettings.createStub(CloudFilestoreManagerStubSettings.java:494)
    at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.<init>(CloudFilestoreManagerClient.java:517)
    at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:499)
    at com.google.cloud.filestore.v1.CloudFilestoreManagerClient.create(CloudFilestoreManagerClient.java:490)

Docker image is based on CentOS 7:

FROM centos:7
...

Here's a look at the dependency tree:

INFO] +- com.google.cloud:google-cloud-monitoring:jar:3.42.0:compile
[INFO] |  +- io.grpc:grpc-api:jar:1.62.2:compile
[INFO] |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] |  +- com.google.errorprone:error_prone_annotations:jar:2.26.1:compile
[INFO] |  +- io.grpc:grpc-stub:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-protobuf:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-protobuf-lite:jar:1.62.2:runtime
[INFO] |  +- com.google.api:api-common:jar:2.29.1:compile
[INFO] |  +- com.google.auto.value:auto-value-annotations:jar:1.10.4:compile
[INFO] |  +- javax.annotation:javax.annotation-api:jar:1.3.2:compile
[INFO] |  +- com.google.j2objc:j2objc-annotations:jar:3.0.0:compile
[INFO] |  +- com.google.protobuf:protobuf-java:jar:3.25.2:compile
[INFO] |  +- com.google.api.grpc:proto-google-common-protos:jar:2.37.1:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-monitoring-v3:jar:3.42.0:compile
[INFO] |  +- com.google.guava:guava:jar:33.1.0-jre:provided
[INFO] |  +- com.google.guava:failureaccess:jar:1.0.2:compile
[INFO] |  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] |  +- org.checkerframework:checker-qual:jar:3.42.0:compile
[INFO] |  +- com.google.api:gax:jar:2.46.1:compile
[INFO] |  +- com.google.auth:google-auth-library-credentials:jar:1.23.0:compile
[INFO] |  +- io.opencensus:opencensus-api:jar:0.31.1:compile
[INFO] |  +- io.grpc:grpc-context:jar:1.62.2:compile
[INFO] |  +- io.opentelemetry:opentelemetry-api:jar:1.36.0:compile
[INFO] |  +- io.opentelemetry:opentelemetry-context:jar:1.36.0:compile
[INFO] |  +- com.google.api:gax-grpc:jar:2.46.1:compile
[INFO] |  +- io.grpc:grpc-inprocess:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-core:jar:1.62.2:compile
[INFO] |  +- com.google.android:annotations:jar:4.1.1.4:runtime
[INFO] |  +- org.codehaus.mojo:animal-sniffer-annotations:jar:1.23:runtime
[INFO] |  +- io.grpc:grpc-alts:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-grpclb:jar:1.62.2:compile
[INFO] |  +- org.conscrypt:conscrypt-openjdk-uber:jar:2.5.2:compile
[INFO] |  +- io.grpc:grpc-auth:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-netty-shaded:jar:1.62.2:compile
[INFO] |  +- io.grpc:grpc-util:jar:1.62.2:runtime
[INFO] |  +- io.perfmark:perfmark-api:jar:0.27.0:runtime
[INFO] |  +- io.grpc:grpc-googleapis:jar:1.62.2:runtime
[INFO] |  +- io.grpc:grpc-xds:jar:1.62.2:runtime
[INFO] |  +- io.opencensus:opencensus-proto:jar:0.2.0:runtime
[INFO] |  +- io.grpc:grpc-services:jar:1.62.2:runtime
[INFO] |  +- com.google.re2j:re2j:jar:1.7:runtime
[INFO] |  +- org.threeten:threetenbp:jar:1.6.8:compile
[INFO] |  +- com.google.auth:google-auth-library-oauth2-http:jar:1.23.0:compile
[INFO] |  +- com.google.http-client:google-http-client:jar:1.44.1:compile
[INFO] |  +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile
[INFO] |  +- commons-codec:commons-codec:jar:1.16.1:compile
[INFO] |  +- org.apache.httpcomponents:httpcore:jar:4.4.16:compile
[INFO] |  +- io.opencensus:opencensus-contrib-http-util:jar:0.31.1:compile
[INFO] |  +- com.google.http-client:google-http-client-gson:jar:1.44.1:compile
[INFO] |  \- com.google.code.gson:gson:jar:2.10.1:compile
[INFO] +- com.google.cloud:google-cloud-compute:jar:1.51.0:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-compute-v1:jar:1.51.0:compile
[INFO] |  \- com.google.api:gax-httpjson:jar:2.46.1:compile
[INFO] +- com.google.cloud:google-cloud-storage:jar:2.36.1:compile
[INFO] |  +- com.google.http-client:google-http-client-jackson2:jar:1.44.1:compile
[INFO] |  +- com.google.api-client:google-api-client:jar:2.4.0:compile
[INFO] |  +- com.google.oauth-client:google-oauth-client:jar:1.35.0:compile
[INFO] |  +- com.google.http-client:google-http-client-apache-v2:jar:1.44.1:compile
[INFO] |  +- com.google.apis:google-api-services-storage:jar:v1-rev20240311-2.0.0:compile
[INFO] |  +- com.google.cloud:google-cloud-core:jar:2.36.1:compile
[INFO] |  +- com.google.cloud:google-cloud-core-http:jar:2.36.1:compile
[INFO] |  +- com.google.http-client:google-http-client-appengine:jar:1.44.1:compile
[INFO] |  +- com.google.cloud:google-cloud-core-grpc:jar:2.36.1:compile
[INFO] |  +- com.google.api.grpc:proto-google-iam-v1:jar:1.32.1:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-storage-v2:jar:2.36.1-alpha:compile
[INFO] |  +- com.google.api.grpc:grpc-google-cloud-storage-v2:jar:2.36.1-alpha:compile
[INFO] |  +- com.google.api.grpc:gapic-google-cloud-storage-v2:jar:2.36.1-alpha:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-core:jar:2.16.1:compile
[INFO] |  \- io.grpc:grpc-rls:jar:1.62.2:runtime
[INFO] +- com.google.cloud:google-cloud-filestore:jar:1.42.0:compile
[INFO] |  +- com.google.api.grpc:proto-google-cloud-filestore-v1:jar:1.42.0:compile
[INFO] |  \- com.google.api.grpc:proto-google-cloud-filestore-v1beta1:jar:0.44.0:compile
[INFO] +- com.google.protobuf:protobuf-java-util:jar:3.25.2:compile
suztomo commented 6 months ago

@rnaval Do you observe the problem only happens in the CentOS docker container?

diegomarquezp commented 6 months ago

cc: @suztomo Hi @rnaval, I created a reproducer based on our comments here.

I could run the main function without any issues in IntelliJ. +1 to @suztomo asking if the problem occurs in your host machine as well.

rnaval commented 6 months ago

Sorry for the late response. Based on testing results, we have verified that it works on:

We also tried it on a Centos 7 VM - same error was encountered.

rnaval commented 5 months ago

@diegomarquezp @suztomo Also tried it out with Ubuntu 22.04 docker, seems to work.

diegomarquezp commented 5 months ago

What's the image tag of this CentOS image? Are you using a dockerfile with this image as base or are you running your code directly on it? Maybe a small reproducer such as a bash script showing how you setup your code can help us find out what's missing in this CentOS image.

If no dockerfile is involved, then we should look at the jdk versions in each environment you tested, or more generally the differences in each execution environment.

mdetemad commented 4 months ago

I have the exact same issue after doing 3.25.1 --> 3.25.3.

edporras commented 3 months ago

The circleCI used to generate the docs for our project is failing due to this issue as well although the dependency is from Google's Beam SDK :

    [org.apache.beam/beam-sdks-java-io-google-cloud-platform "2.57.0"

       [com.google.protobuf/protobuf-java-util "3.25.3" :exclusions [[org.checkerframework/jdk8] [com.google.guava/guava-jdk5] [jdk.tools] [com.google.protobuf/protobuf-lite] [org.hamcrest/hamcrest-all]]]

https://app.circleci.com/pipelines/github/cljdoc/builder/53299/workflows/d2e886c4-683d-415a-a8dd-040a88745c5f/jobs/69674

hifly81 commented 1 month ago

same for me, from 3.23.4 to 3.25.4