search

grpc / grpc-java

The Java gRPC implementation. HTTP/2 based RPC
https://grpc.io/docs/languages/java/
Apache License 2.0
11.46k stars 3.85k forks source link

Generated OuterClass has a lot of incompatibilities ! #2300

Closed smallufo closed 8 years ago

smallufo commented 8 years ago

Hi , I am new to gRPC / protobuf , and facing a first step problem...

This is my first .proto file :

syntax = "proto3";
option java_package = "destiny.identity.services.grpc";

service EchoService {
    rpc echo (EchoRequest) returns (EchoReply) {}
}

message EchoRequest {
    string name=1;
}

message EchoReply {
    string message = 1;
}

It is simple , but the generated EchoServiceOuterClass.java has a lot of incompatibilities :

~/destiny-identity/target/generated-sources/protobuf/java/destiny/identity/services/grpc/EchoServiceOuterClass.java
Error:(83, 13) java: incompatible types: java.io.IOException cannot be converted to java.lang.String
Error:(152, 18) java: memoizedSize has private access in com.google.protobuf.AbstractMessage
Error:(159, 7) java: memoizedSize has private access in com.google.protobuf.AbstractMessage
Error:(394, 18) java: cannot find symbol
  symbol:   method unwrapIOException()
  location: variable e of type com.google.protobuf.InvalidProtocolBufferException
Error:(586, 13) java: incompatible types: java.io.IOException cannot be converted to java.lang.String
Error:(655, 18) java: memoizedSize has private access in com.google.protobuf.AbstractMessage
Error:(662, 7) java: memoizedSize has private access in com.google.protobuf.AbstractMessage
Error:(897, 18) java: cannot find symbol
  symbol:   method unwrapIOException()
  location: variable e of type com.google.protobuf.InvalidProtocolBufferException

For example :

Image

This is my [ partial ] pom.xml

<dependencies>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-netty</artifactId>
      <version>${grpc.version}</version>
    </dependency>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-protobuf</artifactId>
      <version>${grpc.version}</version>
    </dependency>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-stub</artifactId>
      <version>${grpc.version}</version>
    </dependency>
   ... other skipped ...
</dependencies>

// the related plugin :
<plugin>
  <groupId>org.xolstice.maven.plugins</groupId>
  <artifactId>protobuf-maven-plugin</artifactId>
  <version>0.5.0</version>
  <configuration>
    <!--
      The version of protoc must match protobuf-java. If you don't depend on
      protobuf-java directly, you will be transitively depending on the
      protobuf-java version that grpc depends on.
    -->
    <protocArtifact>com.google.protobuf:protoc:3.0.0:exe:${os.detected.classifier}</protocArtifact>
    <pluginId>grpc-java</pluginId>
    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.0.0:exe:${os.detected.classifier}</pluginArtifact>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>compile</goal>
        <goal>compile-custom</goal>
      </goals>
    </execution>
  </executions>
</plugin>

The grpc-netty , grpc-protobuf , grpc-stub are all of version of 1.0.0

Where goes wrong here ?

Thanks.

I am new to gRPC / protobuf , sorry I don't know whether the problem is related to grpc-java project or not.

ejona86 commented 8 years ago

I think you're getting an older version of protobuf pulled in from somewhere. The missing method noted in the screenshot is present in v3.0.0. You can run mvn dependency:tree -Dverbose to try to debug where the old copy may be coming from.

smallufo commented 8 years ago

Hi , I check the dependencies , but found no error : 

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.classifier: osx-x86_64
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building destiny-identity 0.0.1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ destiny-identity ---
[WARNING] Using Maven 2 dependency tree to get verbose output, which may be inconsistent with actual Maven 3 resolution
[INFO] artifact io.netty:netty-codec-http2: checking for updates from nexus
[INFO] artifact io.netty:netty-codec-http2: checking for updates from spring-snapshots
[INFO] artifact io.netty:netty-codec-http2: checking for updates from spring-milestones
[INFO] artifact io.netty:netty-codec-http2: checking for updates from central
[INFO] artifact io.grpc:grpc-core: checking for updates from nexus
[INFO] artifact io.grpc:grpc-core: checking for updates from spring-snapshots
[INFO] artifact io.grpc:grpc-core: checking for updates from spring-milestones
[INFO] artifact io.grpc:grpc-core: checking for updates from central
[INFO] destiny:destiny-identity:jar:0.0.1
[INFO] +- io.grpc:grpc-netty:jar:1.0.0:compile
[INFO] |  +- io.netty:netty-codec-http2:jar:4.1.3.Final:compile
[INFO] |  |  +- io.netty:netty-codec-http:jar:4.1.3.Final:compile
[INFO] |  |  |  \- io.netty:netty-codec:jar:4.1.3.Final:compile
[INFO] |  |  |     \- (io.netty:netty-transport:jar:4.1.3.Final:compile - omitted for duplicate)
[INFO] |  |  \- io.netty:netty-handler:jar:4.1.3.Final:compile
[INFO] |  |     +- io.netty:netty-buffer:jar:4.1.3.Final:compile
[INFO] |  |     |  \- io.netty:netty-common:jar:4.1.3.Final:compile
[INFO] |  |     +- io.netty:netty-transport:jar:4.1.3.Final:compile
[INFO] |  |     |  +- (io.netty:netty-buffer:jar:4.1.3.Final:compile - omitted for duplicate)
[INFO] |  |     |  \- io.netty:netty-resolver:jar:4.1.3.Final:compile
[INFO] |  |     |     \- (io.netty:netty-common:jar:4.1.3.Final:compile - omitted for duplicate)
[INFO] |  |     \- (io.netty:netty-codec:jar:4.1.3.Final:compile - omitted for duplicate)
[INFO] |  \- io.grpc:grpc-core:jar:1.0.0:compile
[INFO] |     +- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO] |     \- com.google.code.findbugs:jsr305:jar:3.0.0:compile
[INFO] +- io.grpc:grpc-protobuf:jar:1.0.0:compile
[INFO] |  +- com.google.guava:guava:jar:19.0:compile
[INFO] |  +- io.grpc:grpc-protobuf-lite:jar:1.0.0:compile
[INFO] |  |  +- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO] |  |  \- (io.grpc:grpc-core:jar:1.0.0:compile - omitted for duplicate)
[INFO] |  +- com.google.protobuf:protobuf-java:jar:3.0.0:compile
[INFO] |  +- com.google.protobuf:protobuf-java-util:jar:3.0.0:compile
[INFO] |  |  +- (com.google.protobuf:protobuf-java:jar:3.0.0:compile - omitted for duplicate)
[INFO] |  |  +- (com.google.guava:guava:jar:18.0:compile - omitted for conflict with 19.0)
[INFO] |  |  \- com.google.code.gson:gson:jar:2.7:compile (version managed from 2.3)
[INFO] |  \- (io.grpc:grpc-core:jar:1.0.0:compile - omitted for duplicate)
[INFO] +- io.grpc:grpc-stub:jar:1.0.0:compile
[INFO] |  \- (io.grpc:grpc-core:jar:1.0.0:compile - omitted for duplicate)
// skipped

It is com.google.protobuf:protobuf-java:jar:3.0.0:compile , and there are no other protobuf strings in the following skipped dependencies.

ejona86 commented 8 years ago

I pasted your proto into the grpc-java/examples/src/main/proto/echo_service.proto and it built without issue.

This log message looks interesting though:

[WARNING] Using Maven 2 dependency tree to get verbose output, which may be inconsistent with actual Maven 3 resolution

The Dependency Resolution of the compatibility notes provides a little bit of guidance, but not much.

I would still investigate dependency issues though, because InvalidProtocolBufferException did not have the IOException overload in 2.6.1.

smallufo commented 8 years ago

Is there any fake protobuf 3.0 on the internet ?

I remove the local repos: smallufo@MacBookPro:~/.m2/repository/com/google $ rm -rf protobuf/

and re run mvn compile

$ mvn compile
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.classifier: osx-x86_64
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building destiny-identity 0.0.1
[INFO] ------------------------------------------------------------------------

Downloading: https://repo.spring.io/snapshot/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.pom
Downloading: https://repo.spring.io/milestone/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.pom
Downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.pom
Downloaded: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.pom (4 KB at 3.2 KB/sec)
Downloading: https://repo.spring.io/snapshot/com/google/protobuf/protobuf-parent/3.0.0/protobuf-parent-3.0.0.pom
Downloading: https://repo.spring.io/milestone/com/google/protobuf/protobuf-parent/3.0.0/protobuf-parent-3.0.0.pom
Downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-parent/3.0.0/protobuf-parent-3.0.0.pom
Downloaded: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-parent/3.0.0/protobuf-parent-3.0.0.pom (7 KB at 9.9 KB/sec)
Downloading: https://repo.spring.io/snapshot/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.pom
Downloading: https://repo.spring.io/milestone/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.pom
Downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.pom
Downloaded: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.pom (4 KB at 5.6 KB/sec)
Downloading: https://repo.spring.io/snapshot/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.jar
Downloading: https://repo.spring.io/snapshot/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.jar
Downloading: https://repo.spring.io/milestone/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.jar
Downloading: https://repo.spring.io/milestone/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.jar
Downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.jar
Downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.jar
Downloaded: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java-util/3.0.0/protobuf-java-util-3.0.0.jar (64 KB at 59.2 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.jar (1274 KB at 897.7 KB/sec)
[INFO] 
[INFO] --- protobuf-maven-plugin:0.5.0:compile (default) @ destiny-identity ---

Downloading: https://repo.spring.io/snapshot/com/google/protobuf/protoc/3.0.0/protoc-3.0.0.pom
Downloading: https://repo.spring.io/milestone/com/google/protobuf/protoc/3.0.0/protoc-3.0.0.pom
Downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.0.0/protoc-3.0.0.pom
Downloaded: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.0.0/protoc-3.0.0.pom (5 KB at 10.1 KB/sec)
Downloading: https://repo.spring.io/snapshot/com/google/protobuf/protoc/3.0.0/protoc-3.0.0-osx-x86_64.exe
Downloading: https://repo.spring.io/milestone/com/google/protobuf/protoc/3.0.0/protoc-3.0.0-osx-x86_64.exe
Downloading: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.0.0/protoc-3.0.0-osx-x86_64.exe
Downloaded: https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.0.0/protoc-3.0.0-osx-x86_64.exe (4698 KB at 3847.3 KB/sec)
[INFO] Compiling 1 proto file(s) to /Users/smallufo/Dropbox/destiny/destiny-identity/target/generated-sources/protobuf/java
[INFO] 
[INFO] --- protobuf-maven-plugin:0.5.0:compile-custom (default) @ destiny-identity ---
[INFO] Compiling 1 proto file(s) to /Users/smallufo/Dropbox/destiny/destiny-identity/target/generated-sources/protobuf/grpc-java
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ destiny-identity ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 5 resources
[INFO] Copying 3 resources
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ destiny-identity ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 83 source files to /Users/smallufo/Dropbox/destiny/destiny-identity/target/classes
[INFO] /Users/smallufo/Dropbox/destiny/destiny-identity/src/main/java/destiny/tools/CookieTools.java: Some input files use unchecked or unsafe operations.
[INFO] /Users/smallufo/Dropbox/destiny/destiny-identity/src/main/java/destiny/tools/CookieTools.java: Recompile with -Xlint:unchecked for details.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.604 s
[INFO] Finished at: 2016-09-27T02:18:28+08:00
[INFO] Final Memory: 37M/342M
[INFO] ------------------------------------------------------------------------

But it still generates incompatible EchoServiceOuterClass.java

smallufo commented 8 years ago

This is the generated code , FYI

http://pastebin.com/bNaQK9cy

ejona86 commented 8 years ago

I'm not aware of any fake protobuf 3, but it looks like something strange may be going on.

But it still generates incompatible EchoServiceOuterClass.java

What do you mean? It looks like it worked.

smallufo commented 8 years ago

The generated EchoServiceOuterClass.java still cannot be compiled because of incompatibilities:

screenshot

smallufo commented 8 years ago

I decide to create another clean project , and compile again. It seems OK now . It is very weird. See below :
screen

The front is window is the new(clean) test project . The generated EchoServiceOuterClass.java is Ok , and the dependent jar InvalidProtocolBufferException has IOException constructor.

But the old version included jar only has String constructor.

The two jars are the same !!! Without source , but IntelliJ decompile one with bytecode version 50.0 , but another id version 49.0 !!?

I try to Reveal in Finder , both protobuf-java-3.0.0.jar are locating in ~/.m2/repository/com/google/protobuf/protobuf-java/3.0.0/protobuf-java-3.0.0.jar . It's ONE file, not two different jar.

I am really puzzled...

ejona86 commented 8 years ago

Wat? Yeah, that's strange. The one on my machine is bytecode version 50. I've also confirmed there isn't a duplicate class in the JAR (which is possible with zips). I'd also still believe that the IDE was accidentally showing you a different protobuf version.

But in either case, it seems you're working now.

smallufo commented 8 years ago

OK . I found where goes wrong. TIL libphonenumber also dependent on protobuf ! And the project is dependent on an older libphonenumber , not via maven , it is via groovy's @Grab !. That's why mvn's dependency:tree cannot inspect the problem ! And maybe intelliJ regard groovy's @Grab has higher priority.

Anyway , it is fixed. sorry for disturb.

onnorh commented 6 years ago

I am experiencing similar issues. Generated code has lots of errors in Outer cass, main class looks ok. I've attached sample project. GRPC connector is here... https://github.com/mulesoft-labs/grpc-connector

grpcconnectortest.zip