search

square / wire

gRPC and protocol buffers for Android, Kotlin, Swift and Java.
https://square.github.io/wire/
Apache License 2.0
4.24k stars 571 forks source link

Compiling a proto that uses a vendored descriptor.proto #2901

Open anuraaga opened 5 months ago

anuraaga commented 5 months ago

I am considering writing a protoc-gen plugin using wire for handling CodeGeneratorRequest. However, I'm having trouble getting the proto to compile due to interactions with handling of built-in types.

The protos I am trying to compile are the same as in the protobuf.dart project.

https://github.com/google/protobuf.dart/tree/master/protoc_plugin/protos

The compiler complains that google/protobuf/descriptor.proto isn't imported even though these are defined in the vendored descriptor.proto.

> descriptor.proto needs to import google/protobuf/descriptor.proto
    for field file (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/descriptor.proto:58:3)
    in message google.protobuf.FileDescriptorSet (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/descriptor.proto:57:1)
  descriptor.proto needs to import google/protobuf/descriptor.proto
    for field message_type (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/descriptor.proto:75:3)
    in message google.protobuf.FileDescriptorProto (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/descriptor.proto:62:1)
  plugin.proto needs to import google/protobuf/descriptor.proto
    for field proto_file (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/plugin.proto:91:3)
    in message google.protobuf.compiler.CodeGeneratorRequest (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/plugin.proto:68:1)
  plugin.proto needs to import google/protobuf/descriptor.proto
    for field generated_code_info (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/plugin.proto:180:5)
    in message google.protobuf.compiler.CodeGeneratorResponse.File (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/plugin.proto:121:3)
    in message google.protobuf.compiler.CodeGeneratorResponse (/Users/anuraag/git/connect-kotlin/protoc-gen-connect-kotlin/src/main/proto/plugin.proto:99:1)

If I remove descriptor.proto and change the import from vendored to standard google/protobuf/descriptor.proto, it succeeds and generates a file referencing descriptor.proto

package com.google.protobuf.compiler

import com.google.protobuf.FileDescriptorProto
import com.squareup.wire.FieldEncoding
import com.squareup.wire.Message
import com.squareup.wire.ProtoAdapter

But FileDescriptorProto doesn't exist, and I couldn't generate it because of the above error. I looked through wire-runtime and wire-schema and couldn't find the class published officially.

I can workaround by changing the package of the protos, but protoc can compile these files. I'm wondering if there is a metric that wire compiler is supposed to be able to compile any file that protoc can.

I'm wondering if either, the descriptor built-in-types are being eagerly used even when not relevant in this case? Or should the wire versions of these protos be published in an artifact?

oldergod commented 5 months ago

Catching up. Do you mean that if Wire generates google/protobuf/descriptor.proto, it won't generate FileDescriptorProto that you need?

anuraaga commented 5 months ago

Hi @oldergod - there are two cases I tried. One is with the exact protos linked. Then it doesn't compile at all with the above error message.

If I change this import to google/protobuf/descriptor.proto

https://github.com/google/protobuf.dart/blob/master/protoc_plugin/protos/plugin.proto#L55

And remove descriptor.proto from that folder, the compilation succeeds, but naturally it didn't generate FileDescriptorProto since it wasn't part of the build.

oldergod commented 5 months ago

Wire doesn't generate FileDescriptorProto. But there is a way to get it by using the https://github.com/square/wire/blob/master/wire-schema/src/commonMain/kotlin/com/squareup/wire/schema/internal/SchemaEncoder.kt

Look at the tests to see how it can be done https://github.com/square/wire/blob/master/wire-schema/src/jvmTest/kotlin/com/squareup/wire/schema/internal/SchemaEncoderTest.kt

Note that some things are not supported as described in SchemaEncoder's doc.