FasterXML / jackson-dataformats-binary

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile
Apache License 2.0
316 stars 136 forks source link

[protobuf] . can't resolve inner types #73

Open knoguchi opened 7 years ago

knoguchi commented 7 years ago

When messages are defined as below, the parser failed to resolve t1.i1 type.

package mypackage;

message t1 {
        message i1 {
                optional uint32 x = 1;
                optional uint32 y = 2;
        }
}

message t2 {
        optional t1.i1 z = 1;
}

Error

Exception in thread "main" java.lang.IllegalArgumentException: Unknown protobuf field type 't1.i1' for field 'z' of MessageType 't2' (known enum types: ; known message types: t1, t2)
    at com.fasterxml.jackson.dataformat.protobuf.schema.TypeResolver._resolve(TypeResolver.java:141)
    at com.fasterxml.jackson.dataformat.protobuf.schema.TypeResolver.resolve(TypeResolver.java:93)
    at com.fasterxml.jackson.dataformat.protobuf.schema.NativeProtobufSchema.forType(NativeProtobufSchema.java:67)
    at com.fasterxml.jackson.dataformat.protobuf.schema.ProtobufSchemaLoader.load(ProtobufSchemaLoader.java:51)
cowtowncoder commented 7 years ago

Unfortunately I can not reproduce this: if I load schema you specify, there is no error. This could either be due to version you are using (if it's old, could be fixed), or your code does something different that what I have. My test has:

 ProtobufSchema schema = MAPPER.schemaLoader()
                .load(new StringReader(SCHEMA_STR));

with schema s above.

knoguchi commented 7 years ago

Thanks for the quick response. I simplified my test code. The only difference is that I specified the rootTypeName "t2". I'm using the HEAD of the 2.8 branch pulled today.

ProtobufSchema schema = MAPPER.schemaLoader()
                .load(new StringReader(SCHEMA_STR), "t2");

And then the stacktrace

Exception in thread "main" java.lang.IllegalArgumentException: Unknown protobuf field type 't1.i1' for field 'z' of MessageType 't2' (known enum types: ; known message types: t1, t2)
    at com.fasterxml.jackson.dataformat.protobuf.schema.TypeResolver._resolve(TypeResolver.java:141)
    at com.fasterxml.jackson.dataformat.protobuf.schema.TypeResolver.resolve(TypeResolver.java:94)
    at com.fasterxml.jackson.dataformat.protobuf.schema.NativeProtobufSchema.forType(NativeProtobufSchema.java:67)
    at com.fasterxml.jackson.dataformat.protobuf.schema.ProtobufSchemaLoader.load(ProtobufSchemaLoader.java:97)
cowtowncoder commented 7 years ago

Thanks! Yes, that does trigger the issue for me too, for 2.8 branch and probably master too.

cowtowncoder commented 7 years ago

(accidental fix by github, due to comment)

knoguchi commented 7 years ago

I am trying to understand the code base, and I saw the type resolver tries to find the name recursively but it's a little hard for me to fix. I wonder if it's easy for you to fix.

My idea is to calculate fully qualified name beginning with the package name at the time of parsing so that the type resolver does not have to traverse the type tree. Do you think it's a sane approach?

knoguchi commented 7 years ago

I just noticed FieldElement or DataType.NamedType class does not have methods that returns qualified name or type unlike TypeElement class that has .qualifiedName(). So we still need type resolution for the fields.

Looks like the Wire, the successor of the ProtoParser, was created for the very reason.

I hope someday ProtoParser will be replaced for Wire in this project. For now, I decided to use other solutions. Thanks.

cowtowncoder commented 5 years ago

@knoguchi I was hoping to fix this along with #140, as I now understand the issue finally (it has been a while but I got back to here). Scoped resolution works with latest fixes but only in certain orders of traversal; I'll have to rewrite code a bit. Should get done for 2.10.0 however, need to get 2.9.8 out first.