JuliaIO / ProtoBuf.jl

Julia protobuf implementation
Other
205 stars 55 forks source link

Module decleration is only for one _pb file #227

Open schnetzlerjoe opened 1 year ago

schnetzlerjoe commented 1 year ago

I have directories with nested directories and such and am compiling dozens of .proto files at a time. When I do this I noticed that in the module definition for each directory (the non prefixed _pb.jl files), it is only including one of the _pb files although the module has 4+.

You can test this by looping through all the directories and compiling the .proto files here https://github.com/cosmos/cosmos-sdk/tree/main/proto and check the declaration files. cosmos.jl for example.

Drvi commented 1 year ago

Hey @schnetzlerjoe can you share the code you use to generate the julia definitions? E.g. what I did was to collect all proto files at once like this:

# in cosmos-sdk repo root
protos = replace.(readlines(`find ./proto -name "*.proto"`), "./proto/" => "")
protojl(protos, "proto", ".")

NOTE: I needed https://github.com/JuliaIO/ProtoBuf.jl/pull/228 to parse some RPC definitions with trailing semicolons. With this the protojl is complaining it cannot find following files tim import: ["gogoproto/gogo.proto", "cosmos_proto/cosmos.proto", "google/api/annotations.proto"]

schnetzlerjoe commented 1 year ago

Hey @schnetzlerjoe can you share the code you use to generate the julia definitions? E.g. what I did was to collect all proto files at once like this:

# in cosmos-sdk repo root
protos = replace.(readlines(`find ./proto -name "*.proto"`), "./proto/" => "")
protojl(protos, "proto", ".")

NOTE: I needed #228 to parse some RPC definitions with trailing semicolons. With this the protojl is complaining it cannot find following files tim import: ["gogoproto/gogo.proto", "cosmos_proto/cosmos.proto", "google/api/annotations.proto"]

Alright looks like your looping method actually ends up pointing to the correct modules!

However, I am now obtaining a ERROR: LoadError: syntax: invalid module name "module" error.

Those missing proto files you are referring to are

./proto/gogoproto/gogo.proto: https://github.com/gogo/protobuf/blob/master/gogoproto/gogo.proto and

./proto/google/api/annotations.proto: https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto

./proto/google/api/http.proto: https://github.com/googleapis/googleapis/blob/master/google/api/http.proto

./proto/google/api/httpbody.proto: https://github.com/googleapis/googleapis/blob/master/google/api/httpbody.proto

./proto/google/protobuf/any.proto: https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/any.proto

./proto/google/protobuf/timestamp.proto: https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/timestamp.proto

Once you pull them into there respective locations. Try compiling.

It will work now but when I call

include("../output/defundlabs/defundlabs.jl")

function MsgSubmitInterqueryResult()
    io = IOBuffer();

    e = ProtoEncoder(io);

    encoded = encode(e, defundlabs.InterqueryResult("", "", "", [], cosmos.height(0,0), 10, true, true))

    println(encoded)
end

I get a ERROR: LoadError: syntax: invalid module name "module" error. Maybe a module naming issue? Not sure

schnetzlerjoe commented 1 year ago

@Drvi Can confirm above it is a naming class with the module keyword within Julia. Any suggestions to fix that?

Drvi commented 1 year ago

@schnetzlerjoe Yes, when I put all the dependencies into place and compile the code I can see that we don't validate module names and we hit a problem when we try to import code that touches the generated version of this file https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/params/module/v1/module.proto#L3

I'll look into it, thanks for reporting the issue!