ruby-protobuf / protobuf

A pure ruby implementation of Google's Protocol Buffers
https://github.com/ruby-protobuf
MIT License
463 stars 101 forks source link

Unable to add defs to DescriptorPool: Symtab already has a def named #381

Open bmarkons opened 6 years ago

bmarkons commented 6 years ago

When the Google::Protobuf::DescriptorPool.generated_pool.build is called twice with the same message types, the error is being thrown saying that DescriptorPool has already registered message with the same name. I see that as expected behavior.

Though, I need to include GRPC generated files within Rails codebase which some parts are eagerly loaded.

Is there any possible way to tell protobuf to generate Ruby classes and modules instead of generating Google::Protobuf::DescriptorPool.generated_pool.build block? In case it has generated classes, the require on that file more than once wouldn't throw an error.

I see here in example that you have generated class: https://github.com/ruby-protobuf/protobuf/wiki/Compiling-Definitions But not sure what I am missing...

I see the only way to fix this is to add some condition before Google::Protobuf::DescriptorPool.generated_pool.build block, letting it register messages unless these are already defined.

BTW I wanted to run gprc server within a rake task when I stumbled upon this issue.

Please tell me if I am missing something, which is quite likely. Thank you

oliverx0 commented 6 years ago

I faced the same issue. More specifically, when loading the rails console in production it would break trying to load the files twice. The reason it loaded the files twice was because: a) First it loaded them with eager load, and then 2) I added the folder with the generated protos to my $LOAD_PATH, so that requiring with absolute paths did not break.

Just fyi, rails rake task do not use eager load for performance purposes (example: https://github.com/becky000/sample_app/blob/c568be5ea301638a2b47f82a391ae9b7bd2dd820/config/environments/production.rb#L10).

In any case, I fixed this by not storing generated protobuf files inside the rails app folder. Instead, store them in lib. Then, in your application.rb do:

    grpc_root = File.join(config.root, "/lib/ruby_proto")
    $LOAD_PATH.unshift(grpc_root)
    Dir[File.join(grpc_root, "**/*.rb")].each do |file|
      require file
    end
bmarkons commented 6 years ago

I fixed this by moving generated files out of lib into custom protobuf directory. And then in initializer just required them once (which requires them once in both rake task and rails server) :)

On Thu, Jun 21, 2018, 1:44 AM Oliver Hoffman notifications@github.com wrote:

I faced the same issue. More specifically, when loading the rails console in production it would break trying to load the files twice. The reason it loaded the files twice was because: a) First it loaded them with eager load, and then 2) I added the folder with the generated protos to my $LOAD_PATH, so that requiring with absolute paths did not break.

Just fyi, rails rake task do not use eager load for performance purposes (example: https://github.com/becky000/sample_app/blob/c568be5ea301638a2b47f82a391ae9b7bd2dd820/config/environments/production.rb#L10 ).

In any case, I fixed this by not storing generated protobuf files inside the rails app folder. Instead, store them in lib. Then, in your application.rb do:

` Load gRPC proto files

grpc_root = File.join(config.root, "/lib/ruby_proto") $LOAD_PATH.unshift(grpc_root) Dir[File.join(grpc_root, "*/.rb")].each do |file| require file end

`

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ruby-protobuf/protobuf/issues/381#issuecomment-398931511, or mute the thread https://github.com/notifications/unsubscribe-auth/AI9iEMXihPTQorwxKbGUP6-2gnqf4qZwks5t-t5XgaJpZM4RNcrF .