Closed psmskelton closed 1 year ago
Codec::load_library()
has two overloads, both do the same thing, but the void* overload assumes you've already called dlopen
somewhere, so the std::string overload is probably the easiest to use (see https://libdccl.org/4.0/classdccl_1_1Codec.html).
After opening the .so
Codec will call the dccl3_load C function (analogous to a constructor) which must be present in that library:
extern "C"
{
void dccl3_load(dccl::Codec* dccl)
{
}
void dccl3_unload(dccl::Codec* dccl)
{
}
}
(the dccl_unload
function (analogous to a destructor) must be present as well to unload the library). For both of these the Codec*
pointer passed is to the Codec that called load_library
.
Within these you can call any Codec methods that you want, so for your use you can call load
on your Protobuf messages, such as:
void dccl3_load(dccl::Codec* dccl)
{
dccl->load<protobuf::MyDCCLMessage1>();
dccl->load<protobuf::MyDCCLMessage2>();
}
void dccl3_unload(dccl::Codec* dccl)
{
dccl->unload<protobuf::MyDCCLMessage1>();
dccl->unload<protobuf::MyDCCLMessage2>();
}
One useful thing would be that I could update the protoc-gen-dccl
plugin to optionally output these load/unload snippets automatically at compile time.
Given that, you can load the .so with the following (simpler than you had above):
dccl::Codec codec_;
codec_.load_library("/usr/local/lib/libsomeprotolibrary.so");
Ahhh, of course, I see where I was mistaken on the process. Thanks for the clarification!
Having either the descriptors or the load/unload snippets automatically generated could be helpful for teams that have multiple developers using the same core message library. What I've implemented in the meantime is a parsing of the .pb.h
files that accompany the .so
, looking for:
class <message_class_name> : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:<message_full_name>) */ {
Parsing for the <message_full_name>
field, and then loading that using:
_codec.load(dccl::DynamicProtobufManager::find_descriptor(<message_full_name>));
As we are controlling what versions of protobuf
(etc.) libraries are used, the glass-house fragility of this method is acceptable at the moment. Although fast as these files are <100k lines, we don't really care even if it takes a second or two as we load the .so
on instantiation, not when we need it 🤷
Take a look at https://github.com/GobySoft/dccl/pull/106 where I added functionality to protoc-gen-dccl
to make it easier to do what you want without relying on fragile preprocessing steps.
I know I owe everyone some documentation on the more advanced features.
What is the recommended method for deploying a
dccl::Codec
in a.so
format?For example, let's assume we have an extensive shared library (
.so
) containingN
messages defined acrossM
.proto
files. This.so
, along with the requisite*_pb.h
files, enables deployment of messages no problems. However, having to managecodec.load()
onN*M
messages across multiple projects is growing tiring, so we'd like to also provide a pre-loadeddccl::Codec
with this library.We have a deployed (hack) solution for Python that uses some Python shenanigans to overcome the inability to iterate over the
google::protobuf::DescriptorPool
(NB: Google's design decision to not facilitate that as they use an internal-to-Google.proto
file that has every message defined across all of their internal projects, so you can imagine how large that is) to programmatically load all messages, and provides the user with a convenient<library>.get_codec()
method that already has the entire set of messages loaded. While we could implement something similar in C++, I believe that would go against the "intended" way of doing things.I believe you have already considered this as the
dccl::Codec
definition states:Which, I assume, for runtime configurable/dynamic
.so
ingestion can be passed adl_handle
as:However, a documented process for achieving it appears to be missing in the docs.