search

dtolnay / typetag

Serde serializable and deserializable trait objects
Apache License 2.0
1.19k stars 38 forks source link

Using `typetag::serde` on a type that is loaded after program startup #78

Open bruceg opened 4 months ago

bruceg commented 4 months ago

I am experimenting with dynamic loading of components into Vector using dynamic-plugin (which uses libloading under the hood FWIW). Those components are configured using typetag::serde on a configuration trait. I have the dynamic loading of the code working, as well as registering items with inventory (see below). The next part is trying to deserialize a struct in the loaded module through typetag::serde where the trait exists outside of the loaded module.

Based on the solution to registration with inventory that I worked up, I need to be able to pass (return) the concrete value that is being submitted there from the plugin to the loading code. That value is the return value from fn <dyn Trait>::typetag_register. However, that value names TypetagRegistration which is hidden in code generated by the proc macro within a const _: () = { … } block and so cannot be named outside of that block.

~Since that type is effectively the same for all instances of the generated code, would it make any sense to move TypetagRegistration into typetag::__private like Strictest and DeserializeFn are? I would be happy to submit a PR if you are open to that.~ That won't work because of needing to do an impl on that type which falls afoul of Rust's orphan rules.

Alternately, I am open to any other solution to the inventory registration issue. I know that the constructor it generates is actually run when the module is loaded (at least on Linux), but the default code generation/linkage results in there being separate registries in both the module and the loading code. Hypothetically working around that would also resolve the problem, but the best shot I had at it requires different rustflags in different crates in the workspace, to which I have not found a solution.

dtolnay commented 2 months ago

the default code generation/linkage results in there being separate registries in both the module and the loading code

This is the actual thing to fix. I don't think you will be able to solve the issue by messing with TypetagRegistration.

It might be sufficient to have your trait defined in its own shared library, which each of the components dynamically links against. There would be a single registry determined by the shared library containing the trait.

I don't have a lot of experience with Cargo's implementation of shared linkage, other than a general understanding that it is not well-exercised by the ecosystem and I am not certain whether it is expressive enough for this.

I do have experience with using shared linkage for Rust with Buck2. My work codebase has many uses of inventory with shared linkage that we build using Buck2 and "link groups", including one use of typetag.