google / flatbuffers

FlatBuffers: Memory Efficient Serialization Library
https://flatbuffers.dev/
Apache License 2.0
23.31k stars 3.25k forks source link

Feature request: Rust reflection API #8033

Open dextero opened 1 year ago

dextero commented 1 year ago

Dear Flatbuffers team,

My team is currently considering using flatbuffers in our Rust project. We saw that reflection is only implemented in C++, and we are wondering if there is any way to add Rust reflection support to flatbuffers.

We believe that adding Rust reflection support to flatbuffers would be a great addition to the library, and it would make it much easier for us to use flatbuffers in our project.

If this is not an effort the Flatbuffers team is already planning or working on, we would love to contribute to the project. Could you share your thoughts on how we could approach this?

The most straightforward solution seems to be to wrap the C++ reflection API, but the documentation describes it as “experimental”. Is the C++ API stable enough that such an approach would make sense?

CasperN commented 1 year ago

Hi, I was the last active maintainer on the Rust part of Flatbuffers, though I'm not active now.

There weren't any reflection efforts and a contribution would be greatly appreciated. There were actually multiple notions of reflection, mini-reflection and (full) reflection. The former asked users to provide a TypeTable, while the latter a reflection.fbs table (which is basically the .fbs file).

I don't think directly wrapping C++ is the best approach, it's probably a better idea to build what you need in a Rust native way. You should depend on the flatbuffers crate for the binary manipulation stuff (this is what the generated code does)

assaframan commented 1 year ago

Thanks @CasperN, do you think we can create the reflection as a new Rust crate (let's call it flatbuffers-reflection) that has the Rust flatbuffers crate as a dependency, or we will need lower level access? (meaning it has to be new code in the existing flatbuffers crate)

CasperN commented 1 year ago

A downstream crate seems preferable and I think you'd be fine with the public API (since that's what the generated code uses too)

steple commented 10 months ago

@dextero and @assaframan , I was wondering if you have given this a try and what you learned.

candysonya commented 10 months ago

@steple , I work with Marcin and Assaf and have just marked the pr as 'Ready for Review'. Please ignore the commits uploaded in Sep because they have been overwritten by the commits in Dec after Marcin's review.

I probably will submit another commit/pr for buffer verification (against schema) but won't work on mini reflection. Let me know what you think!

steple commented 10 months ago

Awesome, I hope to give it a try. I'll let you know what I find.

github-actions[bot] commented 4 months ago

This issue is stale because it has been open 6 months with no activity. Please comment or label not-stale, or this will be closed in 14 days.

akesling commented 4 months ago

Any updates on this? It looks like https://github.com/google/flatbuffers/pull/8102 was last updated in February.