Support &[CustomType]

[Walter-Reactor]

I'm working on a project where I need to expose a multi-language API, and I love diplomat's approach. However, the one big hole in Diplomat that makes it unusable for us is exposing `&[UserDefinedStruct]. Is this within scope and do you have any suggestions on how it might be integrated?

[Manishearth]

How do you envision this working over FFI?

We do support output iterables, and could support input iterables, and that's probably the cleanest way to do it. But any type that has its own ownership that needs to be converted across the boundary is a lot of extra cost. So it really depends on what you're trying to do.

It would be helpful if we could see the shape of the APIs you're trying to expose. Vec<CustomType> can mean a lot of things, despite being a singular concept in Rust when we start talking about FFI it starts meaning more things.

It is possible to implement your own wrapping type that is internally a Vec<CustomType> and expose the APIs you need.

[Walter-Reactor]

Edited the topic and my post to better express intent. Sorry for any confusion, I'm still coming up on Rust.

I'm trying to construct a FFI wrapper around a custom serialization format, a little like FlexBuffers. We have one layer that operates on a pointer to data & a runtime loaded schema, and another using generated code to move a lot of the offsets to compile time. One thing we allow is that that structs may contain one of 2 different custom containers (Map/Vec equivalents our own stable ABI & in-memory layout), but there's no way to express this in diplomat.

Considering it, I think what we really need is the ability to support custom container types, not just slices of user defined types.

[Manishearth]

I'll write more later but to highlight: &[CustomType] in the context of Diplomat still means two things: exposing a rust slice to other languages or vice versa. The former is somewhat supported with indexing/etc methods on a custom wrapper type. It appears that you're looking for the other way: using a custom, say, Java container from Rust. (please lmk if that's not the case)

Are you specifically looking for common stdlib types to work with this, or the ability for a custom collection to implement something that allows it to be passed to Rust? The former is not that hard to design, the latter needs callback support which we might get at some point but don't currently have.

[Walter-Reactor]

The more I think about it the more I think we'll probably need custom generic collection support, which is gonna be a heck of a thing to generate automatically no matter what. I may wind up just using cbindgen and writing custom language specific stuff, since we mostly only care about rust, c++, and python as potential target languages

[Manishearth]

To be clear we're already able to get some custom collection support with how we handle outputting strings: we're able to support different language string types and our older c++ backend even supported custom types that fit a template trait.

So it may not be that hard but I'd really need to know what the precise functionality is.