Ways for easier communication between Dart and Rust

[LucaCoduriV]

Hello, I'd like to share an idea to enhance this project. Protobuf provides services that enable the creation of functions primarily used with gRPC. I think it would be cool to be able to use it to call functions between Rust and Dart. I quickly looked into it, and it seems that Prost for Rust supports this, but I haven't checked the Dart side yet. If you think it's a good idea, I could delve into it further and possibly assist in implementing it.

[temeddix]

Thanks for the idea!

Yes, Protobuf does have 'service', so I considered using those inside Rinf. However the full support for it wasn't quit obvious on both sides of Dart and Rust. Maybe we can keep this as an idea, for now :)

[LucaCoduriV]

Good for me. In the meantime, I checked for dart and it seems that to make it possible there is no choice but to fork the project. It will need a lot of work.

[temeddix]

Hi @LucaCoduriV , maybe we can discuss this project further. I do agree that if we use Protobuf services, instead of something like RustRequest, it can be easier to write code.

• https://github.com/hyperium/tonic/blob/master/examples/helloworld-tutorial.md

I'm planning to make a new branch for experiment and development. If this change turns out to be easier and cleaner than the current system, we might be able to merge it in the next version :)

[thlorenz]

I am currently using rinf for a project (https://luzid.app) which has a Flutter UI via Rinf but also needs to allow controlling the Solana Validator via an SDK, i.e. scripts. While looking into supporting this and reusing the Rinf types I also learned hat tonic provides you with a service interface and with tonic-reflect you can even get intellisense in Postman (and similar tools) while figuring out GRPC requests. (https://blog.postman.com/postman-now-supports-grpc/)

This is all to say that I'd also love to see something like this integrated into Rinf as then the API implemtantation for the Rinf message handling and that GRPC server would be almost the same.

Also the CRUD idea may make sense in some scenarios, but for an app where I basically want to do RPC style function calls the REST pattern gets almost in the way (I just picked whichever Read/Create/Update and repurposed it for what I actually needed).

Also to facilitate reuse of the generated proto types (in my case for the GRPC server/client) it'd be useful to override where rinf writes those Rust definitions, so they can be exposed via a core crate (instead of the server having to depend on the hub crate to get to them). Basically 'native/hub/src/messages' on this line needs to be configurable via a command line arg.

I'm gonna provide a PR for that tomorrow so we can get there step by step assuming this is something you want to add.

Finally I wanted to say great lib! I attempted something similar a while back rid, but went too far trying to provide all boilerplate and a simple API. However that got super complex quickly and I find the solution to use protobuf instead a very elegant one!

[temeddix]

Great! If we do this experiment, these think would be important:

• Easy: Users shouldn't be required to write too much more code than the CRUD system right now.
• Maintainable: Without a logic that's too complex for ourselves to understand.

I've been taking a look at tonic too, and maybe we should be able to answer these questions:

1. tonic server seems to use HTTP/2, which has too much overhead when compared to native FFI. Should we implement the FFI server(that doesn't use HTTP) ourselves?
2. With tonic, Protobuf services becomes Rust traits. Should the user write their own structs to impl this traits, or can we provide a simple codegen mechanism for that?
3. Should each of Protobuf service get a different ABI, in other words, the extern "C" fn?
4. How will 'streaming from Rust to Dart' work, without any first request from Dart?

I'll be happy to accept any kind of ideas or PR, and possibly merge it into the next version if it looks okay. Thank you very much for your participation! I'm looking forward to seeing your solution.

P.S. Thank you for choosing Rinf! If you enjoy it, please consider letting others know if they're looking something like this :)

[thlorenz]

Just wanted to post the solution I currently came up with. https://gist.github.com/thlorenz/6b16d0a9f2202df6c149cc2095c695ab

It includes the build.rs I'm using + a sample proto file and the resulting generated rust.

It's a three step process:

1. Run rinf and have it add the messages to my grpc crate that has tonic as a dependency
2. After capturing rinf generated ID, run tonic to generate the same files again which then includes the service definitions
3. Append the captured rinf ID to the end of the tonic generated file

To be clear the service is used only when running this as a separate GRPC server. The hub and rinf requests still work as before, only that they now use the rust types that are inside that grpc crate.

[temeddix]

Maybe we can declare messages just like before, but only let go of the CRUD concept, and implement a very simple code generation mechanism.

So that this Rust function

#[rinf::handler]
async fn test(message: MyMessage) -> MyReturn { }

becomes this Dart function

Future<MyReturn> test(MyMessage message) async { }

where MyMessage and MyReturn is declared in a .proto file.

Your example looks interesting and respects to the amount of thoughs went into it. However if we want to make it 'efficient' enough, we should not use HTTP for communication for this. We can organize ideas and brainstorm more here :)