Questions on: Memory Overhead and Benchmarks

[ganeshrvel]

Hi,

I have been exploring various projects while looking for a way to use Rust in combination with Flutter. First of all, congratulations! This is one of the most sophisticated Flutter projects I have seen.

I have a few questions though:

a. Do you have any existing benchmarks for the bridge?

b. If I have hundreds of thousands of struct data (mostly metadata info) coming as a list from Rust to Dart, will there be a memory copy overhead?

1. Are there any memory copy overheads while passing data from Dart to Rust?
2. If yes, does the bridge introduce overhead in copying the vector to and from Rust?
3. Are there any benchmarks conducted on this aspect?
4. How much memory overhead can be expected in such scenarios?

c. I believe some kind of communication channel, such as message ports or signals, is used in the Flutter bridge for passing data, or is it 100% FFI with zero memory copy?

1. If it uses message ports or signals to communicate, is there any noticeable delay or overhead observed?
   1. When sending a communication from Flutter to Rust, a message-port signal request is sent to Rust from Flutter. Does this cause a very brief UI freeze?
   2. If yes, to avoid UI freezing for a few milliseconds, do we need to start the call to Rust using an isolate or is it all done under the hood?
   3. If yes, will starting an isolate add noticeable delay in communication due to the overhead of cold starting an isolate in Dart?

Thanks

[welcome[bot]]

Hi! Thanks for opening your first issue here! :smile:

[fzyzcjy]

Thank you!

a. Do you have any existing benchmarks for the bridge?

Yes, doc -> https://cjycode.com/flutter_rust_bridge/guides/performance

b. If I have hundreds of thousands of struct data (mostly metadata info) coming as a list from Rust to Dart, will there be a memory copy overhead?

Btw, I would suggest doing some benchmarking and see whether the overhead is big or small for your particular scenario, since it vary according to concrete cases.

Moreover, I wonder whether we should really copy between Dart and Rust? If we use the opaque types, then it will live forever in Rust (or Dart), so no copy happens at all. For example, suppose we have a huge dict of 100MB memory and 100000 entry. Then, instead of transfering it between Dart and Rust, we can indeed let it live in Rust memory. Suppose we now want to search a key in the dict, then our Dart code calls a Rust function with the key, and Rust function returns the value. Then we only copy e.g. several dozen bytes (the key and value).

Are there any memory copy overheads while passing data from Dart to Rust?

Yes, because Dart does not provide an API for us to do zero copy IIRC (but feel free to ping me if I am wrong)

If yes, does the bridge introduce overhead in copying the vector to and from Rust?

Copying things like Vec<u8> from rust to dart is zero-copy thanks to Dart API.

Are there any benchmarks conducted on this aspect?

Check the doc above (IIRC there are some)

How much memory overhead can be expected in such scenarios?

Depending on your concrete case imho. Thus it would be great to explain the scenario a bit more.

I believe some kind of communication channel, such as message ports or signals, is used in the Flutter bridge for passing data, or is it 100% FFI with zero memory copy?

frb has multiple codecs. Thus, under different scenarios it uses Dart "port" feature, direct ffi, serialization, etc.

If it uses message ports or signals to communicate, is there any noticeable delay or overhead observed?

Nobody has complained noticeable delays yet (you can search the issue section) and I personally also do not see any. But again feel free to ping me if you find any!

When sending a communication from Flutter to Rust, a message-port signal request is sent to Rust from Flutter. Does this cause a very brief UI freeze?

Only if something is as slow as e.g. 15ms or 50ms or 500ms, will you see a freeze... IMHO frb's function call is much much faster than that.

[fzyzcjy]

Again, I would suggest to firstly describe your concrete scenarios, then we can discuss a bit more, e.g. like above - maybe we just use opaque types and never do copies.

[ganeshrvel]

Thanks a lot for the detailed explanation. Looking forward to use this.

[fzyzcjy]

You are welcome!

[sanbei101]

What a positive and exceptional developer! @fzyzcjy

[fzyzcjy]

Thank you!

[ganeshrvel]

@fzyzcjy I think you might be interested in the newer Dart version release.

Blog: https://medium.com/dartlang/dart-3-5-6ca36259fa2f

Dart native interoperability In Dart 3.5, we’ve made incremental improvements to support passing a pointer from Dart TypedData objects directly to FFI, avoiding having to first copy the memory from Dart to Native (details).

[fzyzcjy]

@ganeshrvel That looks quite interesting!

Quickly glanced (I may be wrong), that will be helpful if we want to convert it to borrowed types, e.g. Uint8List -> &[u8]. But if we want to convert Uint8List to a Vec<u8>, then it may not be trivial, since Rust's Vec needs to own the memory, and Vec requires the memory layout etc to be very strict.

Anyway, if you face any specific performance issue (especially caused by this copying), feel free to ping me!