welcome[bot]
commented
1 week ago Hi! Thanks for opening your first issue here! :smile:
Closed Litt1eQ closed 1 week ago
welcome[bot]
commented
1 week ago Hi! Thanks for opening your first issue here! :smile:
fzyzcjy
commented
1 week ago Hmm interesting. Looks like vec_resize caused that.
Firstly, could you please install LLVM (https://cjycode.com/flutter_rust_bridge/manual/ffigen-troubleshooting#llvm-needs-to-be-installed) and enable full_dep: true in config yaml? Because for your case DCO codec may be much faster than the SSE codec without full_dep enabled.
Secondly, curious why do you need to transfer 2GB data? Maybe we can discuss and see whether it is really needed (or maybe e.g. things can live on Rust side without moving to Dart side)
Litt1eQ
commented
1 week ago Firstly, thank you for your response. I enabled the configuration and reran flutter_rust_bridge_codegen generate, but the error still persists.
thread '<unnamed>' panicked at library/alloc/src/raw_vec.rs:25:5:
capacity overflow
stack backtrace:
0: 0x100ca529c - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h243268f17d714c7f
1: 0x100ce81d4 - core::fmt::write::hb3cfb8a30e72d7ff
2: 0x100c9b280 - std::io::Write::write_fmt::hfb2314975de9ecf1
3: 0x100ca77ac - std::panicking::default_hook::{{closure}}::h14c7718ccf39d316
4: 0x100ca73d0 - std::panicking::default_hook::hc62e60da3be2f352
5: 0x100a9bcd4 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::ha3771cb704b88a16
6: 0x100aa3404 - flutter_rust_bridge::misc::panic_backtrace::PanicBacktrace::setup::{{closure}}::hf7eef2f7421ac658
7: 0x100ca83c8 - std::panicking::rust_panic_with_hook::h09e8a656f11e82b2
8: 0x100ca7cb0 - std::panicking::begin_panic_handler::{{closure}}::h1230eb3cc91b241c
9: 0x100ca5728 - std::sys::backtrace::__rust_end_short_backtrace::hc3491307aceda2c2
10: 0x100ca79a0 - _rust_begin_unwind
11: 0x100d0b014 - core::panicking::panic_fmt::ha4b80a05b9fff47a
12: 0x100cd9bbc - alloc::raw_vec::capacity_overflow::h88d6b8d70532516f
13: 0x100d0abac - alloc::raw_vec::handle_error::h3874354ba7b66dcc
14: 0x100a9ff2c - <u8 as alloc::vec::spec_from_elem::SpecFromElem>::from_elem::h2af9457ae79cd135
15: 0x100aa4f18 - alloc::vec::from_elem::hf2fba866b1b568fd
16: 0x100aa35f8 - flutter_rust_bridge::for_generated::pointer::new_leak_vec_ptr::h18db222fc1c5c161
17: 0x100a8cf30 - _frbgen_flutter_via_create_cst_new_list_prim_u_8_loose
thread '<unnamed>' panicked at library/core/src/panicking.rs:221:5:
panic in a function that cannot unwind
stack backtrace:
0: 0x100ca529c - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h243268f17d714c7f
1: 0x100ce81d4 - core::fmt::write::hb3cfb8a30e72d7ff
2: 0x100c9b280 - std::io::Write::write_fmt::hfb2314975de9ecf1
3: 0x100ca77ac - std::panicking::default_hook::{{closure}}::h14c7718ccf39d316
4: 0x100ca73d0 - std::panicking::default_hook::hc62e60da3be2f352
5: 0x100a9bcd4 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::ha3771cb704b88a16
6: 0x100aa3404 - flutter_rust_bridge::misc::panic_backtrace::PanicBacktrace::setup::{{closure}}::hf7eef2f7421ac658
7: 0x100ca83c8 - std::panicking::rust_panic_with_hook::h09e8a656f11e82b2
8: 0x100ca7cb0 - std::panicking::begin_panic_handler::{{closure}}::h1230eb3cc91b241c
9: 0x100ca5728 - std::sys::backtrace::__rust_end_short_backtrace::hc3491307aceda2c2
10: 0x100ca79a0 - _rust_begin_unwind
11: 0x100d0b044 - core::panicking::panic_nounwind_fmt::h91ee161184879b56
12: 0x100d0b0bc - core::panicking::panic_nounwind::heab7ebe7a6cd845c
13: 0x100d0b234 - core::panicking::panic_cannot_unwind::hedc43d82620205bf
14: 0x100a8cf24 - _frbgen_flutter_via_create_cst_new_list_prim_u_8_loose
thread caused non-unwinding panic. aborting.Secondly, In my current situation where I need to transfer 2GB of data, I need to implement an encryption and decryption function. The content being processed is quite large. For the PC platform, I can use files for transmission, but for other platforms like the Web, it's not feasible to use files for transmission. Therefore, I'm passing a &[u8] array as the input parameter, and the return value is also the same type of array.
fzyzcjy
commented
1 week ago I see. But I am worried about:
I need to implement an encryption and decryption function
For this specific scenario, I personally suggest to implement some chunking mechanism. For example, chunk 2GB into many 10MB blocks, and encrypt/decrypt each of them separately.
Litt1eQ
commented
1 week ago Thank you very much for your suggestion. I should consider modifying my design to process the content in chunks.
fzyzcjy
commented
1 week ago You are welcome!
fzyzcjy
commented
1 week ago Close since this seems to be solved. Feel free to reopen if needed!
Describe the bug
In flutter capacity overflow when send large Uint8List to rust(> 2gb)
Steps to reproduce
Hint: A simple way to reproduce is to clone and modify the https://github.com/fzyzcjy/flutter_rust_bridge/tree/master/frb_example/flutter_via_create example package according to your needs.
import 'package:flutter/material.dart'; import 'package:flutter_via_create/src/rust/api/simple.dart'; import 'package:flutter_via_create/src/rust/frb_generated.dart';
Future main() async {
await RustLib.init();
runApp(const MyApp());
}
class MyApp extends StatelessWidget { const MyApp({super.key});
@override Widget build(BuildContext context) { return MaterialApp( home: Scaffold( appBar: AppBar(title: const Text('flutter_rust_bridge quickstart')), body: Center( child: TextButton( onPressed: () { final result = greet(name: "world"); print(result); testVec(data: Uint8List(1024 1024 1024 * 2)).then((value) => print(value)); }, child: const Text("data"), ), ), ), ); } }
Expected behavior
No response
Generated binding code
No response
OS
MacOS 15.1 Apple chip
Version of
flutter_rust_bridge_codegen2.6.0
Flutter info
Version of
clang++Apple clang version 16.0.0 (clang-1600.0.26.3)
Additional context
No response