search

AMythicDev / minus

An asynchronous, runtime data feedable terminal paging library for Rust
https://crates.io/crates/minus/
Apache License 2.0
321 stars 23 forks source link

async runtime segfaults when working with large strings #26

Closed rezural closed 3 years ago

rezural commented 3 years ago

Please refer to the segfault example in this branch:

https://github.com/rezural/minus/tree/segfault

run with:

cargo run --example segfault --features=async_std_lib

if you uncomment the line with prints the bytes_written, you can see that it segfaults semi randomly. sometimes at 50000 bytes_written, sometimes at 4 million.

I am actually hitting these segfaults when paging the output of ls */ (or find) in a folder with a large number of files.

rezural commented 3 years ago

using rust-lldb: backtrace is giving me:

* thread #9, name = 'async-std/runti', stop reason = signal SIGSEGV: invalid address (fault address: 0x7ffff6960a7e)
  * frame #0: 0x000055555570234a segfault`core::slice::memchr::memchr_general_case::hbae8e5f2524397e5 [inlined] core::slice::memchr::memchr_general_case::_$u7b$$u7b$closure$u7d$$u7d$::hd27538be4abdc77f at memchr.rs:67:67
    frame #1: 0x000055555570234a segfault`core::slice::memchr::memchr_general_case::hbae8e5f2524397e5 [inlined] _$LT$core..slice..iter..Iter$LT$T$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$::position::h2f62ab79226e8f32 at macros.rs:287
    frame #2: 0x0000555555702345 segfault`core::slice::memchr::memchr_general_case::hbae8e5f2524397e5 at memchr.rs:67
    frame #3: 0x00005555555c0e4f segfault`core::slice::memchr::memchr::hc2d045c3939dc8e8(x='\n', text=(data_ptr = "", length = 228605)) at memchr.rs:49:5
    frame #4: 0x00005555555c3774 segfault`_$LT$core..str..pattern..CharSearcher$u20$as$u20$core..str..pattern..Searcher$GT$::next_match::h573f636f2bb561f1(self=0x00007ffff6de31b0) at pattern.rs:410:34
    frame #5: 0x00005555555aecc6 segfault`core::str::iter::SplitInternal$LT$P$GT$::next::h95a4d0c321d0fe09(self=0x00007ffff6de31a0) at iter.rs:583:15
    frame #6: 0x000055555558a5af segfault`_$LT$core..str..iter..SplitTerminator$LT$P$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$::next::h1852f7192929f169(self=0x00007ffff6de31a0) at iter.rs:433:17
    frame #7: 0x0000555555599d29 segfault`_$LT$core..iter..adapters..Map$LT$I$C$F$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$::next::h03cfae97c1688b82(self=0x00007ffff6de31a0) at mod.rs:924:9
    frame #8: 0x00005555555836be segfault`_$LT$core..str..iter..Lines$u20$as$u20$core..iter..traits..iterator..Iterator$GT$::next::h3d443ecd10ab2625(self=0x00007ffff6de31a0) at iter.rs:935:9
    frame #9: 0x000055555558a8c8 segfault`core::iter::traits::iterator::Iterator::fold::ha219a01419955418(self=Lines @ 0x00007ffff6de31a0, init=0, f=0x0000000a0000000a) at iterator.rs:2022:29
    frame #10: 0x000055555558aa53 segfault`core::iter::traits::iterator::Iterator::count::h373d93445212d416(self=<unavailable>) at iterator.rs:253:9
    frame #11: 0x000055555559f7ee segfault`minus::init::dynamic_paging::_$u7b$$u7b$closure$u7d$$u7d$::h8fe93eb08da0c155((null)=ResumeTy @ 0x00007ffff6de3e58) at init.rs:189:26
    frame #12: 0x0000555555585ed2 segfault`_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h9ed8f66f7ac823a0(self=Pin<&mut core::future::from_generator::GenFuture<generator-0>> @ 0x00007ffff6de40c0, cx=0x00007ffff6de4a18) at mod.rs:80:19
    frame #13: 0x000055555558e1db segfault`minus::rt_wrappers::run::_$u7b$$u7b$closure$u7d$$u7d$::ha3e899031a472d8f((null)=ResumeTy @ 0x00007ffff6de4250) at rt_wrappers.rs:121:5
    frame #14: 0x0000555555586192 segfault`_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::haea41cc0077a689e(self=Pin<&mut core::future::from_generator::GenFuture<generator-0>> @ 0x00007ffff6de43a0, cx=0x00007ffff6de4a18) at mod.rs:80:19
    frame #15: 0x000055555558687e segfault`_$LT$async_std..task..builder..SupportTaskLocals$LT$F$GT$$u20$as$u20$core..future..future..Future$GT$::poll::_$u7b$$u7b$closure$u7d$$u7d$::hc793ae70e44e30bc at builder.rs:199:17
    frame #16: 0x0000555555598497 segfault`async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::_$u7b$$u7b$closure$u7d$$u7d$::hd4f89850f35fffff(current=0x00007ffff6de7540) at task_locals_wrapper.rs:60:13
    frame #17: 0x000055555559626f segfault`std::thread::local::LocalKey$LT$T$GT$::try_with::h4dc546f9aa0a5e1e(self=0x0000555555784d28, f=closure-0 @ 0x00007ffff6de45c8) at local.rs:272:16
    frame #18: 0x0000555555595b72 segfault`std::thread::local::LocalKey$LT$T$GT$::with::hb6b681b26cefd77e(self=0x0000555555784d28, f=<unavailable>) at local.rs:248:9
    frame #19: 0x00005555555980e2 segfault`async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::hc0967c23745e7b28(task=0x000055555579b330, f=closure-0 @ 0x00007ffff6de4618) at task_locals_wrapper.rs:55:9
    frame #20: 0x000055555558674b segfault`_$LT$async_std..task..builder..SupportTaskLocals$LT$F$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h40342db45b667851(self=Pin<&mut async_std::task::builder::SupportTaskLocals<core::future::from_generator::GenFuture<generator-0>>> @ 0x00007ffff6de4648, cx=0x00007ffff6de4a18) at builder.rs:197:13
    frame #21: 0x000055555559b517 segfault`async_executor::Executor::spawn::_$u7b$$u7b$closure$u7d$$u7d$::h2194021579e482a2((null)=ResumeTy @ 0x00007ffff6de47a0) at lib.rs:138:13
    frame #22: 0x0000555555585db2 segfault`_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h393b0a2445f13f5c(self=Pin<&mut core::future::from_generator::GenFuture<generator-0>> @ 0x00007ffff6de48f0, cx=0x00007ffff6de4a18) at mod.rs:80:19
    frame #23: 0x0000555555592925 segfault`async_task::raw::RawTask$LT$F$C$T$C$S$GT$::run::h5dc104bc3fdfd401(ptr=0x000055555579b260) at raw.rs:489:20
    frame #24: 0x0000555555668ac6 segfault`async_task::runnable::Runnable::run::ha4dc255cab4465d1(self=Runnable @ 0x00007ffff6de4d18) at runnable.rs:309:18
    frame #25: 0x000055555562b2a2 segfault`async_executor::Executor::run::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hfd50b19953bd8b51((null)=ResumeTy @ 0x00007ffff6de4e50) at lib.rs:229:21
    frame #26: 0x0000555555634b8b segfault`_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hd94686d36979447e(self=Pin<&mut core::future::from_generator::GenFuture<generator-0>> @ 0x00007ffff6de4ee8, cx=0x00007ffff6de5528) at mod.rs:80:19
    frame #27: 0x000055555562645d segfault`_$LT$futures_lite..future..Or$LT$F1$C$F2$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h2ed321c0c95461c4(self=Pin<&mut futures_lite::future::Or<futures_lite::future::Pending<()>, core::future::from_generator::GenFuture<generator-0>>> @ 0x00007ffff6de4f38, cx=0x00007ffff6de5528) at future.rs:529:33
    frame #28: 0x000055555562aa44 segfault`async_executor::Executor::run::_$u7b$$u7b$closure$u7d$$u7d$::h53771c95885cf905((null)=ResumeTy @ 0x00007ffff6de5178) at lib.rs:236:9
    frame #29: 0x000055555563466b segfault`_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h5b49efa0ba058cc0(self=Pin<&mut core::future::from_generator::GenFuture<generator-0>> @ 0x00007ffff6de51f8, cx=0x00007ffff6de5528) at mod.rs:80:19
    frame #30: 0x000055555562650d segfault`_$LT$futures_lite..future..Or$LT$F1$C$F2$GT$$u20$as$u20$core..future..future..Future$GT$::poll::ha9634d9b4b8312a0(self=Pin<&mut futures_lite::future::Or<core::future::from_generator::GenFuture<generator-0>, core::future::from_generator::GenFuture<generator-0>>> @ 0x00007ffff6de5248, cx=0x00007ffff6de5528) at future.rs:529:33
    frame #31: 0x000055555563ee28 segfault`async_io::driver::block_on::h694537b77cc0cc14(future=Or<core::future::from_generator::GenFuture<generator-0>, core::future::from_generator::GenFuture<generator-0>> @ 0x00007ffff6de5b50) at driver.rs:142:33
    frame #32: 0x000055555561ad74 segfault`async_global_executor::reactor::block_on::_$u7b$$u7b$closure$u7d$$u7d$::h1f9fc28b9c4dee84 at reactor.rs:3:18
    frame #33: 0x000055555561ac50 segfault`async_global_executor::reactor::block_on::h1655d981809b4c16(future=<unavailable>) at reactor.rs:12:5
    frame #34: 0x0000555555627c90 segfault`async_global_executor::threading::thread_main_loop::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hdfc6ef2d7ae0512b(executor=0x00007ffff6de7558) at threading.rs:95:17
    frame #35: 0x000055555561c38d segfault`std::thread::local::LocalKey$LT$T$GT$::try_with::hc6b0509c3904ed05(self=0x00005555557875d8, f=closure-0 @ 0x00007ffff6de6748) at local.rs:272:16
    frame #36: 0x000055555561b294 segfault`std::thread::local::LocalKey$LT$T$GT$::with::hbb477ad5aec7f42c(self=0x00005555557875d8, f=closure-0 @ 0x00007ffff6de67a0) at local.rs:248:9
    frame #37: 0x0000555555627d03 segfault`async_global_executor::threading::thread_main_loop::_$u7b$$u7b$closure$u7d$$u7d$::h5cd525f38109b108 at threading.rs:89:13
    frame #38: 0x000055555562267b segfault`std::panicking::try::do_call::h4d0ac430fa053487(data="") at panicking.rs:381:40
    frame #39: 0x000055555562513d segfault`__rust_try + 29
    frame #40: 0x00005555556224c7 segfault`std::panicking::try::h0128ffa6c2aa4e83(f=closure-1 @ 0x00007ffff6de6880) at panicking.rs:345:19
    frame #41: 0x000055555562bd7f segfault`std::panic::catch_unwind::hdbe51de6ee788da7(f=closure-1 @ 0x00007ffff6de68d0) at panic.rs:382:14
    frame #42: 0x000055555562772d segfault`async_global_executor::threading::thread_main_loop::h83d34eab7d3416d3 at threading.rs:88:12
    frame #43: 0x000055555562c5fa segfault`core::ops::function::FnOnce::call_once::hfe112fdc63760817((null)=0x0000000000000000, (null)=<unavailable>) at function.rs:227:5
    frame #44: 0x000055555562240a segfault`std::sys_common::backtrace::__rust_begin_short_backtrace::h3129133d7d9b22aa(f=0x0000000000000000) at backtrace.rs:137:18
    frame #45: 0x0000555555633557 segfault`std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h927dfe1a95f0a634 at mod.rs:464:17
    frame #46: 0x000055555562b99a segfault`_$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h68cea2437b56cc47(self=AssertUnwindSafe<closure-0> @ 0x00007ffff6de6ab0, _args=<unavailable>) at panic.rs:308:9
    frame #47: 0x000055555562269f segfault`std::panicking::try::do_call::h6fdda2f3a308202e(data="\x90\xa8yUUU") at panicking.rs:381:40
    frame #48: 0x000055555562513d segfault`__rust_try + 29
    frame #49: 0x0000555555622554 segfault`std::panicking::try::h3171bdd067fd1530(f=AssertUnwindSafe<closure-0> @ 0x00007ffff6de6b68) at panicking.rs:345:19
    frame #50: 0x000055555562bd1a segfault`std::panic::catch_unwind::h9b059f022337c754(f=AssertUnwindSafe<closure-0> @ 0x00007ffff6de6bb0) at panic.rs:382:14
    frame #51: 0x000055555563339c segfault`std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::h3ff39be4746c0371 at mod.rs:463:30
    frame #52: 0x000055555562c3f9 segfault`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h5358611557d3b660((null)=0x000055555579a900, (null)=<unavailable>) at function.rs:227:5
    frame #53: 0x00005555556eba0a segfault`std::sys::unix::thread::Thread::new::thread_start::h33c6d7cf25f9291f [inlined] _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::hd4d30e37db281e8f at boxed.rs:1042:9
    frame #54: 0x00005555556eba04 segfault`std::sys::unix::thread::Thread::new::thread_start::h33c6d7cf25f9291f [inlined] _$LT$alloc..boxed..Box$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::h019185a215ba16e4 at boxed.rs:1042
    frame #55: 0x00005555556eb9fb segfault`std::sys::unix::thread::Thread::new::thread_start::h33c6d7cf25f9291f at thread.rs:87
    frame #56: 0x00007ffff7c02609 libpthread.so.0`start_thread(arg=<unavailable>) at pthread_create.c:477:8
    frame #57: 0x00007ffff7e8d293 libc.so.6`__clone + 67

which corresponds in minus to the following line:

let line_count = guard.lines.lines().count();
rezural commented 3 years ago

Just off the top of my head,

perhaps moving to a Vec of strings, may at least make things more performant when working with larger strings.

I can have a look into changing to a Vec, if this sounds like a good idea to you.

Cheers

rezural commented 3 years ago

OK, I have produced a minimal repro. And the cuplrit appears to be the PagerMutex & PagerGuard code.

I believe you could use Mutex from futures or async_std.

I will send a patch with this if I get it working.