Error on install: 'nul byte found in provided data'

[qouoq]

I get an error on a fresh install. Running

rm -r ~/.mcfly # just in case
export RUST_BACKTRACE=full
eval "$(mcfly init zsh)"

Results in

McFly: Importing shell history for the first time. This may take a minute or two...thread 'main' panicked at 'McFly error: Insert to work (nul byte found in provided data at position: 0)', src/history/history.rs:818:51

Backtrace

stack backtrace:
   0:           0x52a43d - std::backtrace_rs::backtrace::libunwind::trace::h596593836c6b6d3a
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:           0x52a43d - std::backtrace_rs::backtrace::trace_unsynchronized::hbee209c07f0843b2
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:           0x52a43d - std::sys_common::backtrace::_print_fmt::hc7c6f11cbc15d951
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/sys_common/backtrace.rs:67:5
   3:           0x52a43d - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hc092a197028c14a9
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/sys_common/backtrace.rs:46:22
   4:           0x46586c - core::fmt::write::h3ff7a2cbe501613a
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/fmt/mod.rs:1149:17
   5:           0x529c14 - std::io::Write::write_fmt::h8031bfb369f96bb6
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/io/mod.rs:1697:15
   6:           0x5298fb - std::sys_common::backtrace::_print::hc016cee3bccf63da
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/sys_common/backtrace.rs:49:5
   7:           0x5298fb - std::sys_common::backtrace::print::h6866e45b8ebc5ccc
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/sys_common/backtrace.rs:36:9
   8:           0x5298fb - std::panicking::default_hook::{{closure}}::h9c5117e922e480bb
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/panicking.rs:211:50
   9:           0x5298fb - std::panicking::default_hook::ha84fb34f61e4f117
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/panicking.rs:228:9
  10:           0x5298fb - std::panicking::rust_panic_with_hook::hd5546f2a0ef20897
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/panicking.rs:606:17
  11:           0x54cfc8 - std::panicking::begin_panic_handler::{{closure}}::hfa1808dc3899df9a
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/panicking.rs:502:13
  12:           0x54cf46 - std::sys_common::backtrace::__rust_end_short_backtrace::he35776a92ffb9dc9
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/sys_common/backtrace.rs:139:18
  13:           0x54cf02 - rust_begin_unwind
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/panicking.rs:498:5
  14:           0x4014f0 - core::panicking::panic_fmt::ha1b95e66beb52b4f
                               at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/panicking.rs:107:14
  15:           0x4958fd - mcfly::history::history::History::from_shell_history::{{closure}}::hb268aa9bbcb49eeb
  16:           0x494a80 - mcfly::history::history::History::load::hb1bbc1fc14a647f0
  17:           0x40ad8e - mcfly::main::h957b2bcf60610ce0
  18:           0x4bcfd3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h84ed9a8b1da7e631
  19:           0x40e0fa - main

This happens with the Arch Linux package and the github release, both version 0.5.13, with zsh 5.8.1 (it works if I init the db from .bash_history).

I would venture a guess that my .zsh_history contains something wrong, there's a lot of garbage in it from copy-paste mishaps. It's about 20k lines though, so manually curating it is not something I'd be looking forward to.

--edit Mmh, I'm thinking that doing binary splits of the zsh history could be an approach, although a bit time-consuming (14-15 iterations if I'm not mistaken). Would that be the best way to proceed?

[cantino]

Hey @qouoq, thanks for your report! Is it easy for you to try https://github.com/cantino/mcfly/pull/237 and see if it helpfully prints an error instead of crashing?

[qouoq]

I want to take my chapeau off here! Going from an issue to a proposed fix in this amount of time, I'm astounded :no_mouth:

So, err, no, the proposed fix doesn't print a helpful error, in the sense that I don't know which history line is the offender (but it does look like there is only one). Now, it does work since the history import completes and McFly functions as expected upon installation. And I guess it doesn't matter to point out the problem line. I'd say the message could be friendlier (such as [newline]History line skipped during import due to parsing error for example?). The init message is now as follows:

McFly: Importing shell history for the first time. This may take a minute or two...McFly error: Insert did not work (nul byte found in provided data at position: 0)
done.

But in any case, fix is good and I'm proposing to close the issue, unless you want to keep it open until pushed to master?

By the way, since this is my first contact with McFly, I see that timestamps of imported history are all at import time, would it be something reasonable to grab the timestamps from the history file, or maybe is there some existing feature request/ongoing work along these lines?

[cantino]

No problem :)

I just added slightly more verbose logging if you want to try deleting ~/.mcfly and running #237 again.

[qouoq]

New init message:

McFly: Importing shell history for the first time. This may take a minute or two...Insert did not work due to 'nul byte found in provided data at position: 0'. Command was ''
done.

[cantino]

Thanks @qouoq!