Critical error (SIGSEGV/ SIGBUS) with very long PATH variable

[dmilith]

Problem cargo build fails with SIGBUS / SIGSEGV when PATH variable is long (1942 chars in my case)

Steps I tried with Meilisearch repository:

git clone https://github.com/meilisearch/MeiliSearch.git
cd MeiliSearch
export PATH="/User/.cargo/bin:/Services/Libidn2/exports:/Services/Libarchive/exports:/Services/Git/exports:/Services/Pinentry/exports:/Services/Tar/exports:/Services/Nghttp2/exports:/Services/Geoip/exports:/Services/Siege/exports:/Services/Sofin11/exports:/Services/M4/exports:/Services/Bison/exports:/Services/Openldap-lib/exports:/Services/Gdbm/exports:/Services/Leveldb/exports:/Services/Curl/exports:/Services/Neon/exports:/Services/Gold/exports:/Services/Mc/exports:/Services/Slang/exports:/Services/Ccache/exports:/Services/Pcre/exports:/Services/Syslog-ng/exports:/Services/Freetype/exports:/Services/Bison27/exports:/Services/Sofin12/exports:/Services/Opus-tools/exports:/Services/Llvm/exports:/Services/Pkcs11-helper/exports:/Services/Nasm/exports:/Services/Glib/exports:/Services/Pcre2-jit/exports:/Services/X265/exports:/Services/Atop/exports:/Services/Libheif/exports:/Services/Gettext/exports:/Services/Msgpack/exports:/Services/Dovecot/exports:/Services/Libde265/exports:/Services/Ack/exports:/Services/Python27/exports:/Services/Apr-util/exports:/Services/Htop/exports:/Services/Clamav/exports:/Services/Clang/exports:/Services/Perl/exports:/Services/Nghttp2-utils/exports:/Services/Popt/exports:/Services/Glances/exports:/Services/Courier/exports:/Services/Tmux/exports:/Services/Lsof/exports:/Services/Gd/exports:/Services/Qemu/exports:/Services/Phc-winner-argon2/exports:/Services/Pcre2/exports:/Services/Ncurses/exports:/Services/Perl-static/exports:/Services/Libtool/exports:/Services/Gnutls/exports:/Services/Meson/exports:/Services/Flex/exports:/Services/C-ares/exports:/Services/Riak/exports:/Services/Pcre-jit/exports:/Services/Icu/exports:/Services/Cmake/exports:/Services/Clojure/exports:/Services/Openldap/exports:/Software/Zsh/exports:/Software/Git/exports:/Software/Mc/exports:/Software/Tmux/exports:/Software/Htop/exports:/Software/Sofin/exports:/Software/Zsh/exports:/bin:/usr/bin:/sbin:/usr/sbin:/usr/pkg/bin:/usr/pkg/sbin"
source $HOME/.cargo/env
cargo build

Solution(s)

git clone https://github.com/meilisearch/MeiliSearch.git
cd MeiliSearch
export PATH="/bin:/usr/bin:/sbin:/usr/sbin:/usr/pkg/bin:/usr/pkg/sbin"
source $HOME/.cargo/env
cargo build

Notes

System: HardenedBSD 12.1 (FreeBSD) x86_64 Rust: 1.43.1 from rustup (standard)

[alexcrichton]

Could you provide more logs of the run? Are you sure cargo itself is segfaulting and not something it's spawning?

[dmilith]

Here's output from truss -s 10000 cargo build 2> long-path-cargo-build-after-clean.truss.log with long PATH:

long-path-cargo-build-after-clean.truss.log

And here's one from export PATH="$HOME/.cargo/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/pkg/bin:/usr/pkg/sbin" && truss -s 10000 cargo build 2> long-path-cargo-build-after-clean-short-path.truss.log long-path-cargo-build-after-clean-short-path.truss.log

[alexcrichton]

Is it possible to get a stack trace of the segfault? (e.g. via a core dump or something like that). Given the logs this looks unrelated to PATH and that's probably just moving things around on the heap/stack. My best guess is stack overflow happening, but my second guess is that the resolver tried to eat too much memory and then malloc failed and something tried to read/write at null.

In any case a stack trace would help debug this further.

[dmilith]

I did core dump, and this is what I get:

[1592294455] cb4-bh:~/MeiliSearch λ lldb /User/.cargo/bin/cargo -c /User/cargo.core
(lldb) target create "/User/.cargo/bin/cargo" --core "/User/cargo.core"
Core file '/User/cargo.core' (x86_64) was loaded.
(lldb) r
There is a running process, kill it and restart?: [Y/n] n
(lldb) bt
* thread #1, name = 'cargo', stop reason = signal SIGSEGV
  * frame #0: 0x00000000013a5a5f cargo`regex_syntax::parser::Parser::parse::h2ff55ce8967781fc + 18335
    frame #1: 0x00000000013a1c44 cargo`regex_syntax::parser::Parser::parse::h2ff55ce8967781fc + 2436
    frame #2: 0x000000000155452d cargo`std::sys::unix::process::process_common::Stdio::to_child_stdio::h0aa73c7e7cbda445 [inlined] _$LT$i32$u20$as$u20$std..sys..unix..IsMinusOne$GT$::is_minus_one::heffe42119eacacc5 at mod.rs:120:12
    frame #3: 0x000000000155452c cargo`std::sys::unix::process::process_common::Stdio::to_child_stdio::h0aa73c7e7cbda445 [inlined] std::sys::unix::cvt::ha9ce4582b85c7289 at mod.rs:128
    frame #4: 0x000000000155452c cargo`std::sys::unix::process::process_common::Stdio::to_child_stdio::h0aa73c7e7cbda445 [inlined] std::sys::unix::fd::FileDesc::duplicate::h4f40e2c2fafcd998 at fd.rs:248
    frame #5: 0x00000000015544da cargo`std::sys::unix::process::process_common::Stdio::to_child_stdio::h0aa73c7e7cbda445 at process_common.rs:325
    frame #6: 0x000000000129146a cargo
    frame #7: 0x00000000012909c5 cargo`core::ptr::real_drop_in_place::h29dfeb3fa68505c7 + 357
    frame #8: 0x00000000011cc4b1 cargo`clap::completions::zsh::get_subcommands_of::hecb0fa8f250c6555 + 1297
    frame #9: 0x00000000011cbc07 cargo`clap::completions::zsh::get_args_of::hbbb4d926bca0e43b + 13015
    frame #10: 0x000000000117d200 cargo`rustup_init::rustup_mode::show::hf42fff5c60e3619c + 4320
    frame #11: 0x00000000011953de cargo`rustup_init::run_rustup_inner::h3c93911d8b708155 + 174
    frame #12: 0x0000000001187e55 cargo`rustup_init::self_update::install_bins::ha8e1aa7878223cb6 + 325
    frame #13: 0x000000000118a7f6 cargo`rustup_init::self_update::uninstall::h866c9739f369b243 + 2966
(lldb)

[dmilith]

I can confirm that this issue doesn't happen on Ubuntu 18 host… So that PATH is not core of the problem… which makes it even more interesting.

[ehuss]

From the stack trace, it doesn't look like this is a Cargo issue. The cargo executable is actually a rustup wrapper which is responsible for selecting the correct version of cargo to run. I'm a little confused by the stack trace, since some of the frames look a little random. I tried reproducing on a normal FreeBSD system, but I can't get it to fail. Perhaps there is something unique to HardenedBSD?

[dmilith]

Good to know. Will investigate it further. Thanks!