Test "trivial_shell_nix" is flaky

curiousleo commented 4 years ago

This failed on master: https://travis-ci.org/target/lorri/jobs/604982303#L1682

---- tests::trivial_shell_nix stdout ----

thread 'tests::trivial_shell_nix' panicked at 'stdout:

stderr:error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I), at (string):2:17

', src/main.rs:100:9

note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

stack backtrace:

   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace

             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39

   1: std::sys_common::backtrace::_print

             at src/libstd/sys_common/backtrace.rs:71

   2: std::panicking::default_hook::{{closure}}

             at src/libstd/sys_common/backtrace.rs:59

             at src/libstd/panicking.rs:197

   3: std::panicking::default_hook

             at src/libstd/panicking.rs:208

   4: std::panicking::rust_panic_with_hook

             at src/libstd/panicking.rs:474

   5: std::panicking::continue_panic_fmt

             at src/libstd/panicking.rs:381

   6: std::panicking::begin_panic_fmt

             at src/libstd/panicking.rs:336

   7: lorri::tests::trivial_shell_nix

             at src/main.rs:100

   8: lorri::tests::trivial_shell_nix::{{closure}}

             at src/main.rs:96

   9: core::ops::function::FnOnce::call_once

             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libcore/ops/function.rs:231

  10: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once

             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/liballoc/boxed.rs:702

  11: __rust_maybe_catch_panic

             at src/libpanic_unwind/lib.rs:87

  12: test::run_test::run_test_inner::{{closure}}

             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panicking.rs:272

             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panic.rs:388

             at src/libtest/lib.rs:1468

curiousleo commented 4 years ago

It looks like nix-instantiate fails with the error message

file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)

I don't understand how, given that the test already passes -I:

    #[test]
    fn trivial_shell_nix() -> std::io::Result<()> {
        let out = std::process::Command::new("nix-instantiate")
            // we can’t assume to have a <nixpkgs>, so use bogus-nixpkgs
            .args(&["-I", "nixpkgs=./nix/bogus-nixpkgs/"])
            .args(&["--expr", TRIVIAL_SHELL_SRC])
            .output()?;
        assert!(
            out.status.success(),
            "stdout:\n{}\nstderr:{}\n",
            std::str::from_utf8(&out.stdout).unwrap(),
            std::str::from_utf8(&out.stderr).unwrap()
        );
        Ok(())
    }

curiousleo commented 4 years ago

Sanity check shows that calling Command::args multiple times is not the issue here since they just get appended: https://doc.rust-lang.org/src/std/process.rs.html#535-542

Profpatsch commented 4 years ago

@curiousleo did the merged PR fix this issue?

curiousleo commented 4 years ago

@curiousleo did the merged PR fix this issue?

No, #250 just added some checks, it did not aim to fix the issue.

target / lorri

Test "trivial_shell_nix" is flaky #250