sharkdp / hyperfine

A command-line benchmarking tool
Apache License 2.0
22.91k stars 366 forks source link

Inconsistent behavior of Integration tests using `cat` on Windows #776

Open ritvikos opened 3 days ago

ritvikos commented 3 days ago

The following tests (in tests/integration_tests.rs) have issues in Windows:

  1. can_pass_input_to_command_from_a_file - fails,
  2. fails_if_invalid_stdin_data_file_provided - passes, but false positive due to assertion.

Reason: The use of cat command (supported in unix-like environments) OS: Windows 11


logs:

---- can_pass_input_to_command_from_a_file stdout ----
thread 'can_pass_input_to_command_from_a_file' panicked at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\ops\function.rs:250:5:
Unexpected failure.
code=1
stderr=``````
\'cat\' is not recognized as an internal or external command,\r
operable program or batch file.\r
Error: Command terminated with non-zero exit code 1 in the first benchmark run. Use the \'-i\'/\'--ignore-failure\' option if you want to ignore this. Alternatively, use the \'--show-output\' option to debug what went wrong.
command=`"C:\\Users\\ritvikos\\Desktop\\OSS\\hyperfine\\target\\debug\\hyperfine.exe" "--runs=1" "--input=example_input_file.txt" "--show-output" "cat"`
code=1
stdout="Benchmark 1: cat\n"
stderr=```
\'cat\' is not recognized as an internal or external command,\r
operable program or batch file.\r
Error: Command terminated with non-zero exit code 1 in the first benchmark run. Use the \'-i\'/\'--ignore-failure\' option if you want to ignore this. Alternatively, use the \'--show-output\' option to debug what went wrong.


Stack Backtrace:

stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/std\src\panicking.rs:662
   1: core::panicking::panic_fmt
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/core\src\panicking.rs:74
   2: core::panicking::panic_display<assert_cmd::assert::AssertError>
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\panicking.rs:264
   3: assert_cmd::assert::impl$35::panic::panic_cold_display<assert_cmd::assert::AssertError>
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\panic.rs:100
   4: assert_cmd::assert::AssertError::panic<assert_cmd::assert::Assert>
             at C:\Users\ritvikos\.cargo\registry\src\index.crates.io-6f17d22bba15001f\assert_cmd-2.0.16\src\assert.rs:1033
   5: core::ops::function::FnOnce::call_once<assert_cmd::assert::Assert (*)(assert_cmd::assert::AssertError),tuple$<assert_cmd::assert::AssertError> >
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\ops\function.rs:250
   6: enum2$<core::result::Result<assert_cmd::assert::Assert,assert_cmd::assert::AssertError> >::unwrap_or_else<assert_cmd::assert::Assert,assert_cmd::assert::AssertError,assert_cmd::assert::Assert (*)(assert_cmd::assert::AssertError)>
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\result.rs:1456
   7: assert_cmd::assert::Assert::success
             at C:\Users\ritvikos\.cargo\registry\src\index.crates.io-6f17d22bba15001f\assert_cmd-2.0.16\src\assert.rs:158
   8: integration_tests::can_pass_input_to_command_from_a_file
             at .\tests\integration_tests.rs:305
   9: integration_tests::can_pass_input_to_command_from_a_file::closure$0
             at .\tests\integration_tests.rs:304
  10: core::ops::function::FnOnce::call_once<integration_tests::can_pass_input_to_command_from_a_file::closure_env$0,tuple$<> >
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library\core\src\ops\function.rs:250
  11: core::ops::function::FnOnce::call_once
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14\library/core\src\ops\function.rs:250
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
ritvikos commented 3 days ago

PR #777 😄