josecelano / cargo-pretty-test

A Rust command that prettifies the ugly `cargo test` output into a beautiful one.
https://users.rust-lang.org/t/cargo-test-output-with-indentation/100149
135 stars 1 forks source link

add integration test #6

Closed zjp-CN closed 11 months ago

zjp-CN commented 11 months ago

A few changes:

zjp-CN commented 11 months ago
test check_text ... FAILED

failures:
error: test failed, to rerun pass `-p cargo-pretty-test --test mocking_project`

---- check_text stdout ----
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: check_text
Source: tests/mocking_project.rs:24
────────────────────────────────────────────────────────────────────────────────
Expression: cargo_test()
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
    8     8 │ 
    9     9 │ failures:
   10    10 │ 
   11    11 │ ---- submod::panic::panicked stdout ----
   12       │-thread 'submod::panic::panicked' panicked at tests/integration/src/lib.rs:8:13:
   13       │-explicit panic
         12 │+thread 'submod::panic::panicked' panicked at 'explicit panic', tests/integration/src/lib.rs:8:13
   14    13 │ note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
   15    14 │ 
   16    15 │ 
   17    16 │ failures:
────────────┴───────────────────────────────────────────────────────────────────

Well, it's odd to see the error differs on stable and nightly Rust. I use nightly Rust which produces the old snapshot.

josecelano commented 11 months ago

Hi @zjp-CN, I like the idea of having a mocking project. Good idea!

I think it decouples tests from the current cargo test output format.

zjp-CN commented 11 months ago

It's weird that the second to last CI failed: cargo test --tests --benches --examples --workspace --exclude integration --all-targets --all-features works fine on my machine with both nightly and stable Rust. (fixed)


Here's another problem that we don't have a tree that shows the distinct types of test. cargo test displays tests from lib/bins/tests/examples/benches/docs...

$ cargo t
running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/main.rs (target/debug/deps/cargo_pretty_test-8c7b430bdf9bb007)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/golden_master_test.rs (target/debug/deps/golden_master_test-4199e959280bbe2f)

running 1 test
test golden_master_test ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/mocking_project.rs (target/debug/deps/mocking_project-bcadb14f9c92d30f)

running 2 tests
test snapshot_testing_for_parsed_output ... ok
test snapshot_testing_for_pretty_output ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.08s

   Doc-tests cargo-pretty-test

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
$ cargo r
running 0 tests
test
├─ ✅ golden_master_test
├─ ✅ snapshot_testing_for_parsed_output
└─ ✅ snapshot_testing_for_pretty_output

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.08s

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
zjp-CN commented 11 months ago

Oh I didn't fix the parsing issue yet. I'm working on it.

josecelano commented 11 months ago

Hi @zjp-CN I guess you know it, but there is also the option to get the output in JSON format:

cargo test -- -Z unstable-options --report-time --format json

Maybe that could help. Although it's also a flat format, at least we do not have to deal with regular expressions.

zjp-CN commented 11 months ago

get the output in JSON format

I didn't know it. It doesn't work on stable Rust. If it's stablized, we can try it out.

I think this PR can be merged now, though to deal with types of test we need another PR.

josecelano commented 11 months ago

ACK 61f0af48e51047bfc9d2199d9347396167991603