mitsuhiko / insta

A snapshot testing library for rust
https://insta.rs
Apache License 2.0
2.26k stars 102 forks source link

control directory for .snap.new files #621

Closed fowles closed 1 month ago

fowles commented 2 months ago

I am working on a project that uses bazel (which is very strict about hermiticity and directories). In order to make insta play well in this ecosystem, I would like to be able to designate a specific directory that it will write output files to and then to pass that directory to insta review for acceptance.

Roughly something like:

INSTA_NEW_SNAP_DIR=... $TEST_RUNNER
INSTA_NEW_SNAP_DIR=... cargo insta review

Do you think that such a change would be accepted if offered?

max-sixty commented 2 months ago

Does snapshot_path do this? https://github.com/mitsuhiko/insta/blob/8753f2cc59284bd8bc8c645594e524fd16495423/insta/src/settings.rs#L480-L487

max-sixty commented 2 months ago

I haven't used bazel much myself, but it's also worth checking out INSTA_WORKSPACE_ROOT (and may require CARGO_MANIFEST_DIR until #614 is merged) to set paths across all tests. @thoughtpolice had similar-sounding issues in https://github.com/mitsuhiko/insta/issues/575.

Keen to support this sort of setup, so will close the gap if one still exists.

fowles commented 2 months ago

My current (very hacky) environment has a test:

use insta;

#[test]
fn test_all() {
  insta::with_settings!({snapshot_path => "/Users/beaker/dev/bronto/bronto/tool/e2e/"}, {
    insta::glob!("e2e/*.c", |path| {
      let input = std::fs::read_to_string(path).unwrap();
      insta::assert_snapshot!(input);
    })
  });
}

which when run fails with trying to write the .snap.new file (I think).

$ bazel test //bronto/tool:runner_test --test_env=INSTA_WORKSPACE_ROOT=$(pwd) --test_env=CARGO_MANIFEST_DIR=$(pwd)
WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
INFO: Analyzed target //bronto/tool:runner_test (0 packages loaded, 4556 targets configured).
FAIL: //bronto/tool:runner_test (see /private/var/tmp/_bazel_beaker/66706307a1c7f7ec3458df3c2b2f96d7/execroot/_main/bazel-out/darwin_x86_64-fastbuild/testlogs/bronto/tool/runner_test/test.log)
INFO: From Testing //bronto/tool:runner_test:
==================== Test output for //bronto/tool:runner_test:

running 1 test
test test_all ... FAILED

failures:

---- test_all stdout ----
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot file: bronto/tool/e2e/runner__all@id.c.snap
Snapshot: all@id.c
Source: bronto/tool/runner.rs:8
Input file: bronto/tool/e2e/id.c
────────────────────────────────────────────────────────────────────────────────
Expression: input
────────────────────────────────────────────────────────────────────────────────
+new results
────────────┬───────────────────────────────────────────────────────────────────
          0 │+int id(int noep) { return noep; }
────────────┴───────────────────────────────────────────────────────────────────
thread 'test_all' panicked at bronto/tool/runner.rs:8:7:
called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I have played around with a bunch of things to help it find the .snap files and can get that to work reliably, however, whenever the test wants to write a .snap.new it fails with a permission error. That is the setting I am trying to control (without much luck).

max-sixty commented 2 months ago

@fowles I don't get the permission error yet, but I see some other problems. Will have a look. At the very least we should say what file it was trying to write to...

max-sixty commented 1 month ago

@fowles #631 should give us a better error — do you want to try that?

I fixed one other issue. I have a couple of other things I want to look at, but otherwise don't know a precise reason behind the issue you see.

fowles commented 1 month ago

Thanks! I will try this in the next day or two.

max-sixty commented 1 month ago

I also refactored the code that handles paths, so I would now be quite surprised if INSTA_WORKSPACE_ROOT doesn't work, though not 100% as don't have a test case.

If you find there is still an issue, please reopen / add a new issue and I'll investigate.