assert-rs / snapbox

Snapshot testing for a herd of CLI tests
docs.rs/trycmd
Apache License 2.0
137 stars 18 forks source link

Can a trycmd file have a per-command CWD? #43

Open epage opened 2 years ago

epage commented 2 years ago

This would offer some flexibility if it is obvious and doesn't complicate the syntax since we have to detect the end of one command's output from the start of the next.

pwinckles commented 2 years ago

Personally, I would like something like this. Here's my current use case.

I have a cli that does a transformation on a set of input files. Ideally, I'd like to be able to define a single *.in directory for a given *.trycmd file, write multiple test cases in the file, and have each of those test cases map to a different *.out directory. This would allow me to run the cli multiple times with different arguments against the same input data and assert that the unique outputs are correct. To do this currently I'd need to create an input directory, output directory, and trycmd file per test case.

epage commented 2 years ago

The main question for doing this is syntax. The more involved syntax we have, the more likely we are to collide with application output. Already we have the following reserved:

I have a cli that does a transformation on a set of input files. Ideally, I'd like to be able to define a single .in directory for a given .trycmd file, write multiple test cases in the file, and have each of those test cases map to a different *.out directory. This would allow me to run the cli multiple times with different arguments against the same input data and assert that the unique outputs are correct. To do this currently I'd need to create an input directory, output directory, and trycmd file per test case.

How come you are wanting multiple test cases in a .trycmd file?

For me, I keep test cases separate, whether I'm using .trycmd or .toml except when I'm using trycmd to verify documentation (like clap's tutorials).

pwinckles commented 2 years ago

After your response in the other issue, I dug through what the toml files can do, and see that I can reuse an input directory with them by setting fs.base.

To me, it is desirable to write multiple test cases per file because it seems like a natural way to organize similar types of test cases and avoids a proliferation of small files.

epage commented 2 years ago

Note that we also have https://github.com/assert-rs/trycmd/issues/25

pwinckles commented 2 years ago

The main question for doing this is syntax. The more involved syntax we have, the more likely we are to collide with application output.

What about something like:

# fs.sandbox = true

$ write = "file.txt = Goodbye\nWorld" cat = "file.txt" bin-fixture

Where lines prefixed with "# " are used to configure trycmd settings?

epage commented 2 years ago

If we supported those comments anywhere, then can conflict with markdown output. If we only support it before the first command or in the markdown infostring then it doesn't help us with per-command actions like setting CWD. We could maybe do $ # fs.sandbox = true but parsing comments gets into a weird area because users might be adding comments if they are using the file for documentation (speaking of, it doesn't look like we support comments?).

If I didn't have to worry about conflicts with users, my ideal CWD syntax would be

parent/child $ cmd