Provide simple, concrete, contained example

[SUPERCILEX]

See #145 for what that looks like. There are currently no concrete examples unless you count the "Users" section, but those are prod crates that are big and complicated.

[SUPERCILEX]

This example should include insert_var examples as explained in https://github.com/assert-rs/trycmd/issues/140#issuecomment-1267542273

[epage]

What is the role of this example?

• Why do the trycmd production cases are not sufficient as examples?
• What cases do we care about covering, including
  • readme vs other documentation vs tests
  • trycmd vs toml
  • simple input/output, using the file system
  • Use of variables, wildcards, etc
• How this should fit within the existing documentation

[SUPERCILEX]

Why do the trycmd production cases are not sufficient as examples?

Why is reading the linux kernel not a great way to learn about operating systems? Why isn't it easy to learn html by just looking at the google.com source page? Why can't you just learn Rust by reading the stdlib? Because there is a massive amount of complexity in production code that completely detracts from the goals of education.

To be a little less snarky, when you're just learning something, you don't know what you're looking for. So the production examples become a game of "Can I find the trycmd relevant parts?" I think production examples actually answer a different question: why the library is useful. That is, after you've seen toy examples and understand how a thing theoretically works, now you can see how people use it in the real world.

What cases do we care about covering

• README: I just have beef with the minimal example because if you don't know what a trycmd file is (which you shouldn't since it's not an industry standard) then the example is completely meaningless. What the heck is a .trycmd file and where do I get one? And when you open the docs you have to scroll very far before you get to a spec of the trycmd language which is not useful when you're trying to figure out what trycmd even is. I mean, imagine if the rust-lang.org website just had a link to the grammar or the reference book.
  • The documentation is fine as is (assuming examples aren't considered documentation) because it's a reference. Once people get the gist of how the library works, they'll want to look up specific knobs in a reference style thing (but those knobs should have examples!).
  • I don't see how tests are relevant. I've personally never looked at a library's tests to try and understand how to use it.
• trycmd vs toml: I personally care more about trycmd because it's a friendlier way to use the lib, but toml examples could also be added as an advanced way to use the lib.
• simple input/output, using the file system: the more examples the merrier! The clap examples are simply amazing, so I'm honestly kind of confused about the resistance here given that you're the same dude. :)
• Use of variables, wildcards, etc: At the very least a basic ground up example is needed on how variables work. The fact that they're optional is pretty subtle, but very useful for clap tests.

How this should fit within the existing documentation

I mean, I really do think #145 is an improvement. Now that I know how variables work, I would demo that, but otherwise I think it's enough to get you started and not be left scratching your head for hours. Now I know you want everything to live in lib.rs, but that doesn't work when the example needs to be tested by the lib. Though actually, maybe we can just run trycmd on the lib.rs file? I don't really care where the example lives as long as trycmd is run on it, though putting the example in lib.rs does make it less realistic since most people won't be doing that.

[epage]

To be a little less snarky,

Please refrain from the snark. The snark, accusatory language, and combativeness have not been helpful to there being an healthy, productive dialog for improving things. This is a large part for why I have made minimal fixes to issues and closed them out; to put a stop to the endless, draining conversation.

The question was asked in earnest, to dig into the differences and to discover requirements. If behavior does not change, I will ask that you leave.

I'm honestly kind of confused about the resistance here given that you're the same dude. :)

See above.

[epage]

when you're just learning something, you don't know what you're looking for. So the production examples become a game of "Can I find the trycmd relevant parts?"

Maybe I wasn't clear enough or I misunderstood your answer. I was specifically referring to the trycmd's use of trycmd. While this is trycmd testing trycmd, it neatly breaks down trycmd's features so people can see different parts of it. There is little unrelated stuff to siphon out. For now, this also doesn't get into the normal weird cases that crop up when code is used to test itself.

This doesn't mean its a perfect solution but to understand the resources at hand, where they might work, and where they won't.

README: I just have beef with the minimal example because if you don't know what a trycmd file is (which you shouldn't since it's not an industry standard) then the example is completely meaningless. What the heck is a .trycmd file and where do I get one?

This is a good point and would be worth an issue on its own.

I mean, I really do think https://github.com/assert-rs/trycmd/pull/145 is an improvement.

I pointed out when I closed it why this does not fit narratively which is why I made sure to include this question.

Now I know you want everything to live in lib.rs, but that doesn't work when the example needs to be tested by the lib.

It doesn't have to all live on docs.rs but we need to be intentional about where it does live in a way that is clear to the end user. For example, maybe we create a cookbook directory that is full of example projects that highlight major workflows.

[SUPERCILEX]

I was specifically referring to the trycmd's use of trycmd.

Wait, are you talking about https://github.com/assert-rs/trycmd/tree/main/tests/cmd? Is looking at those what the example on the README was supposed to tell you? If so, that was a total r/woosh for me. Something as simple as saying "Here's how trycmd uses itself" instead of "Here's a trivial example" and then "Take a look inside $folderLink to see the trycmd files" would have made this all infinitely clearer.

That said, I took a look at the tests and while beneficial for their depth, they're not comprehensive minimal examples. You'll still end up having to jump around a bunch and put pieces together when you could instead have a single example that gets you 90% of the way there and leaves 10% for the tests to showcase or the reference etc.

this does not fit narratively

I guess we disagree on what constitutes a "workflow" then. To the point above, there should be one place where most of what you need will be shown and I don't count that as a workflow.

For example, maybe we create a cookbook directory that is full of example projects that highlight major workflows.

I guess? Again, I'm not thinking workflow here. The library just needs a basic what does it do example. Just the tagline has so many baked in assumptions for someone coming in with no knowledge: "trycmd is a test harness that will enumerate test case files and run them to verify the results, taking inspiration from trybuild and cram." I think the tagline is great if you're coming from trybuild or cram, but otherwise you don't know what the test case files are or what it means to verify them. The basic example I'm talking about seeks to answer those questions.

This is a key point that I feel like I'm not getting through: the example should start from nothing. No assumptions about having used cram or knowing what test case files are. So if the tagline were rewritten (I don't think it needs to be as long as its parts are explained), I would say "trycmd is a test harness that executes your binaries, ensuring that their output remains the same." My point is that most of the docs feel written with the assumption that you know how this style of library works and just need to get up-to-speed on the details. The example I've been trying to push is supposed to introduce you to that style of testing.

[epage]

this does not fit narratively

I guess we disagree on what constitutes a "workflow" then. To the point above, there should be one place where most of what you need will be shown and I don't count that as a workflow.

I feel there is a disconnect here as my discussion of narrative had nothing to do with covering different workflows but is more "does this fit within this specific location in the document". We have a lib doc comment that is enumerating different features and how they work and #145 tries to insert a full example covering multiple sections into one of those sections. This makes it harder to find if you happen to not be in that section and creates a large gap in between closely related sections, making the overall document harder to follow.

I guess? Again, I'm not thinking workflow here.

I feel this is getting stuck on the wrong point. This is a reply to an example I gave on how things could be structured but gets hung up on the example rather than talking structure.

I'll try to be more concrete. Say we have the following

• examples/
  • demo_trycmd
  • README.md (explains the different parts, is also tested)
  • Cargo.toml
  • src/
    • main.rs
  • tests/
    • `cli.rs
    • cmd/
  • demo_toml
  • README.md (explains the different parts)
  • Cargo.toml
  • src/
    • main.rs
  • tests/
    • `cli.rs
    • cmd/

We then link out to examples/demo_trycmd and examples/demos_toml from the README and from the lib's doc comment.

My comment about workflows was we could then add examples/read_files_{trycmd,toml}, examples/write_files_{trycmd,toml}, etc for the different types of applications people are writing and want tested. I feel like those three pretty much cover the major workflows (or styles of CLIs being tested).

[SUPERCILEX]

does this fit within this specific location in the document

Ohhh, gotya. Yeah, I guess I've been confused because I thought you wanted the example to be in lib.rs. Regarding where the example lives, I personally have no preference or opinion, that's totally up to you. And I think you make a great point about it taking up way too much space in the lib.rs.

gets hung up on the example rather than talking structure.

Completely accurate, I think we're on the same page now, thanks! :) The layout you presented seems perfectly reasonable to me, though I don't know if the demos need to be full blown crates. Then again, I don't think it hurts to have full blown crates, so totally up to you. As you mentioned, I'm more concerned about the content than where it shows up.

[epage]

I prefer a full crate, partly from the problems I saw in #145. It was trying to provide an example but content was scatted in places that weren't actually the right place for it and hoping the user picked up on that or had to explicitly describe what was intended. By having a full crate, we can show the user exactly how it would be and we can limit our comments in it to explaining those specific parts rather than mixing in caveats on the limitation of the documentation.

[SUPERCILEX]

I'm onboard with that. Do you want me to give it another go or you just throw me on for content review?

[epage]

Go ahead and give it another try