Open e2 opened 8 years ago
myronmarston
commented
8 years ago Thanks for looking into this, @e2. You're absolutely right that our relish docs could use an overhaul. I'm not sure that I like the organization you've come up with, though; I'll have to think about it some more. Having only a few top-level categories and having some really large categories (e.g. one has 20 items) will make it hard to navigate, I think.
e2
commented
8 years ago Having only a few top-level categories and having some really large categories (e.g. one has 20 items) will make it hard to navigate, I think.
I was looking for reasonable "overall" categories that can be broken down by "what people are looking for". E.g. "exclude pattern option" has nothing to do with "Excluding lines from the backtrace.", even though both currently appear close to each other.
I absolutely agree that too few top-level categories is bad! I'd avoid "trees" as much as possible.
Ideally, there'd be no more than e.g. 10 items per category. If you have ideas for finer categories, that better, because these are just my initial suggestions.
E.g. "controlling output" could be split into "controlling warnings and deprecations" vs whatever other grouping.
Here are examples why I chose these areas:
spec/examples.txt location - and I can't quickly see if it's documented in relish or not.rspec --help, so a functional grouping in Relish is better--next-failure is a very useful temporary workflow option - but it's buried under a list of other command lines parameters which I almost never useThe irony is, that the more I know where everything is and the fast I learn to navigate, the less I need to even go to the docs. Probably not a good thing though, because it only means I'm memorizing things.
That's all I can think of. Grouping alone has helped me already "memorize" which option is where, so I'm already too "tainted" to figure out a structure for more casual RSpec users.
JonRowe
commented
8 years ago Your suggested categories are too few and too broad, especialy "integration" which is a bit like saying "Misc" or "utility". Also not everyone is going to be looking for information the same way you do, if you organise everything like "cheat sheets" then you could argue "configuration options" should all be documented together somewhere as sometimes people won't know exactly what they are looking for.
Why don't you write your own cheat sheet if you're having to look up the docs all the time? It'll be easier than trying to rewrite our docs to suit everyone. Not that I'm against reorganising and improving our docs just your needs seem particularly unique.
e2
commented
8 years ago Your suggested categories are too few and too broad
Yes, intentionally. This is just the "minimal" separation to help work out if it makes sense and/or what other categories could make better sense. If I split it up into smaller categories at this point it would hide the overall "pattern".
especialy "integration" which is a bit like saying "Misc" or "utility".
Notice that 2 existing entries already have "integration" in their name.
From a maintainer's point of view (like yours), sure, it probably makes no sense whatsoever.
From a user's perspective it means: "getting RSpec to work with my favorite editor, my favorite script, my favorite formatter, my favorite mock library, output to HTML, etc.".
It's not a good name in of itself, but it does separate those options from everything else. E.g. if a person is writing specs, they don't care about integration and vice-verse. A category like that helps users skip irrelevant chunks of options they don't care about.
Also not everyone is going to be looking for information the same way you do
Yes, and not everyone sees the documentation the same way a maintainer does. I've realized the difference - and what I've been missing throughout the years. So I'm sharing feedback so other users can become proficient faster with much less frustration than I used to have. I'm still frustrated every time I spend more than 5 seconds trying to find what I need in the Relish docs. Probably especially because after so much time I expect to find things quickly.
I'm just presenting what would've helped me, but it's "too late" now. It's not "too late" for other users, though. That's why if what I say makes sense, that's great. If I'm too weird/dumb/alien, then feel free to just close this. I'm just giving feedback about what I WISH I had years ago.
The relish docs are frustrating, because they're nice to use, but the time it takes to find something still leaves me frustrated, clicking around and sometimes just using Google to find what I need. E.g. I may expect "how to configure spec directory", but instead it's a cryptic "Configuration -> Pattern". To me, almost everything (except example syntax) is "configuration". Even "command-line" is pretty much configuration if I use scripts for running RSpec.
you could argue "configuration options" should all be documented together somewhere
Yes. That's why I learned to just copy configurations from other projects to save the frustration of clicking around the docs trying to work out what X is called in "RSpec lingo". E.g. I'm always shocked what I find under "Hooks", because to me those are more like a part of the DSL (because of how I use contexts).
Of course, it's easier to find something if you're a maintainer - but that's not who those docs are for...
as sometimes people won't know exactly what they are looking for.
That's 99% of the cases :D
It's rare that someone would want to find "all the available commandline options". Because it's not like someone wakes up one day and decides "I want to know all the possible configuration options, no matter if they're for filtering, ordering, turning off monkey-patching, backtrace filtering, deprecation output stream, profiling". People are usually either writing specs, configuring their project, doing one-off overrides to run rspec manually with selected specs or getting it to work with their editor/tool. But they're not doing all that all at once and all at the same time. So it doesn't make sense for the doc navigation to optimize for that most exotic case.
Why don't you write your own cheat sheet if you're having to look up the docs all the time?
I wish I did, years ago. That way I'd benefit a lot, even just from preparing the cheat-sheet. Except it would be out of date fast, it wouldn't be in the search engines, etc. You can search for existing cheatsheets to get my point. If I'm "different", it's probably because I tweak my settings a lot.
I opened this issue not as something I need for myself, but something I wish I had years ago - but I couldn't understand why until today. I never stopped to think why using the relish docs was so frustrating, even though they looked so nice.
Just as an extreme example: I pretty much despite Windows (just my personal gripes), so I have no idea why the second "Configuration" option is about setting colors in Windows. And I'm never going to be interested in the "deprecation output stream". Those are exotic options that I doubt people use.
Or, why is "Spec files" in the main menu? It's pretty much a "dead" option, while "Bisect" is just fantastic to know about - yet it's buried in the "Command line" section. It probably shouldn't even be in a category, not even in "integration" but in the top level - as an important feature.
Other categories could be "typical everyday options" and "workarounds for issues". That's a useful separation, but those categories would be huge. I'd suggest a "Writing specs / DSL" category, which would include:
It'll be easier than trying to rewrite our docs to suit everyone.
I'm just thinking of newbies that want to make quick progress by quickly finding what they will likely need. Personally, I pretty much just run rspec --help and copy spec_helper.rb and .rspec* files from other projects. And if I can't find what I need in 5 seconds, I get frustrated and use Google to search Relish, then I'm frustrated, because I still have to switch the docs from RSpec 2.4 to 3.4, because Google thinks the older docs are "more relevant" or something.
Not everyone understands "maintainer terms". I'm sure the word "hooks" makes sense, but the only time I ever see that word in the context of RSpec is the Relish documentation. To me, "hooks" suggest something to do with advanced integrations. Maybe I'm old or something.
I'm almost never looking for "helper methods", but I would expect let() to be classified as part of the syntax. describe and context would seem helper methods too, but the're not named that way.
In short: the names for categories are adapted to their contents, and not to what a user is searching for.
I hope that explains things better.
If you believe I'm the only person frustrated and everyone else is thrilled with the way the docs are organized, I'd be happy to find out I'm the only weird one here. Close this issue if so.
If you can propose a better structure and you want my feedback (based on how I would've wanted things to be years ago that would've helped), I'd be glad to. Just ask.
Just so it's clear: I'm here to make RSpec better by sharing frustrations and counting on you guys to come up with brilliant solution. I'm not complaining, just voicing frustrations I feel others might also have. Frustrations that may go away just by reorganizing a few things.
I'd create a PR, but I don't want to put in the time right of the bat and then hear "why don't you create your own cheatsheet". So if you have objections that would make a PR pointless, I'd like to know before I try to really prepare a "final" structure, possibly add docs for missing sections/features (while I'm at it) and organize the PR into logical commits - where at least the least "controversial" changes are separated from those that need better ideas.
Maybe I should suggest lots of smaller categories first - and then maybe talk about merging them so there aren't 2-3 entries per category? (Which is the opposite approach of what I tried here).
My goal: if I can help other users become more RSpec-proficient with less frustration (like the ones I had) everyone wins down the road. Otherwise, frustrated people won't complain out of "being polite" (like I never did) and everyone loses.
I'm out of things to say, so ask if you like.
myronmarston
commented
8 years ago @e2: we previously went through a total re-do of the rspec-mocks cukes to similarly make them better organized. Compare what the rspec-mocks cukes used to be in 2.99:
https://relishapp.com/rspec/rspec-mocks/v/2-99/docs
...to what they were in 3.1:
https://relishapp.com/rspec/rspec-mocks/v/3-1/docs
The work was discussed in rspec/rspec-mocks#591 and then done in rspec/rspec-mocks#695.
Similar to what you've suggested here, we re-organized the rspec-mocks cukes to focus on end-user use cases rather than organizing things from an implementation perspective.
Does the organizational approach for rspec-mocks satisfy what you'd like to see here for rspec-core?
FWIW, I agree that the current organization (which has separate directories for command line options vs other config options -- as if the user is doing to know which way of doing config is going to support the feature they are looking for) is not good, and am keen to see it improved. But I think we need to come up with more top-level categories (and keep them smaller) to make it feasible for the user to navigate. Maybe the organizational approach for rspec-mocks can help suggest organizational changes for this repo?
e2
commented
8 years ago we previously went through a total re-do of the rspec-mocks cukes
Yes, I don't recall having issues navigating there. It's actually probably where I got the idea from - I subconsciously expected the docs for rspec to be as organized as those for rspec-mocks.
I especially like how the basics are at the top and the 4 "special-case" entries (legacy code, old syntax, outside rspec and changelog) are at the bottom, out of the way.
The work was discussed in rspec/rspec-mocks#591 and then done in rspec/rspec-mocks#695.
I really resonate with the discussions there! Awesome work!
Even just moving legacy-related features, changelog, etc. to the bottom would make things significantly less overwhelming and confusing.
I remember how in 2.14 I used to waste time trying to find good and up-to-date docs on expectations and mocks. I think ever since 3.0 the docs in Relish helped me not only navigate better, but often "accidentally" learn about other features. Features I'd be aware of and look up in later situations.
But I think we need to come up with more top-level categories (and keep them smaller)
I absolutely agree!
Here is an alternative grouping, where the above are broken down in to smaller groups:
Bold items are those that I feel are too important for users to miss.
e2
commented
8 years ago I think rspec-core docs are especially important, since they lead with creating the first impression of RSpec. I'd expect them to more focused on user friendliness (especially when setting up a project) than docs for expectations or mocks.
"RSpec documentation" and "rspec-core docs" are expected to by somewhat synonyms for users, so if rspec-core docs are confusing and overwhelming, that sets up bad expectations, no matter how organized rspec-mocks or rspec-expectations are.
myronmarston
commented
8 years ago Your updated suggested organization is definitely an improvement over the earlier ones, IMO.
A few random thoughts, though:
subject exists more for niche situations and for integration with extensions, and should not be emphasized, IMO. Helpers methods and modules are a more simple, fundamental, "just ruby" technique and should be given more prominence.spec/integration -- that way we retain the end-to-end coverage the cuke previously provided without surfacing it in our relish docs.Anyhow, are you willing to take on this work, @e2? If so, I think the best way forward is if you could start taking baby steps with a bunch of small PRs. Large PRs are harder to review and will take much longer to merge. If you could take many small, logically distinct steps in many PRs we could review each individually and get the merged (assuming the small steps can stand on their own, of course) and I think it'll go faster for all involved.
One thing to be aware of: we're probably going to ship RSpec 3.5 in the next couple weeks and it would be nicer for users if the re-organization was released at a single point in time instead of being staggered across multiple releases. As such, we may want to create an integration branch you can target your PRs at so that we can begin merging small PRs w/o worrying about them going out with 3.5 before the entire thing is done.
Thoughts?
e2
commented
8 years ago Ok, to start small until I get a hang of things, I created: https://github.com/rspec/rspec-core/pull/2262
Once I work out the process, I can open other small "independent" PRs (e.g. for the "organizing specs" one). I started with the legacy section, since it should be easiest to discuss, the least controversial and with least impact to novice users.
I'll avoid modifying feature files themselves until the overall structure improves first (since coverage needs to be properly tested and related changes have to be done carefully and piece-by-piece).
I'll focus on master/3.5. If PRs are accepted, I can backport them to 3.4, etc. (If they're worth it).
There are some things with your organization suggestions that I'm not quite happy with,
I'm very open to suggestions. It's a draft-of-a-draft for me, so I'm having just as much (if not more) trouble verbalizing what I think and feel. I'll do my best to express my doubts in the PRs.
subject exists more for niche situations
I find it very useful when objects have many edge cases related to initializing an object. Probably most projects test initialization through testing accessors after initialization, while I often test that initializing properly passed the right options to given mocks of members. And that each edge case is clear and concise.
It's not uncommon to pass arguments and an option has to an initializer - and a generic "subject" can really clean things up without requiring lots of custom helper methods. I can use context like with valid options and break it down into sub context, like with option :foo not provided, etc. Possibly a personal preference though...
xaviershay
commented
7 years ago Really like the direction here, leaving this one open - our docs definitely need a reorg!
I just realized that it's hard to quickly find docs on what I need at any given moment.
Even if the documentation is nice, many unrelated categories are clumped together: https://relishapp.com/rspec/rspec-core/v/3-4/docs/configuration
So here are some ideas on how to change/organize things:
First, I'd break up functionality into understandable categories:
(Note: individual items are in no order)
"Picking which tests to run" a.k.a "filtering" / "limiting" / "selecting" which examples are run
"Picking the order in which to run examples"
"Controlling output"
Integrations
Those are the main areas I'd use for navigation.
The "Picking which tests to run" area could be split into files and examples/groups - since that section is so big.
The reason why I chose those areas is: it's easier to see if RSpec can do what I'm looking for, or if I should open an issue or create a PR.
E.g. if I want to change something in the output, (e.g. backtrace filtering), I'd look in the "Controlling output" section. If I want to select which files to run, I can go to the "picking which tests to run" section and immediately see an exhaustive list of options I have.
Also this way, a user doesn't have to "read everything" to know what's available.
I considered the idea of separating configuration options from command-line options, but I really believe they should be together. Mostly because that way the reader can sometimes decide to solve their problem from the command-line or vice verse. And they're more likely to get such an epiphany if those options are together.
Thoughts?