Feature request: Add 'User Contributed Notes' to Crystal docs

[simonhf]

As a Crystal newbie I found myself having to figure out how Crystal does certain stuff. That means spending lots of time searching and poring over the docs.

The good thing is that the docs are substantial. The bad thing is that they are a little terse, and worse, there are few or no examples. I get the impression that the docs might make more sense when a newbie has built up a certain critical mass of Crystal knowledge. So it's a kind of chicken and egg situation, and a definite friction to learning Crystal. Therefore, I suggest adding example sections and/or 'User Contributed Notes' features to the Crystal docs inspired by the aging but excellent PHP manual, e.g. [1].

[1] https://www.php.net/manual/en/function.getenv.php

[Blacksmoke16]

PRs to improve documentation are welcome. There are definitely areas where this can be improved. Read thru https://crystal-lang.org/reference/conventions/documenting_code.html if you haven't already.

User Contributed Notes probably isn't going to be possible natively. The current implementation of docs is just generating some HTML/JSON that can be displayed in a browser; i.e. there wouldn't be a place to persist those notes.

[asterite]

I think StackOverflow already covers this scenario and it's actually better because it responds questions to problems, not general notes about functions. Plus it's easier for us to do nothing, we don't need to moderate comments, etc.

[simonhf]

The problem is that PRs are only half of the battle. If a newbie has a question about X then they don't make a PR to ask the question :-) And they are definitely not going to discover the answer elsewhere and then make a PR with their finding :-) I have been asking in the Crystal gitter chat and getting answers from kind folks such as yourself, but although this works nicely for me, it is not very efficient for when the next newbie comes with a similar question.

there wouldn't be a place to persist those notes.

thinking out loud... If...

1. A unique github issue could somehow be tied / linked to a unique Crystal docs page, the User Contributed Notes could be much lower friction (easier to add a comment on a github issue than to make a PR) and just be github issues.
2. If the generated HTML for a given Crystal docs page could somehow display a sub web page (the associated github issue holding the User Contributed Notes) then a single Crystal docs page would have something very similar to the PHP User Contributed Notes feature for its docs...

[asterite]

Ruby doesn't have this, but whenever I want to find something about Ruby I find it in StackOverflow in under a minute. In fact, iñI find answers to things in Java, Elixir, Haskell, Elm. None of those have user notes. I don't think they are really needed.

[jhass]

PHP was the first programming language I learned and the comments/notes in their section was never helpful in my experience there. Too often it had outdated, confusing or conflicting advice by different users.

[simonhf]

I think StackOverflow already covers this scenario

Yep, it might be the next best thing. But IMO the difficultly with SO is often formulating the search keywords to find the answer to your question. Not a problem if you are posting a new question :-) Often you have to search over and over using different combinations of keywords... in addition to poring over the Crystal docs...

[simonhf]

PHP was the first programming language I learned and the comments/notes in their section was never helpful in my experience there.

I only learned PHP some years ago and I had the opposite experience to you. Sure there's lots of outdated, confusing or conflicting advice, but as an experienced developer it's easy to sift / scroll through that stuff on a single web page... what's more friction for me is constantly typing in new search terms to try and find the answer I'm looking for... I agree that for a learning-first-programming-language developer, then that sifting is going to be more challenging...

[simonhf]

Maybe it's as simple as each Crystal docs page using its unique title as a github tag like already used [1]. For example, [2] is entitled "module ENV" which could be turned into a github tag e.g. user-contributed-notes-module-env ?

[1] https://github.com/crystal-lang/crystal/issues?q=label%3Aplatform [2] https://crystal-lang.org/api/0.34.0/ENV.html

[asterite]

The thing is that there's not enough bandwidth in this open source project to do it. It's just too much work for a very small gain.

[asterite]

Where are the notes stored, and how do one adds them?

[simonhf]

not enough bandwidth in this open source project to do it

I hear you. And whether there is bandwidth or not then I'm hoping it's good to at least capture the idea in this issue for when bandwidth might be more plentiful :-)

However, I'm also assuming that the docs are generated by some kind of script. And I'm hoping that the project contributor who knows about that stuff will eventually read this issue :-) And I'm also hoping that it'd be only a few lines of code to get the generate page title, munge it into a github tag, and generate the HTML at the bottom of the page which shows all the github issues with that tag... :-) So from this -- maybe blissfully naive POV -- it seems like a potentially low hanging fruit, or?

[jhass]

Before we jump into technical architecture, did we agree yet they would actually be beneficial? In my opinion they could even be harmful, unless we rebuild stackoverflow essentially, that is a full fledged voting and third party editing system.

[jhass]

Github issues are not documentation, they are proposals and problem descriptions. Their purpose is to get obsolete.

[simonhf]

Where are the notes stored, and how do one adds them?

My thinking out loud proposal is to generate a web page footer for pages like [1], which would show the embedded github sub page [2] which shows the tagged notes just for that page.

So notes would be stored as github issues... just some of them are today. But they would be organized better and accessible right from the HTML Crystal docs.

[1] https://crystal-lang.org/api/0.34.0/ENV.html [2] https://github.com/crystal-lang/crystal/issues?q=label%3Ausercontributednotes%3Amoduleenv

[Blacksmoke16]

I vote to close. Given the docs are versioned you would essentially need an issue per version (in order to account for changes in between versions). If anything, a PR to improve existing documentation would be better than (and more maintainable) than supporting a separate set of user supplied documentation.

The problem is that PRs are only half of the battle. If a newbie has a question about X then they don't make a PR to ask the question :-)

Correct, they would either look on SO, make a thread in the forums, or ask in the Gitter channel. The answers from these sources could then be used to improve the documentation, gradually improving it over time.

[asterite]

And I'm hoping that the project contributor who knows about that stuff will eventually read this issue :-)

That would be me :)

And as I said, I don't think this is useful so at least I won't implement it.

[simonhf]

Well thanks for giving this request consideration. Much appreciated! And much better than few or no comments at all :-)

[Blacksmoke16]

@simonhf An action item that could result from this is a PR related to the type you had trouble with by adding more documentation and/or examples.

[simonhf]

An action item that could result from this is a PR related to the type you had trouble with by adding more documentation and/or examples.

Yeah, it's frustrating for me because as you know -- as a Crystal newbie -- I have asked many questions via gitter -- after having pored over the Crystal docs to try and minimize the gitter questions - and I learned a lot thanks to the generous attention of many people such as yourself on gitter, and... I think I could create PRs for at least a dozen doc pages, but... those PRs would not be a trivial amount of time for me to prepare, and on top of that, there would likely be a prolonged back and forth with the PR because I'd be trying to update the official docs, etc, and with totally new features like examples. It seems like a larger time investment and making docs is not my best skill :-) However, making notes about docs... that I can do! :-)

The frustration is there because it seems a waste that my questions on gitter will not end up benefiting other Crystal newbies like me over the months and years to come. The proposed solution mentioned above are to ask the newbie questions on the Crystal lang forum [1] with 456 questions currently, or on SO [2] with 554 questions currently. Interestingly I did have a question about "nested hash" tables of which there are 3 in the forum [3] and 1 relevant thread [4]... which I contributed to [5]! Unfortunately in this instance the contribution did not result in any useful feedback for me... whereas the similar question about nested hashes on gitter did! I don't know but I suspect that if I had posted on SO then I would also have no useful responses, or they would happen too slowly?

However, responses are lovely and fast both on gitter and (as I have seen with my own two Crystal github issues) on github :-) I'm hoping that this feedback from a real life Crystal newbie is useful for you guys to improve Crystal education to other newbies. It seems to me like gitter and github issues are much less formal -- more like a direct messaging system -- whereas Crystal forums and SO might be more formal, and therefore have less content because there is higher friction to create the content? For example, whereas the Crystal forum and SO have +/- 500 questions each, Crystal github [6] issues are currently 890 open and 4,231 closed... that's 10x bigger! Just saying... :-)

[1] https://forum.crystal-lang.org/c/help-support/11 [2] https://stackoverflow.com/questions/tagged/crystal-lang [3] https://forum.crystal-lang.org/search?q=nested%20hash%20category%3A11 [4] https://forum.crystal-lang.org/t/how-do-i-create-a-nested-hash-type/885 [5] https://forum.crystal-lang.org/t/how-do-i-create-a-nested-hash-type/885/14?u=simonhf [6] https://github.com/crystal-lang/crystal/issues?q=is%3Aissue+

[Blacksmoke16]

For example, whereas the Crystal forum and SO have +/- 500 questions each, Crystal github [6] issues are currently 890 open and 4,231 closed... that's 10x bigger! Just saying... :-)

Issues on Github should be reserved for actionable things, not really the place for questions/help/advice. Possibly could look into using discussions if/when that becomes available. However I'm not sure it would be any better than the forums.

It seems like a larger time investment and making docs is not my best skill :-) However, making notes about docs... that I can do! :-)

Kinda related to https://forum.crystal-lang.org/t/documentation-writing-organization-effort/2115 a central place to find/suggest improvements might be helpful as a means of at least documenting what could be improved. I.e. separate the suggestion side from actually implementing those suggestions.

[jhass]

the Crystal forum and SO have +/- 500 questions each, Crystal github [6] issues are currently 890 open and 4,231 closed... that's 10x bigger! Just saying... :-)

That's just because the GIthub issues existed from day one. The forum is relatively new and the StackOverflow community didn't pick up steam until way into the project.

[simonhf]

Kinda related to https://forum.crystal-lang.org/t/documentation-writing-organization-effort/2115

Thanks for posting that link which I navigated to and contributed!

[simonhf]

Github issues are not documentation, they are proposals and problem descriptions. Their purpose is to get obsolete.

Yep, I agree with you. But... consider source code foo.cr for which people might write github issues with questions. Why shouldn't the same happen for docs for module Foo (which also effectively has a 'source file' for the docs page)? A bit like Wikipedia has a discussion wiki page for each of it's 'official pages', which is not intended to be the page itself, but discusses what is on the page, or what could be on the page, or what shouldn't be on the page. So as a concrete example, I couldn't figure out everything I wanted to know about Crystal nested hashes from the official docs page. I tried in the Crystal forums but that also didn't work. If there was something like the Wikipedia discussion page, but for the Crystal hash table docs page, then I would have written there what I learned, and then either it would eventually get 'implemented' in the official docs (as you say, their purpose is to get obsolete!), or kick around as 'user contributed notes' on the docs, i.e. a reminder why the docs shouldn't be updated with that info...

[asterite]

I tried in the Crystal forums but that also didn't work

What do you mean? Someone replied to you: https://forum.crystal-lang.org/t/how-do-i-create-a-nested-hash-type/885/14

I was going to reply, but I had a hard time understanding the code inside a crystal eval with a bunch of flags. I suggest sending the minimum snippet of code to get your point clear.

[simonhf]

What do you mean? Someone replied to you

That's right, but the reply -- although polite and well intended -- did not offer any useful info for me :-(

And in the spirit of educating newbies like me, I replied [1] to the reply with what I learned in Crystal gitter chat :-)

I was going to reply, but I had a hard time understanding the code inside a crystal eval

Thanks for thinking about replying! It's the thought that counts :-)

Hmmm... but when I asked about nested hashes in the gitter forum -- also using the crystal eval syntax -- there seemed to be more people wanting to understand the code and willing to help.

And IMO crystal eval has a huge advantage over snippets because it's easier to copy and paste entire simple but working examples. Whereas snippets implicitly assume that the user knows how to create and run all the code necessary around the snippet...

This is my biggest newbie issue with the Crystal docs... they are nicely terse like a snippet, but often -- and without associated complete working examples -- it's difficult for newbies to plumb the terse snippet into working code without building up the necessary critical mass of Crystal know-how first. A sort of chicken and egg situation... As a newbie, gitter chat is bridging this gap nicely, but I'm suggesting it might be good to have a more efficient solution, and not necessarily the one I'm suggesting in this issue.

[1] https://forum.crystal-lang.org/t/how-do-i-create-a-nested-hash-type/885/16?u=simonhf

[straight-shoota]

I don't think including user notes into the API docs is a good idea. I have the same experience as @jhass with that. You need to dig through a lot of unhelpful advice, duplicates, tangential topics and conflicting suggestions.

The problem that you @simonhf mention exists though. The current API docs are missing lots of information. And asking on chat or elsewhere helps you individually, but it's not sustainable. Stack overflow can be helpful for some cases, but it's not ideal format for general enhancements to usage instructions and it's a separate tool. Integrating it with the API docs doesn't seem technically feasible (nor would it be substantially useful). But it's definitely a good thing to have the API docs as the main source for information about API features. It should just be more extensive than it currently is. But it's best to manage and moderate that in a single place, i.e. this repository. This also means all documentation is directly attached to the source code. If you distribute the source code, you also ship its documentation.

To improve this situation, we need to find ideas how we can make it easier to contribute to the API docs. Ideally, if a question is asked on chat and someone answers that, it should be encouraged to directly add that to the API docs. Either the one who asked or who answered. Both should be able to do this. And it should be easy to do.

[straight-shoota]

One idea that popped into my mind would be to add an edit link to the rendered API docs. There's already a link to view the source. But that usually points to a commit tree, so it can't be edited directly in GitHub's webui (that only works on branches). So we could have an edit link that points to the source file in master branch. I don't think there's a way to jump to the respective line in the code editor, though.

An improved iteration of that idea would be to provide a custom webapp for editing API docs. It could interface with GitHub API to retrieve the source and at the end publish a PR. This would allow adding nice editing features, like a preview of the rendered API docs. And instructions and guidelines about writing API docs.

[simonhf]

To improve this situation, we need to find ideas how we can make it easier to contribute to the API docs.

Agreed.

Ideally, if a question is asked on chat and someone answers that, it should be encouraged to directly add that to the API docs. Either the one who asked or who answered. Both should be able to do this. And it should be easy to do.

Yep, I would definitely have no problem doing that -- as the asker -- if the process of contributing to the docs is similarly low friction as editing a github issue, or a wiki page. But for example, creating a PR is probably more friction than I'm willing to take on...

Although at the risk of sounding like I'm harping on about the github issue mechanism... it might be more efficient to do it that way if the end result is to eventually have a pristine set of HTML docs with no User Contributed Notes? Let's say 5 contributors make github issue "notes" to the hash table docs pages over a 6 month period... They can be dealt with and consolidated in a single PR batch -- by somebody who knows what they are doing with the docs -- more efficiently than doing 5 PRs as they occur, or? Plus they linger around for 6 months, before disappearing but are still useful for newbies to browse during the 6 months... just like @jhass said, "Their purpose is to get obsolete" :-)

[simonhf]

One idea that popped into my mind

If you come up with anything and want a newbie guinea pig to try it out and give feedback, then feel free to hit me up :-)

[oprypin]

One thing that popped into my mind is... Pull requests are not always appropriate for user contributed examples, even if just because currently Crystal docs don't have a good way to collapse additional details, not to take up space at the cost of other methods on the same page. Sometimes in response to questions I can produce awesome examples that are better than the docs themselves, but it never occurred to me to make a PR to add such an example directly to Crystal source, for not being generic enough. Maybe there's something to this idea after all.

[straight-shoota]

If we wanted something like that, maybe the API docs could have a link to a collection of more detailled examples.

[simonhf]

FYI for want of a better place, I dumped my newbie Crystal knowledge here (which hopefully will be found by Google eventually and benefit other Perl developers interested in Crystal and searching for such comparison info): https://gist.github.com/simonhf/078f0874c622ef9b276fa58554fcd1fc

[girng]

This could easily be implemented. A little button can be shown on the top right of the container of the doc name the user is reading. "Show User-Contributed Notes" or whatever. Which could show some examples that other users submitted to the system (asterite is asking where this is stored), it would have to be in a DB on some server. We'd also have to create a login system for the docs page, and filter out spam, filter out actual good snippets of examples, etc. This will create a vacuum where developers have yet another thing to maintain.

I think the better alternative is to do what blacksmoke said and simply create a PR so it can be added to the docs with your succinct and cohesive code example. Also, just engage with the community on gitter, forums, reddit, or wherever you feel most comfortable. Then maybe once after engaging, you can get in touch with someone and work together on a nice PR for the docs