Raku needs a modern doc format that is familiar to most people

[AlexDaniel]

For documentation, Raku natively supports only POD6 format. POD6 is unique to Raku.

Most developers coming to Raku will not know much about POD6, unless they've been perl5 developers in the past (in which case they are familiar with POD). For everyone else Raku doc format is completely alien. However, most developers are very familiar with Markdown.

In our ecosystem, I was able to find:

• 1543 documents in markdown: https://gist.github.com/Whateverable/ce47b3d73121b6785fd865baa03a7916
• 533 documents in POD/POD6: https://gist.github.com/Whateverable/3dc9fc3e386e8d16109a8c6887d4cac2

That said, most of the markdown documents are READMEs.

There are languages that support markdown flavors natively:

• Julia https://docs.julialang.org/en/v1/manual/documentation/index.html
• Crystal https://crystal-lang.org/reference/conventions/documenting_code.html
• Swift https://nshipster.com/swift-documentation/#documentation-comments-swift-flavored-markdown
• Rust https://doc.rust-lang.org/stable/rust-by-example/meta/doc.html
• … possibly more, feel free to edit this list!

[AlexDaniel]

I propose this solution: Make Raku 7 (or whatever the next release will be) use Raku-flavored Markdown for documentation by default.

[ShimmerFairy]

To summarize my position:

• I'm fine with Raku documentation-viewing tools supporting the installation of documents in other formats.
• I'm fine with people modifying the Pod grammar through slangs to add their preferred formatting syntax to their own Pod files.
• I'm not fine with modifying default Pod to add more and incompatible syntax to Pod, or tossing the format all together.

Lots of languages have their own documentation formats that rarely (if ever) see use beyond them, so the argument that "only Raku uses it!" is not a legitimate argument in my eyes.

The comparison gists are not valid for a few reasons:

1. (minor) The Pod gist doesn't account for new extensions like .rakupod (much like how looking for .pm or .pm6 wouldn't be adequate to find every module file nowadays).
2. The reason for the usage of Markdown, especially for READMEs, is simple: if you want your README to be nicely rendered in github's web interface, you have a limited selection of formats to choose from, and markdown is a popular one of them. (And, it at least used to be that github didn't recognize README.pod6 as a valid README, so it wouldn't display the README under the directory listing at all, compounding matters.)
3. Pod documentation is designed to be embedded in Raku code, so pure-documentation files don't tell the full story of usage.

I think the main issue is that we have never had a fully-fledged Pod parser/renderer implemented anywhere, which severely limits how much people are willing to use it. I think it's disingenuous to say the format has failed or that there's no point in keeping it when we've never really tried to have it.

Also, what would "Raku-flavored markdown" look like? Would it be markdown syntax inside Pod blocks? Would we have a brand new, backwards-compatibility-breaking syntax for jumping into the documentation parser? Or are we just ditching formatting codes?

Most importantly, why? Why change or replace a documentation format that already exists, and has been designed to integrate well with Raku code? What's the justification for changing? It's worth noting that most people don't really "know" Markdown, they just know the very basics of its syntax. And even if developers do really know Markdown, it wouldn't really impede their ability to learn Pod (especially since the blocks and formatting codes are generally more descriptive; it's not hard to guess what I<text> does, even less hard to remember once you learn, but I bet many markdown writers still couldn't tell you what *text* does on-the-spot, no matter how many lines of markdown they've written).

Basically, the main argument for change is that "Pod is a completely alien format" implies that it's too difficult to learn, which is completely unfounded, rendering us without a proper motive for change. I would wager it's a lot more work to learn how to write Raku code in the first place than to learn how to write Pod. Markdown, with its use of various combinations of symbol characters as it's only form of syntax, is actually more arcane than Pod, especially when you consider the wealth of documentation formats that use the same combinations of symbols for slightly different things. Essentially, there are so many "super easy to use" documentation formats like Markdown that none of them are super easy to use in practice, if for no other reason than their similarities in syntax.

In short: I'm seriously struggling to understand how "it looks different" is a valid complaint; should we make Raku look more like C too, then?

Ultimately, I don't see a good reason for rocking the boat on the Pod format. I only see good reasons to improve its support in rakudo and other Raku implementations :) . There's nothing wrong with making people type use Pod::WithMarkdown; to get markdown-esque syntax; this sort of thing is why we let users modify the language in the first place. We never expected people to be happy with all of the default Raku syntax, so if you're unhappy with some part of default Raku, write a slang to change it! We could even make such a module part of the Raku standard, if you like.

And you don't even need to modify the language, by the way! Just creating a custom block that disables formatting codes and has its contents reparsed by the block class itself for Markdown would be sufficient:

DOC use MyMarkdownBlock;

=begin Markdown
# Look ma, no `=head1` !

This is my **markdown** documentation, ain't it neat?
=end Markdown

Point is, Pod is a perfectly fine documentation format, especially for documenting things within Raku. The reasoning is insufficient and there's absolutely no description of what this change would look like, limiting my input to just the general concept of modifying/tossing Pod.

Also, one final note: Markdown is a bad format for long-form technical documentation. In order to support any of the more clever features Pod has in Markdown (which you'd need to make it a suitable replacement), you'd have to do what literally every project on the planet inevitably does to Markdown: add more, nonstandard syntax to it. This would render the "everybody knows it!" argument completely useless, and the process of designing the extra syntax just makes you wonder why we're going through a second design process when Pod already exists from the first go-around.

[nxadm]

For my own private and work code, I ignore pod6 and just document in markdown (and I am one those people that knows pod from Perl). The verboseness of the documentation text blocks makes the code unreadable to me.

I am not saying what I do is common, but I do think many casual users of Raku won't be bothered to learn just-another-doc-format with no benefits for them. If github/gitea/gitlab can show their documentation correctly, then it's good enough to use what you already know.

[AlexDaniel]

For my own private and work code, I ignore pod6 and just document in markdown (and I am one those people that knows pod from Perl). The verboseness of the documentation text blocks makes the code unreadable to me.

If personal experience matters, I've been using all kinds of stuff throughout the years. When I was working on Oddmuse, there was this thing called Creole. At the time I couldn't understand how something so flawed (Markdown) can become mainstream. Over the years, however, I got really tired of switching between markup languages that do exactly the same:

• Writing something on my own oddmuse wikis? Creole.
• Editing a Wikipedia page? Some kind of mediawiki format.
• Writing a local file for myself? Emacs org mode.
• Writing my thesis? LaTeX.
• Writing some C code? Doxygen.
• Helping with raku/doc repo? POD6.
• Everything else? Markdown

Yes, it is easy to learn each of these formats, but I just can't take it anymore. More so when I have to switch between them in a single session. I don't care if something already exists in Raku, I no longer care that it might be superior. I'd be much happier if I used less markup languages, not more, even if something else is better.

So, for me, everything is in markdown now, unless I'm contributing to an existing document. I could maybe make an exception for something that is widespread (ReST, or maybe Asciidoc), but I'm not willing to waste my nerves on anything specific to a single language.

@ShimmerFairy You're making good points, but this is not about having great debating skills, it doesn't change the situation. Raku would've been more enjoyable, useful and attractive if it didn't use a doc format that nobody else is using.

[ShimmerFairy]

You're making good points, but this is not about having great debating skills, it doesn't change the situation. Raku would've been more enjoyable, useful and attractive if it didn't use a doc format that nobody else is using.

Have you tested that assertion? Do you know that a significant portion of the coding world has rejected Raku specifically because they didn't like the documentation format? So far, all I've seen is two people that don't like Pod, but that does not imply a larger resentment of the format, much less a resentment large enough to require significant change.

And please don't blithely dismiss my arguments as just a show of great debate skills. Either you have the counterpoints and explanations to rebut my points and answer my questions, or you don't. Suggesting that I just wanted to show off my ability to write persuasive essays is not only not a defense of your position, but also very belittling and rude.

[nxadm]

So far, all I've seen is two people that don't like Pod, but that does not imply a larger resentment of the format, much less a resentment large enough to require significant change.

Yes, everything in this tread is anecdotal and subjective, including your position. And that's OK. Some people think pod6 suffices, some people don't. And we have no idea which group is bigger, besides our experiences with our surrounding.

[jnthn]

I think the main issue is that we have never had a fully-fledged Pod parser/renderer implemented anywhere, which severely limits how much people are willing to use it

At least for me, having every last speculated feature of it implemented is not really the issue; 95% of the time all I want is code, bold, and italic anyway. The much bigger issue is that I don't really get much in return for using it.

When did I start writing declarator comments reliably, at least on types, methods, and exported subs? Basically, the week after support landed in Comma for displaying them (including for modules, by introspecting the .WHY). For the first time, I could see a point in using them because I actually got something useful in return - or in my modules, it felt like at least some language users would get something useful from them. Prior to that, they were a nice idea, something I knew I maybe should write, but I never got any benefit.

Ideally (yes, I know, "well volunteered"...), modules.raku.org would be able to take the declarator block comments, along with the class names, routine names and signatures they are applied to, and produce a javadoc-like browsable documentation. And if I was getting that, and all I had to do to include some nice usage examples and details of the module overall was to write a bit of Pod inside the module, I'd surely do it. But at the moment, there tooling isn't there to make it feel valuable, so anything beyond the declarator comments just ends up in a README.md.

[ShimmerFairy]

I think @jnthn 's comment perfectly illustrates the real issue, which is that there isn't enough incentive to use Pod right now (of which "there's no full parser" is part of the issue; the trickier features are more likely to be intriguing to newcomers to the format, but less likely to actually work). The tools just aren't mature or existent enough to make people want to use it. After all, if you couldn't render Doxygen out to a nice website or PDF manual, nobody would use that either.

Pod has been critically underloved for pretty much the entirety of Raku's history, so it's no surprise that it doesn't get used much. Blaming the syntax for its unpopularity is focusing on the wrong thing entirely. And once we improve Pod support enough, people who want to write documentation in alternate syntax can create their own custom blocks and formatting codes for them, which would solve this ticket without polluting the Pod format with duplicate and relatively weird ways of doing what Pod can already do. And I cannot stress enough how much Raku is designed to have its language modified by its own code, so if you really want Pod to parse differently, you can do that without forcing the change on everyone else.

If you want Pod used more, adding the missing tooling and parser support is the way to go (something which I hope to start working on soon, for what it's worth). Changing the standard to suit your preferred document syntax won't magically make the tooling (and thus incentive) appear.

(And if you're about to argue that Markdown syntax already has tools so a switch would suddenly give you access to all the tooling you need, that's irrelevant to this scenario. Adding Markdown to Pod (whatever that's supposed to look like) just means that Raku implementations now need to add Markdown parsing to their Pod grammar; the result of its parsing will still be the same old under-spec'd and under-supported tree of Pod classes as ever.)

[alabamenhu]

There really shouldn't be a reason that we can't enable a different comment language. ShimmerFairy's suggestion to use

=begin markdown
# Header
Text belonging to the header
=end markdown 

Could be written as a documentation slang, only requiring use Documentation::Markdown somewhere. One wouldn't necessarily even be limited to markdown, since virtually all of the documentation formats map cleanly onto POD6's (at least, I can't think of any feature any of them have that POD isn't supposed to support, although they might not support all of POD's).

This is probably Yet Another Reason™ to work out the kinks in slangs, which, I feel like is now almost on the horizon, pending completion of RakuAST.

[AlexDaniel]

There really shouldn't be a reason that we can't enable a different comment language. […] Could be written as a documentation slang, only requiring use Documentation::Markdown somewhere. One wouldn't necessarily even be limited to markdown, since virtually all of the documentation formats map cleanly onto POD6's

Well, there actually is, and this is what people are talking about in this thread. It's extremely unlikely that tools will support slangs. Maybe some raku-specific ones like Comma, but everything else won't. So, yes, you would be able to add a use statement and write your stuff in markdown, but it'd be as practical as doing the same without a use statement.

Looking back at some arguments in this thread:

Lots of languages have their own documentation formats that rarely (if ever) see use beyond them, so the argument that "only Raku uses it!" is not a legitimate argument in my eyes.

Well, that's exactly the point. We shouldn't be pushing a new doc language knowing that it's unlikely it'll see use beyond raku, instead let's base our language on something that already exists (and is not perl's POD, come on).

I think @jnthn 's comment perfectly illustrates the real issue, which is that there isn't enough incentive to use Pod right now

And that's exactly the point too. Most people wouldn't use rakudoc unless it's the only thing our tooling supports (i.e. unless we force them). If you need any evidence for that you can take a look yourself to see that even core devs are pushing documents in all kinds of formats (instead of using POD5 as the closest relative).

[JJ]

And that's exactly the point too. Most people wouldn't use rakudoc unless it's the only thing our tooling supports (i.e. unless we force them). If you need any evidence for that you can take a look yourself to see that even core devs are pushing documents in all kinds of formats https://github.com/MoarVM/MoarVM/blob/master/docs/contributing.org (instead of using POD5 as the closest relative).

That's not evidence for anything other than people will use whatever they want for document files for a project that's not even written in Raku. They will probably use LaTeX for their papers too.

[AlexDaniel]

That's not evidence for anything other than people will use whatever they want for document files for a project that's not even written in Raku. They will probably use LaTeX for their papers too.

So are you arguing for or against having our own different format? You're basically saying that yes, people would much rather use something else.

[JJ]

Gee, caught me red-handed, I write papers using LaTeX, and books using gasp word processors. Guilty as charged. So I guess I only have to agree with you that Raku should include a word processor or what was it precisely?

El dom., 9 ago. 2020 18:21, Aleks-Daniel Jakimenko-Aleksejev < notifications@github.com> escribió:

That's not evidence for anything other than people will use whatever they want for document files for a project that's not even written in Raku. They will probably use LaTeX for their papers too.

So are you arguing for or against having our own different format? You're basically saying that yes, people would much rather use something else.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Raku/problem-solving/issues/207#issuecomment-671071707, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAD5HVQO26GBH67Y5RVCLR73EKBANCNFSM4OKRYPXA .

[AlexDaniel]

So I guess I only have to agree with you that Raku should include a word processor or what was it precisely?

That Raku needs a modern doc format that is familiar to most people.