Documentation makeover | migrate developer README to the manual?

[aadcg]

I propose a complete re-organisation of Nyxt's documentation.

One key aspect of any documentation system is the format. Firstly, I think it makes little sense to put effort into printed copies. A more interesting question is whether we care about PDFs. I would say we don't, but even if do need, it might not be a priority.

For obvious reasons, HTML is the format we use. It allows us to embed the documentation, and to publish (its static counterpart) externally (e.g. Nyxt's website).

But that doesn't mean that we write the docs directly in that format, since we generate the HTML with the spinneret library.

I suggest the following:

~~- [ ] Convert all org docs under documents/ to .lisp files (akin to manual.lisp). The only relevant piece is documents/README.org, which is the developer's manual. Org shall only be used as our preferred markdown format for READMEs and the like.~~ ~~- [ ] The developer's manual is a spaghetti dump that defeats the purpose of being useful. Complete re-write.~~ ~~- [ ] changelog.lisp and tutorial.lisp are located at source/. Shouldn't they be at documents/?~~ ~~- [ ] Documentation is scattered for no good reason. I suggest aggregating it (manual, tutorial, dev manual, etc). These can still exist as "independent" entities since they can be referenced with anchors.~~ - [ ] Add syntax highlighting to the help buffers by defining a css style. - [ ] Review the generation of static documentation at compile time.

• [x] Develop an info-like interface as already suggested in #623.

[Ambrevar]

• Why convert documents/README.org to Lisp? I don't believe it to be useful from a running Nyxt instance. If there are some bits that could be, like the Slynk snippet, then I suggest we move it to the manual instead.

• The developer's manual is a spaghetti dump that defeats the purpose of being useful. Complete re-write.

This is a bit harsh... We've spent lots of time working on it and it's a decent state in my opinion. Of course, improvements are always welcome. I don't believe it qualifies for a complete rewrite, which would be a waste of time in my opinion, with little guarantee that we end up with something better.

• changelog.lisp and tutorial.lisp are located at source/. Shouldn't they be at documents/?

  documents/ has only Org files that are not processed nor used in the final product. The changelog and the tutorial are manipulated in various ways (e.g. for the web site).

There is also a semantic difference. The changelog and the manual are of interest to the end user, while the documents/ file is only of interest to the developer.

• Documentation is scattered for no good reason.

Well, there are some good reasons actually, see my other points.

• Add syntax highlighting to the help buffers by defining a css style.

  It's already there, but of course it begs for improvements.

• Review the generation of static documentation at compile time.

  What exactly?

There are multiple ways we can improve our documentation system, I suggest you consult https://github.com/atlas-engineer/nyxt/issues?q=is%3Aissue+is%3Aopen+label%3Adoc

The main step forward, I believe, is to develop an Info-like interface. From there, everything will be automatically improved since all the documentation will be accessible from a single entry point.

[aadcg]

Let me tackle what I see as the main point. You say that it makes little sense to have the developer docs available from a running Nyxt instance. More generally, you see a division between users and developers when it comes to docs. I say that a manual should be a complete document used as a reference by any user or contributor. For example, creating a Nyxt extension is a topic for a manual (as of today it's an article AFAIK). Also, it must be available in 2 ways: from Nyxt itself, and as a (static) HTML document that can hosted somewhere else (Nyxt's website). Currently, we have articles, an unpolished looking manual hosted on Nyxt's website and an org document that covers hacking Nyxt. If we'd go with my approach, we wouldn't have to sync all of this information coming from multiple sources. We build a single static and complete manual that is published at Nyxt's website.

This is a bit harsh... We've spent lots of time working on it and it's a decent state in my opinion.

Sorry, it wasn't intended to offend anyone. Actually, my impression is that everyone added their bits but it was never really consolidated as a whole. It feels like a flowsheet to me. Some bits are repeated and, while I understand why, we can do better.

Anyway, let's focus on what we agree: the Info-like interface.

[Ambrevar]

I agree that parts of the developer's manual and of the articles (such as developing the extensions) belong to the manual. The main reason why it was not included yet is that we lack a good manual interface, so if we add all this, we would bloat the manual and no one would ever read it. We must fix the interface first.

[aadcg]

Fair enough. I'm on it.

[aadcg]

It seemed to me that the most logical way to approach this would be to use CSS and to dynamically hide or show certain parts of the HTML document. I'm not very comfortable with CSS, but it seems that it's not so trivial.

Another way would be to have several independent linked HTML documents, but I find that undesirable.

It seems to me that we already have lots of commands that come close to an info-like interface (namely next/previous-heading or jump-to-heading). What's missing are tables of contents (global and per-section), and some style adjustments.

The question then is: how important is it to have the ability to hide/show sections, and does it really contribute to a cleaner manual? Personally, I don't think that those things go hand in hand.

@Ambrevar @aartaka @jmercouris @pdelfino

[aartaka]

It seemed to me that the most logical way to approach this would be to use CSS and to dynamically hide or show certain parts of the HTML document. I'm not very comfortable with CSS, but it seems that it's not so trivial.

There's also an infinitely cool <details> HTML tag :)

Another way would be to have several independent linked HTML documents, but I find that undesirable.

Indeed, having just one document/page is better. And it's easier to add pagination to a monolythic document than to condence a bunch of pages into a single page too :)

It seems to me that we already have lots of commands that come close to an info-like interface (namely next/previous-heading or jump-to-heading). What's missing are tables of contents (global and per-section), and some style adjustments.

Indeed, by adding good navigation we improve navigation on Nyxt-specific pages too :D

The question then is: how important is it to have the ability to hide/show sections, and does it really contribute to a cleaner manual? Personally, I don't think that those things go hand in hand.

I don't believe that it's necessary either. For quick visual glance over the text we have jump-to-heading and the like. However, these commands are almost invisible to the newcomer. We can duplicate those with a TOC (table of contents) in the beginning on the manual, or we can add a command for TOC generation. Give me an hour and I'll post a prototype of a TOC panel command here :)

[aartaka]

Here is this prototype. It's neither pretty nor semantically correct, but it's a good starting point. Maybe we can automatically open TOC panel for manual?

#+nyxt-3
(nyxt::define-panel-global toc (&key (buffer (id (current-buffer))))
    (panel "*Table of Contents*" :right)
  "Show a table of contents of the currently open buffer."
  (let* ((buffer (nyxt::buffers-get buffer))
         (headings (nyxt/web-mode::get-headings :buffer buffer)))
    (labels ((get-level (heading)
               (parse-integer (subseq (plump:tag-name (nyxt/web-mode::element heading)) 1))))
      (spinneret:with-html-string
        (dolist (heading headings)
          (:button :class "button"
                   :onclick (ps:ps (nyxt/ps:lisp-eval
                                    `(nyxt:with-current-buffer (nyxt::buffers-get ,(id buffer))
                                       (nyxt/web-mode::scroll-page-to-heading
                                        (elt (nyxt/web-mode::get-headings) ,(position heading headings))))))
                   (make-string (get-level heading) :initial-element #\*)
                   " " (nyxt/web-mode::inner-text heading)))))))

[aartaka]

Oh wait, there's headings already xD

[aartaka]

I've just rewritten headings to be more structured and TOC-like :)

[aadcg]

Thank you for the input @aartaka. I believe I can take it from here. I will have another look at this tomorrow.

[aartaka]

@aadcg, anything left for this issue?

[aadcg]

Whether we should migrate the developer's documentation to the manual is still an open question, I believe.

[Ambrevar]

Yup, we should implement an Info-like pager, then we can move everything over.

[aadcg]

@Ambrevar, I've discussed this with @aartaka and it seemed to us that, while being possible, it wouldn't be so important. We actually have the paging facility, it just that we don't hide/collapse all sections expect the current one.

[Ambrevar]

We actually have the paging facility, it just that we don't hide/collapse all sections expect the current one.

Can you rephrase? Do you mean that all we need to do is automatically collapse / expand sections as we navigate the page?

[aadcg]

Open the manual from Nyxt. We have the keybindings below available. In particular, next-heading and previous-heading are very similar to what the Info interface provides. The only difference, as I've mentioned, is that all sections meaning visible while navigating them.

(define-keyscheme-map "help-mode" ()
      keyscheme:default
      (list
       "q" 'delete-current-buffer
       "n" 'nyxt/document-mode:next-heading
       "p" 'nyxt/document-mode:previous-heading
       "m" 'nyxt/document-mode:jump-to-heading
       "s" 'nyxt/search-buffer-mode:search-buffer
       "t" 'nyxt/document-mode:headings-panel
       "?" 'describe-bindings))

[Ambrevar]

Understood, but "showing all the manual" is a big no-no for newcomers. The point of paging is to make it less intimidating. In the current state, I'm afraid practically no one is going to read the manual simply because it stands on a single page.

[aartaka]

75bddd3cfa1b712a6aec125d5688cf0adffaf187 and bdddf2f42173bd5b16e47511fdd739986eb1ef26 should cut it by making all the <nsection> s to also be <details>, and contracting some of those to mere headings.

[aadcg]

@aartaka I actually oppose those changes.

I seem to disagree with @Ambrevar when he says that "showing all the manual is a big no-no". To me, his arguments feels like "no one is going to ever read a whole book because all the chapters stand on a single bookbinding".

I understand that @Ambrevar likes the UI of Info, in the sense that it's possible to hide all but the current section of a manual. I find that not so important, since what I really like about Info is how easy it is to navigation sections. Navigation-wise, we're on par with Info!

Still, I don't oppose @Ambrevar's idea if it's implemented properly.

<details> and the idea of sections can be hidden by default doesn't seem like a proper implementation. Visually, it looks off since there's no way to understand the hierarchy of the document. Compare below the TOC with the manual where every heading can be collapsed.

Also, I don't see the point in setting whether a section should be collapsed by default.

[aartaka]

Yes, collapsed-by-default sections were quite annoying to me too :( I guess this part (auto-collapsing) we should change.

[aadcg]

Commit bdddf2f42173bd5b16e47511fdd739986eb1ef26 introduced the collapsed sections by default, which break commands such as nyxt/document-mode:next-heading.

Even if we can have collapsed sections by default and the commands working, I'd really lean towards reverting this change.

Could you please do it @aartaka?

Also, changes like those introduced in the aforementioned mentioned commit would be better suited for a PR than merging straight to master.

[aartaka]

@aadcg, reverted in 654e1637fc9eed53c00cd17a48cdce67d16dbce3. I would've done it as a PR if at that time it seemed as bad as it turned out to be :D

[aadcg]

@aartaka thanks. To fully revert this change, I think 75bddd3cf also needs to be revised since it makes no sense to have the :open-p keyword.

[aartaka]

@aadcg, no :open-p I won't remove, as it is a valid use-case to have some sections (like method listing in describe-function) collapsed or un-collapsed by default.

[aadcg]

I think we're not on the same page. After the commit you have reverted, no manual section is collapsed. The moment you add any :open-p nil, then you won't be able to navigate the manual anymore (give it a try).

I'm not saying that the idea of collapsible sections isn't worth exploring, but the implementation is certainly not there yet.

[aartaka]

Manual is not collapsible, true. But there are other use-cases where search or navigation are not as critical, as de-cluttering the view, like generic functions' describe-function. The implementation is fine for those cases. Searching or jumping inside the collapsed <details> is a problem of search/jumping, not the problem of <details>.

[aadcg]

Do you mean that :nsection could be used outside manual or tutorial?

[aartaka]

@aadcg, yes. As all the spinneret-tags we define.

[aadcg]

Ok then. It's just that :nsection was created with manual and tutorial in mind (as far as I can tell).

[aartaka]

Ok then. It's just that :nsection was created with manual and tutorial in mind (as far as I can tell).

Not exactly. I try to make every tag as reusable as it's possible, given that we don't yet have a UI Kit (#946) or our own GUI protocol. Spinneret tags are the closest thing we have to a reusable UI library :)

And the mix of <section> with <h*> is actually a semantic analogue of a <div> in a bad HTML—there are many applications for this structural pattern of block-with-header.

[aadcg]

All clear @aartaka. I now see the rationale, and it all makes sense!