Feature: Edit/add pages via Discourse

[erlend-sh]

It would be far easier for the average user to contribute if they didn't have to leave the Discourse workflow to create & edit Docs articles. What if a Discourse-to-GitHub (i.e. to our learndiscourse repo) integration made it possible for Discourse users to contribute to Docs without ever leaving the forum?

From what I can tell, basically you'd need an app that's got API keys for both meta.discourse.org as well as GitHub. This app would:

1. Use the Discourse API to listen for changes (new topics or edits) in the How-to category
2. Once a change is made, it grabs raw contents of that post and...
3. Use the GitHub Git Data API to make a commit to the appropriate file on a dedicated repository.

Is that about it or did I miss anything critical?

[trident523]

(x-post https://meta.discourse.org/t/a-community-effort-to-improve-discourses-documentation/29129/56) From my memory of ~6 months ago watching a topic via the API was not possible without a bot looking for changes on the post (either on a timed schedule, or via a sync button.) Maybe this has changed.

Otherwise, the flow described seems sensible. Still, site categories will never quite align with the documentation page categories. There would have to be some mapping of topics to pages. I see that many of the howtos have been organized already.

One thing I've enjoyed about the discourse API is that a key isn't really needed for public facing data; we can already scrape info like

https://meta.discourse.org/c/howto.json
https://meta.discourse.org/t/running-other-websites-on-the-same-machine-as-discourse/17247.json
(to non-developer friends, it may be handy to use a JSONviewer; either as an extension for your browser or online elsewhere)

by appending .json onto the URL as long as it can be viewed by all. A github API or deploy hook key would still be needed; but any discourse install could be a source of information.

I think how I had envisioned it before was that the documentation page would load the data from discourse; and a mapping of topics <-> pages would be maintained by documentation owners. However, with the direction of work in here it may make more sense to attempt this synchronization so that we can add more doc-specific elements as needed instead of being tied to what discourse can generate.

Currently, I've signed away most of my free time and the earliest I could actually do any development work would be august; as my work commitments prohibit me from OSS contributions except for my free time. Happy to help when I can otherwise though.

[erlend-sh]

Another limitation: Interlinking

Jekyll can have an article named How Badges Work (which improves SEO and discoverability in general) with a simple URL like /badges. Discourse on the other hand doesn't allow editing URL slugs independently of their title.

So to keep internal links from breaking, we'd have to choose one or the other:

• Simple URLs,
• or highly descriptive titles.

Personally I'd opt for simple URLs. Most searches in Discourse are full-text (I forget which one is title-only; is it when looking for similar topics?), so the topics should be discoverable anyhow. Meanwhile, on Jekyll's end we have clean, short URL's from the onset and can give articles more descriptive titles in frontmatter.

[ghost]

What do you mean by "highly descriptive titles"?

Is it something like how-badges-work or is it something like a-doc-on-how-badges-work-and-how-you-can-use-them? If its the first then I would say we should use that.

[erlend-sh]

Well, a better example would be http://learndiscourse.org/advanced-manual-method-of-manually-creating-and-restoring-discourse-backups/

I'd definitely prefer to shorten that down to something like manually-restoring-backups in the URL

[ghost]

Lol, I dont think that title is appropriate anyways. It says manual twice. manually-restoring-backups should be the title.

[erlend-sh]

This has been partially implemented for a long time now with the importer script. But I'm curious @fantasticfears , what's your idea of a completely automated version of this? (this is not part of the scope for our planned sprint this December btw. I'm just curious).

I was thinking this might also be doable with a Discourse Plugin. Give it a GitHub secret token and point it to a GitHub repository, and it would push a new, readily formatted foopage.md to the appropriate folder whenever it detects a change.

Yet another way becomes possible if we stop relying on GitHub pages. Then we could use a plugin like jekyll-get - "Jekyll plugin to download data from external JSON sources to use in generating a site". Not quite sure how the post-processing (rewriting internal links) should be done though.

[erlend-sh]

For all intents and purposes, this is as done now as it was a month ago :smile:

New issues should be created to discuss more advanced forms of automation (first next step being detection of newly added topics in the Howto category).

[erickguan]

I am thinking about the difficulties around the automation.

Here at v2, there are still some issues not solved when we made the doc site at first time.

• It's not official.
• i18n translation is not implemented.

I would ask some question before we move on the automation:

• What should be in the site? Docs. Plugins? Themes?
• Where we want the contributors to edit the doc? Meta or here. Erlend said the workflow will remains in the Meta which is cool. A single redirection will work.
• But how to manage the documentation in different language? Where we want the user to manage the data?
• We based the learndiscourse workflow on doc_list.yml which contains the information from Meta. Automation means we have to give up this idea. Doc changes URL, category and content at any time in Meta. Is it possible to leverage the power from Discourse API or plugin?
• As more automation kicks in the way. It means we have to generate the site and push the data to Github Pages which enable us the power of more plugins. Go with a better toolset?

[erickguan]

Here is another follow-up. I have a more clear picture for editing and collecting different documents, plugins and even parise right now.

Firstly, docs and plugins are sorted in the Meta right now. It's easier to get different categories of things for a bot. Secondly, no matter what it is a doc or plugin. It's a good idea to store meta information in the topic. With this meta information, we may display a helpful infobox for a doc or plugin. Thirdly, we can expose an API for everyone to query. Imagine we can have a search box in the admin panel to search plugins? Last, a self-hosted machine can download, generate and push to the right site based on a local Jekyll.

By doing so we need:

• A plugin in Meta: Hard
• Every plugin and Doc needs some more information to fill.
  • Doc:
  • Mandatory:
    • Category (otherwise it will yield a page where no one connects it)
  • Optional
    • Title
    • Slug
    • Weight
    • Other languages markdown source
  • Plugin (Optional):
  • Mandatory:
    • Homepage (GitHub)
    • Downloading page
    • Version
  • Optional
    • Title
    • Slug
    • Weight
    • Other languages markdown source
• A server can query at a regular basis.
• Additional text from GitHub still needs to be sorted manually anyhow.

[erlend-sh]

Interestingly, someone has already done something very similar to what we're trying to do:

https://discourse.osmc.tv/t/contribute-to-the-wiki/7929 https://osmc.tv/wiki/

(Open one of those pages and click "Make it better")

The biggest hurdle for Learndiscourse is canonical URLs. In the example above, the elected to make the wiki the canonical resource. For Meta, this is a tougher choice to make. Let's have a chat about it soon.

[erickguan]

OSMC well done! How they make this? They can hack Discourse to open a modal guiding user which can be a future work.

The biggest hurdle for Learndiscourse is canonical URLs. In the example above, the elected to make the wiki the canonical resource. For Meta, this is a tougher choice to make. Let's have a chat about it soon.

That's insight. We still can make use of old process script.