building sugar-docs without sugar-build?

[ghost]

14:22 < Pro-Panda> Hi walterbender, I want to run the developer docs website locally. I read this https://developer.sugarlabs.org/docs.md.html, but I can't figugre how to use the sugar-build and run 'docs'

• We have several ways of setting up development environments that do not use sugar-build in any way and
• We have removed mention of the sugar-build approach to setting up a development environment

So, some documentation about how to run local tests of changes to the documents might be prudent.

There are many tools for converting Markdown to HTML, but subtle difficulties can arise due to differences amongst Markdown dialects, amongst toolsets, and from inconsistencies in how paths are handled from one context to another.

How is the Markdown in this the https://github.com/sugarlabs/sugar-docs repo converted into the HTML displayed at http://developer.sugarlabs.org ?

Is https://pypi.python.org/pypi/Markdown used? Are there known and documented differences in how it handles things compared to https://github.com/github/markup to which we can make reference?

Can we describe some variant of that process here in our documentation so that people can review their changes before submitting them?

If we are using sugar-build to generate the HTML served at developer.sugarlabs.org then we might either want to take a step back and keep some mention of sugar-build in these documents, or consider whether we can sustain the way that Markdown-to-HTML transformation is being effected in production and whether that needs a change.

It might be sufficient to suggest a developer push to their own github fork and rely on Github to render that, pursuing differences in the way Markdown is rendered between the two environments on a case-by-case basis as bugs. If so, we should edit https://github.com/sugarlabs/sugar-docs/docs.md to reflect that.

Sorry, this is documentation about development of documentation so if I'm confused or am confusing anyone else, my apologies.

[ghost]

Sorry, I get frustrated sometimes when mention is made of components without an explicit link to their code or repository or whatever is appropriate, so here's https://github.com/sugarlabs/sugar-build

I think that covers everything mentioned above that can be specified that way.

[quozl]

sugar-docs deployment is triggered by a package maintained in Sugar Labs infrastructure organisation on GitHub.

https://github.com/sugarlabs-infra/developer-rebuilder

Looks like it uses pygments, not sugar-build? Not sure.

systems@ mailing list may be able to help.

@codewiz, @samdroid-apps.

[ghost]

On Wed, Jan 03, 2018 at 11:47:58AM -0800, James Cameron wrote:

sugar-docs deployment is triggered by a package maintained in Sugar Labs infrastructure organisation on GitHub.

https://github.com/sugarlabs-infra/developer-rebuilder

OK, thanks, that was what I needed to come out the other end of this rabbit hole.

Just to map my path, roughly, lest I lose track of it once I fall asleep, as per

https://github.com/sugarlabs/sugar-docs/blob/master/docs.md

one was meant to run 'docs' from within sugar-build.

I'm guessing that meant running this:

https://github.com/sugarlabs/sugar-build/blob/master/build/commands/broot/docs

which in turn calls this

https://github.com/sugarlabs/osbuild/blob/master/osbuild/docs.py

which in turn assembles and runs a command using the now quite unfortunately named program called "docker" which is not the container application runtime of that same name, but instead a Node.js application for rendering Markdown into HTML

https://github.com/sugarlabs/docker/blob/master/src/docker.js

(not touched in almost 5 years)

from the upstream

https://github.com/jbt/docker

(not touched for about 2 years)

I say the other side of the rabbit hole because sugarlabs-infra makes reference to both kinds of Docker, setting up to run the Node.js program of that name here

https://github.com/sugarlabs-infra/developer-rebuilder/blob/master/main.py

via a container

https://github.com/sugarlabs-infra/developer-rebuilder/blob/master/container.yml

into which is installed a Node.js environment specified by a Dockerfile in the other, container sense of Docker:

https://github.com/sugarlabs-infra/developer-rebuilder/blob/master/Dockerfile

So, bottom line is, the Node.js program 'docker' is what seems to be used currently in production for turning Markdown into HTML for eventual display at developer.sugarlabs.org and also was what sugar-build, eventually, provided for locally testing one's own Markdown edits.

I haven't yet tried getting a node environment running to test docker.js rendering of Markdown to see if I can confirm or deny that it handles newlines in Markdown links poorly or not.

I'm also not sure I can wrap my head around documenting its use for testers, but so long as that is what is used in production that seems like what should be done.

Comments?

[quozl]

Well done! I did not know the word docker had been overloaded; every time someone mentioned it I had presumed they were talking about the container application runtime. Makes a lot more sense now. doc-script.js on the deployed site has reference to the other docker.

And then for sugar-build to be overloaded as also a platform for testing changes to sugar-docs, that's so funny or sad; not sure which yet.

sugar-build is broken at the moment and has no maintainer. The Sugar Labs CI Fedora broot infrastructure has been off since around 2014. We had a sudden disappearance of a key developer. That's why it has been removed (mostly) from sugar-docs.

Let's not lose this learning; we should;

• [ ] look into merging later docker.js changes or switch to another converter,
• [ ] change sugar-docs to not suggest sugar-build for testing changes, and describe what dependencies should be installed by a developer,
• [ ] change sugar-docs to describe briefly how it is deployed into production.

[samdroid-apps]

Great tracking that down @ritjoe; I meant to send a reply but it appears that github didn't get it.

That repo unnecessarily confusing due to the name conflict with docker.

You should be able to run the docker (docs) script by:

    git clone whatever the repo was
    cd docker
    npm install
    ./docker

You can use the command line from here https://github.com/sugarlabs-infra/developer-rebuilder/blob/master/main.py#L19-L20 (when your current working directory is the sugar-docs repo, and you change /docker/docker to the correct path)

I'd also like to add that the https://github.com/sugarlabs-infra/developer-rebuilder is NOT currently working on the sugarlabs server. The whole Kafka queue was a mistake and no longer runs. Happy to look for a better CI solution or give @ritjoe or @quozl access to the server and you can just rsync the build output there yourself.

[samdroid-apps]

FYI @quozl; the docs are stored on freedom.sugarlabs.org in /home/sam/sugar3-doc/sugar3/. And the config is /etc/nginx/sites-enabled/developer.sugarlabs.org. Feel free to overwrite that folder or change the config to a different one.

[quozl]

@samdroid-apps, thanks, but are you sure about it not running? When we make commits the site changes. I agree that /sugar3 doesn't update, but that's a different thing, yeah? I've notes on how to update /sugar3 by hand.

[samdroid-apps]

@quozl - thanks I haven't kept up with where this new site is. For the record; the developer reloader container is active. The code for that container is quite different on freedom than on the git repo; it provides a webhook server rather than that Kafka queue.

[quozl]

@samdroid-apps, thanks. Noted it at https://github.com/sugarlabs-infra/developer-rebuilder/issues/2 but I've no write access to the repository. I've access to freedom; where is it on freedom?

[quozl]

WIth the move away from sugarlabs.org hosting, building of sugar-docs is no longer on any critical path, so we can treat this issue as completed.