Transfer the N-API tutorials into the OpenJS Foundation

jschlight commented 4 years ago

Background

Since February 2018 I've maintained a website called The N-API Resource. The objective has been to have a central resource that contains guides that go beyond the basic core documentation. We've used this website to support the three N-API workshops we've presented so far.

When preparing the N-API workshop for the 2019 Node+JS Interactive conference in Montreal, it became clear that the complexity of the topics the website needs to address now exceed my capacity. Thankfully, other members of the N-API team stepped up to prepare content for the site.

As we collaborated on the tutorials, the weaknesses of the existing publishing model become apparent. The review process is clumsy and non-standard and a single person (me) has become a bottleneck.

Based on this experience, I'd like to see if we can move the content of the existing website under the auspices of the OpenJS Foundation.

I'm happy to take the lead on this project if we decide to move forward.

Basic Assumptions

I'm assuming that we want to maintain Markdown and supporting documents in a repository and then publish these Markdown documents as a navigable and searchable website. The "website" might be a single-page browser application that presents content directly from the Markdown and supporting documents on the repository. Or it could be a traditional static website.

I also assume I'll take responsibility for transferring the existing content to the new solution.

Moving forward, I'd like to see the content take a more "problem solving" approach. Instead of trying to describe a particular technology, I'd like to see the site explain how the technology solves a particular problem the developer is trying to solve.

If there are other approaches or concerns, we should discuss them.

Objectives

An OpenJS-hosted N-API tutorial site that goes beyond the core documentation to explain how to solve various programming conundrums using N-API and node-addon-api.
A process that invites and encourages additional contributors.
A system that can easily grow with new content.
Documents created and maintained in Markdown.
Documents hosted on an OpenJS repository. It could be an existing repository.
Support for images and other media, as required.
Easy to include source code examples by referring to the URL where the source code resides.
Updates made through the regular GitHub pull request and review process.
A simple, reliable, low-maintenance, and automated publishing process.
The resulting site should:
- Be easy to navigate
- Include a searchable index
- Be attractive and inviting
- Secured by https
- Include a way for readers to provide feedback

Plan

[ ] Discuss and document the objectives of the project on this issue.
[ ] Transfer all examples into the nodejs/node-addon-examples repository.
[ ] Implement automated testing for all examples.
[ ] Determine if coordination with the wider Node.js community is helpful.
[ ] Research and catalog possible publishing platforms on this issue.
[ ] Discuss and decide on a publishing platform.
[ ] Determine where the site will be hosted.
[ ] Create a migration plan once the publishing platform and hosting is determined.

Alternative Publishing Platforms

hexojs/hexo

This is the Node.js tool I've used to publish the existing website. It takes Markdown and YML documents in specific directories and generates a static website in another specific directory that can be hosted just about anywhere. The current website is hosted on Amazon S3.

One of the nice features of this tool is that it populates the source code boxes in the documentation with the current source code when the static site is generated.

The hexo tool also includes a Node.js-based development server that permits you to view documentation updates in real-time on a website served by the development server.

Hexo website.

docsifyjs/docsify

This tool creates a Single Page Application (SPA) that retrieves markdown files directly from a Github repository.

docsify website.

docpress/docpress

This tool creates a Single Page Application (SPA) that retrieves markdown files directly from a Github repository.

docpress website.

egoist/docute

This tool creates a Single Page Application (SPA) that retrieves markdown files directly from a Github repository.

Docute website.

mrvautin/githubdocs

This tool creates a Single Page Application (SPA) that retrieves markdown files directly from a Github repository. It also has an option to generate a static website.

GitBook

Started as an open source project, GitBook is now a commercial product which is free for open source projects. It generates and hosts static websites.

Node.js

Does it make sense to look at the publishing approach used for the core Node.js documentation?

jschlight commented 4 years ago

Hi everyone. I'm posting this here for comments and discussion during our weekly team meeting.

vweevers commented 4 years ago

How about moving it to https://nodejs.org/en/docs/guides/?

mhdawson commented 4 years ago

@vweevers another option might be a new tutorials section in the website?

vweevers commented 4 years ago

Guide, tutorial, potato, potahto, I think? The reason I suggested it is because there's no setup involved. It's a matter of copying markdown files. If certain aspects like navigation need to be spruced up then existing guides will benefit from that as well.

jschlight commented 4 years ago

I'd like whatever we do here to be of benefit to the larger Node.js community.

jschlight commented 4 years ago

During our weekly call today, @gabrielschulhof mentioned that Node Core has discussed RunKit for documentation. There is an issue here: https://github.com/nodejs/node/issues/21723

jschlight commented 4 years ago

As a first step, while we're deciding on a specific publishing strategy, I'd like to migrate all the example code used in the tutorials to the nodejs/node-addon-examples repo.

Here's the table of existing examples:

Section	Example	Location
A first project	`a-first-project`	Jim's hexo project.
Object wrap	`object-wrap-demo`	Jim's hexo project.
CMake.js	`build_with_cmake`	GitHub `node-addon-examples`
Object & function refs	`function-reference-demo`	Jim's hexo project. Code from Nicola.
AsyncWorker	`napi-asyncworker-example`	GitHub `inspiredware`
Thread-safe functions		Example is embedded in content from Kevin.
Context awareness	`multiple_load`	GitHub `node-addon-examples`

My plan is to work on migrating my examples, as shown above, to node-addon-examples.

@NickNaso and @KevinEady would you like to migrate the examples you supplied? I'm also happy to do it.

mhdawson commented 4 years ago

Options for where to publish the workshop elements

Nodejs.org
Node.js core documentation
node-addon-api repo
node-examples repo - https://github.com/nodejs/node-addon-examples

Based on the discussion we think the best fit is the node-addon-examples repo.

mhdawson commented 4 years ago

The idea is a new sub-directory called "tutorials" which would then have a directory for each tutorial.

KevinEady commented 4 years ago

Hi @jschlight , I have migrated the example to a working package in node-addon-examples, pr https://github.com/nodejs/node-addon-examples/pull/127

I stuck with the same structure (addon.js, .cc, ...) so the file names do not match exactly from the tutorial but I think that is okay 😄

mhdawson commented 4 years ago

Jim landed PR to get examples into node-addon-examples last week.

As next step he's going to document proposal for getting the labs into one of the repos/website etc.

jschlight commented 4 years ago

The first example of the content migrated to a static website using Gatsby is available for review at:

https://xenodochial-meitner-2e429e.netlify.app/content/about/what

It still needs a lot of work, but I wanted to get this preview up for comments and suggestions.

gengjiawen commented 4 years ago

The first example of the content migrated to a static website using Gatsby is available for review at:

xenodochial-meitner-2e429e.netlify.app/content/about/what

It still needs a lot of work, but I wanted to get this preview up for comments and suggestions.

We probably need a search function.

gengjiawen commented 4 years ago

Also we can use github action to auto-deploy it https://github.com/nodejs/node-addon-api/pull/691/files#diff-2bf3c905e9c6203095ea31e889d43bfbR22.

gabrielschulhof commented 4 years ago

@jschlight I found a nit:

https://xenodochial-meitner-2e429e.netlify.app/content/build-tools/cmake-js -# define NPI_VERSION +# define NAPI_VERSION

gengjiawen commented 4 years ago

https://xenodochial-meitner-2e429e.netlify.app/content/build-tools/node-gyp

Python 2 part can be removed.

Not sure Gatsby is final choice. Any thought on https://docusaurus.io/en/ ?

jschlight commented 4 years ago

We're looking at Gatsby because it's used by https://nodejs.dev.

I will look at Docusaurus, too.

mhdawson commented 4 years ago

We had considered Gatsby first as it is being used by the Node.js redesign team and some consistency is good but will be interested to hear what @jschlight reports/thinks.

jschlight commented 4 years ago

I've deployed a new preview:

https://admiring-dubinsky-535d79.netlify.app/

This preview is generated from Gatsby using a different template from the one before. The big difference here is that the code samples are now embedded directly from examples repo.

We can discuss this site during the next N-API team meeting.

jschlight commented 4 years ago

The site is now published from my fork to GitHub Pages:

https://jschlight.github.io/node-addon-examples/

The site is published to the gh-pages branch on the fork.

The next step is to create the GitHub Action necessary to automate the build and deployment of the site. This will be a challenge since the GitHub Action will need to commit and push to the gh-pages branch.

jschlight commented 4 years ago

@gengjiawen I looked at Docusarus and it is a very nice piece of work. I particularly like that it is focussed on documentation. I'm continuing to proceed with Gatbsy for a couple of reasons:

One of our prime requirements is being able to automatically include current source code from the repository into the documentation pages. Since the examples code is now regularly tested, this gives us confidence that the code presented in the documentation is current and actually works. I was not immediately able to determine how to get this functionality with Docusarus.

You can see source code included into the documentation on this page:

https://jschlight.github.io/node-addon-examples/getting-started/objectwrap
Gatsby is being used by the Node.js redesign team and there is a real benefit to Node.js in being able to share ideas back and forth.

A search function is beyond my capabilities at the moment. My plan is to reach out to the Node.js redesign team, which has about 100 contributors, to see if there is contributor there who can add search for us. It doesn't look that difficult to add, but it does require an in-depth understanding of Gatsby which I do not have at this time.

gengjiawen commented 4 years ago

@jschlight Thanks for the info.

jschlight commented 4 years ago

The licenses for the website, based on Gatsby, are as follows:

├─ MIT: 1704
├─ ISC: 118
├─ BSD-2-Clause: 38
├─ Apache-2.0: 34
├─ BSD-3-Clause: 30
├─ MIT*: 11
├─ CC0-1.0: 4
├─ BSD*: 3
├─ BSD: 3
├─ (MIT OR CC0-1.0): 3
├─ Apache License, Version 2.0: 2
├─ (MIT OR Apache-2.0): 1
├─ CC-BY-4.0: 1
├─ (MIT OR WTFPL): 1
├─ AFLv2.1,BSD: 1
├─ Public Domain: 1
├─ (BSD-3-Clause OR GPL-2.0): 1
├─ (MIT AND Zlib): 1
├─ WTFPL: 1
├─ (WTFPL OR MIT): 1
├─ GPL-2.0: 1
├─ (BSD-2-Clause OR MIT OR Apache-2.0): 1
├─ (MIT AND BSD-3-Clause): 1
├─ CC-BY-3.0: 1
├─ 0BSD: 1
└─ Unlicense: 1

The Unlicense is actually public domain.

jschlight commented 4 years ago

Background notes on licensing

The contents of the website directory were created using the GitHub "Use this template" feature on the kata-ai/grundgesetz-skeleton repository which is MIT licensed by Kata.ai.

Under the terms of the MIT license, I then modified the files in the website directory to create the tutorial site. The contents of the docs directory were essentially replaced. Changes to other files were made to implement formatting and publishing tweaks.

jschlight commented 4 years ago

The PR with the website contents is very close to landing. The steps shown here will make it possible to rebuild and publish the website on every merge to the master branch. I'm able to accomplish this on my fork, but have been unsuccessful in creating a new orphan branch in the examples repo using the traditional pull request mechanism.

1. Create an orphan `gh-pages` branch

Here are the steps I've used on my fork:

git checkout --orphan gh-pages
git rm -rf .
echo "This website is currently under development." > index.html
git add index.html
git commit -a -m "Initial commit"
git push --set-upstream origin gh-pages

2. Configure GitHub Pages to publish the `gh-pages` branch

These are the instructions:

Configuring a publishing source for your GitHub Pages site - GitHub Docs

Once these steps are complete, I can create a PR on master for the GitHub Actions file which I've tested on my fork.

A previous version of this comment included two additional steps that are not required if we use the automatically generated GITHUB_TOKEN secret when rebuilding and publishing the website.

gengjiawen commented 4 years ago

Actually, no token is needed. We can use github action for this.

See https://github.com/nodejs/node-addon-api/pull/691/files.

jschlight commented 4 years ago

@gengjiawen The link to your GitHub Action is very helpful. It appears a value is passed in as secrets.GITHUB_TOKEN on line 24. Is that correct? It would be helpful to know 1. what the value is and, 2. how it's created. The steps I listed above worked in my fork, but may not be appropriate for https://github.com/nodejs/node-addon-examples.

gengjiawen commented 4 years ago

It appears a value is passed in as secrets.GITHUB_TOKEN

It's auto injected by github action, so nothing is need to config :)

With this github action, it can also auto publish github pages when merged doc to master branch.

mhdawson commented 4 years ago

@jschlight I've completed the steps in : https://github.com/nodejs/abi-stable-node/issues/391#issuecomment-660746044

and it looks like https://nodejs.github.io/node-addon-examples/ shows the right content.

jschlight commented 4 years ago

@mhdawson Looks good! Thank you! The PR above adds the GitHun Action to rebuild and publish the site whenever there's a merge to `master.

mhdawson commented 4 years ago

Done, its in place and gets automatically rebuilt when it a PR is made to the examples.

nodejs / abi-stable-node