Create a Docs Repository

ilexp commented 6 years ago

Summary

We're currently in progress of transforming the existing wiki into a Jekyll / Github Pages based docs repository. This issue acts as an overview issue until the first release has taken place.

Progress

The basic setup is still in progress and there are a lot of issues to pick up for contributors. You'll find all required issues for a first release in this milestone. Contributions work via fork and Pull Request in the same way they do in this repository.

Original issue text:

Summary

As discussed in issue #355, we should consider moving away from the GitHub Wiki for documentation in the long term. One alternative would be to create a new AdamsLair/duality-docs repo that hosts all documentation.

Analysis

The GitHub Wiki lacks some features and also has some disadvantages from the management side:

Pages need to have unique names and folder structures are ignored.

Users can't upload images and other media.

Lack of multi-version support (crucial for v3.0).

Lack of multi-language support.

No branches for bigger edits, no PRs or reviews.

Either everyone can edit, or only people with direct push access.

History not available on a global level.

All of these problems are solved by moving docs to a new AdamsLair/duality-docs repository.

The repository's main readme file acts as the home page for documentation.

A proper directory structure needs to be defined.

Anticipate support for multiple versions co-existing.

Anticipate support for multiple languages co-existing.

Maybe something like Pages/en/vX/...?

Investigate the possibility of turning on the github pages feature for the master branch and generating a nice docs website from the markdown files.

This issue is up for grabs. Let me know if you're interested in getting some work done on this. Ideally, do the first prototypes in a public repo in your own account, which we can later either transfer or re-create in the AdamsLair organization.

htw5295 commented 6 years ago

I think creating new repository is a best for management.

BobGneu commented 6 years ago

Are you thinking we generate them? Jekyll is pretty dope from what I have heard. I would love to take part if you want the assistance.

ilexp commented 6 years ago

Are you thinking we generate them?

The main part of this issue would be to create a docs repository for a reasonably organized storage of .md files to replace our current wiki. Generating a jekyll website via github pages is entirely optional, but would be more than welcome. Could turn out to be pretty neat 😃

Jekyll is pretty dope from what I have heard.

From my own experience, totally is! Be aware though that GitHub pages predefines some aspects of Jekyll of which I'm not sure whether they can be customized like they can with a completely custom website. Not an expert here, would appreciate insight by anyone who knows more about this.

I would love to take part if you want the assistance.

Sure, appreciate help on this!

ilexp commented 6 years ago

@BobGneu So, are you on this issue? Otherwise, the call for contributors to tackle this would be open.

BobGneu commented 6 years ago

I am still interested in driving this yes. I'll nose around with a sample repo tonight and start updating this post with details and takeaways.

BobGneu commented 6 years ago

I nosed around a bit, seeing as you are already using the gh-pages feature you likely know better than I, but here are my notes:

As things stand the duality project is actually pretty heavy, due to binaries being in the repo or otherwise, so we ought to put the documentation in a separate repo for that reason if nothing else.
Assuming that route is taken, I can start transferring our documentation into a repo and put together a gh-pages structure that seems intuitive and then we can transfer the repo over to the AdamsLair space. It seems like this would be pretty easy to do, I've transferred repos previously and it's seamless.
The gh-pages approach will address most of the issues you were noting quite easily.
- Pages no longer require unique names, within sub directories
- Images and media are retained as files in the repo, able to be submitted as PRs for editing and proof reading.
- We can separate documentation by version, by creating directories for each major version and grouping files therein.
- Branches and PRs for review come with this feature, as well as security aspects otherwise - like analysis and using tools to validate markdown matches stylistic choices - https://prettier.io/ and others are out there to support more involved editing.
- History and tracking is available through git
gh-pages will generate the documentation pages as we commit/merge to master on the documentation repo, which means we can focus on creating content using markdown and create a custom theme over time to ensure that it ultimately conforms to the Duality branded color and stylistic choices. A B
As a proponent of Markdown, I find that there are definitely aspects that are frustrating. By using markdown we are only using a subset of the capabilities that are available to us with raw HTML. The good news is that we can use HTML or MD files interchangeably, with the caveat being that the HTML pages would need to duplicate content, leverage raw access to Jekyll formatters, or they may look very different, especially over time. Extensions that other editors provide that the gh flavor of markdown doesn't currently support include UML Diagrams, Katex - a latex alternative, and others. I use StackEdit at work, which has many such features if you are interested.

Overall, however, we ought to be strict about making our contributions in markdown, as anything else only makes maintenance more complicated.

Also, the wiki pages we already have are already written in markdown, so we can quickly move them over.

Please let me know if there are other elements you would like to have me look into.

BobGneu commented 6 years ago

Also forgot to mention, keeping it in MD format means we can leverage tools like pandoc to provide PDF, HTML, and so many other formats its insane to even think about.

More here

ilexp commented 6 years ago

As things stand the duality project is actually pretty heavy, due to binaries being in the repo or otherwise, so we ought to put the documentation in a separate repo for that reason if nothing else.

Yep. And even if it was only for keeping it tidy, I'd still agree 100%.

Assuming that route is taken, I can start transferring our documentation into a repo and put together a gh-pages structure that seems intuitive and then we can transfer the repo over to the AdamsLair space. It seems like this would be pretty easy to do, I've transferred repos previously and it's seamless.

Sounds good! GitHub also allows you to configure the repo so all the docs can be on the regular master branch, so we don't have to use this awkward gh-pages naming convention.

gh-pages will generate the documentation pages as we commit/merge to master on the documentation repo, which means we can focus on creating content using markdown and create a custom theme over time to ensure that it ultimately conforms to the Duality branded color and stylistic choices.

👍

Overall, however, we ought to be strict about making our contributions in markdown, as anything else only makes maintenance more complicated.

Yep, agree completely. We should limit the docs to markdown for maintenance reasons, and also to strictly separate content from design / layout.

Please let me know if there are other elements you would like to have me look into.

Nope, your writeup combined with the initial issue description sounds pretty complete for a first approach! I think it would be a good thing to have a first "prototype", and then we can take it from there and talk about how to proceed. The repository you would set up could act as this prototype.

BobGneu commented 6 years ago

Sounds good. I'll get it going this evening. Expect a link when you pop on tomorrow.

BobGneu commented 6 years ago

I have cloned the wiki repo and shoved it into my repo for testing.

BobGneu commented 6 years ago

The site is currently published at https://bobgneu.github.io/duality-documentation

BobGneu commented 6 years ago

Got the index page into place. Gonna take a break for the evening to do some thinking about organization.

ilexp commented 6 years ago

I have cloned the wiki repo and shoved it into my repo for testing.

Okay, let's iterate on this. 👍

First thing that we need to improve vs. the raw wiki export is directory structure. The root folder shouldn't have any content, and we should anticipate docs for multiple Duality versions and, potentially, in multiple languages. Multi-language is not a priority for now, but it only costs us another directory step, so we might as well include it.

I'd currently go with a directory structure like this:

Pages/en/v2/...

Except for the welcome page, which could remain in root. Not yet sure about how to organize folders inside /v2/....

Second point, is there any way to get the footer and sidebar back?

BobGneu commented 6 years ago

The template can be modified. I can look into that if you would like.

BobGneu commented 6 years ago

I reorganized the pages as requested and took the liberty of wrapping my mind around the template system, and its quite straight forward.

{% include footer.html %}

the include statement references a file within the _layouts directory, and explicitly drops its content into the template. By default the default.html template is used, where the {{content}} block is replaced by the rendered markdown.

It is rudimentary, but works.

There is the ability for each md file to include configuration in its header, as well as some customization fields as we see fit.

I tried to include markdown, but the contents are not rendered, and instead just copied in explicitly, even with an md extension.

WRT Organizing the documentation, a good first step would be to classify them based on the perspective of a new developer. Each document could be bucketed based on a Low/Medium/High experience score that we can now derive from a value in the header of the md file, tied to a badge or something in the top right of each page. Some level of flagging of Tutorials would also be beneficial, as when I was starting up the first steps were pretty daunting and I was looking for tutorials to help bridge the gaps when kicking off my first project.

BobGneu commented 6 years ago

For Inspiration: https://github.com/pages-themes

ilexp commented 6 years ago

Re-organized folder structure looks good. Also, thanks for your insight into the templating - seems like we could provide a completely custom template file, complete with footer and potentially even an auto-generated sidebar. Sounds very promising!

WRT Organizing the documentation, a good first step would be to classify them based on the perspective of a new developer. Each document could be bucketed based on a Low/Medium/High experience score that we can now derive from a value in the header of the md file, tied to a badge or something in the top right of each page. Some level of flagging of Tutorials would also be beneficial, as when I was starting up the first steps were pretty daunting and I was looking for tutorials to help bridge the gaps when kicking off my first project.

Good idea, but would defer this to some later point, when we already have all the docs moved and published. The docs repo will then also have its own GitHub project and issue list, so we'll have a good place to keep track of ideas like this too.

For now, here's a list of the things that I think need to be tackled so we can do a full docs switch, not necessarily in order:

Move img and en from root into a Pages subfolder, so we don't clutter the root directory with content specific folders as we add more later on.
Figure out how images and links work, and how they can be linked with a relative URL.
Fix the images and links in all pages.
Decide on a good base theme / template to use and customize.
Integrate the base theme.
Figure out which special templates are required, for example for the home page
Adjust the default template with footer support.
Adjust the default template with sidebar support.
Transfer the repository to the AdamsLair organization account.
Consider renaming the repo to duality-docs.
Just before release, pull all the latest Wiki pages into the new repo again to not miss recent changes. Make sure to fix images and links as done before.

@BobGneu Feel free to add any points that you have on your radar, or address any of them. This issue is somewhat big, so I think it makes sense to at some point just make the cut and turn it into a team effort. In that case, just let me know that you're ready for the switch and we'll do the repo transfer.

BobGneu commented 6 years ago

Let's do the transfer. The repo can be renamed upon transfer and from that point things will be more manageable.

Images are referenced as with standard html, relative or absolute paths are interchangeable. I already did the work to validate this on the home page. It should be pretty straight forward to correct the other images. Once we get the repo transferred we can use the issues system to track the notes above.

ilexp commented 6 years ago

I'll be off for a week, but will get back to you as soon as I'm back 👍

BobGneu commented 6 years ago

One side note, in order to transfer ownership I will need the permission relating to creating repositories within the AdamsLair space.

https://help.github.com/articles/transferring-a-repository-owned-by-your-personal-account/

BobGneu commented 6 years ago

Repo renamed.

BobGneu commented 6 years ago

Moved img and en into pages sub directory.

BobGneu commented 6 years ago

In the future, when making bulleted lists you can make them into checkboxes to simplify and track updates. All checkboxes are tracked in the first post of a PR and Issue. They show up on the issue listing.

BobGneu commented 6 years ago

Images can be referenced using markdown similar to the following

Inline Relative / Exact

![Debug Game Break](../../img/GettingStarted/RunGameButton.png)
![Debug Game Break]({{site.baseurl}}/pages/img/GettingStarted/RunGameButton.png)

Deferred Relative

![Debug Game Break][DebugGameBreak]
![Debug Game Break][DebugGameBreakDirect]

[DebugGameBreak]: ../../img/GettingStarted/RunGameButton.png
[DebugGameBreakDirect]: {{site.baseurl}}/pages/img/GettingStarted/RunGameButton.png

BobGneu commented 6 years ago

Alternatively, we can make image references.

<img src="{{site.baseurl}}/pages/img/GettingStarted/RunGameButton.png" />

BobGneu commented 6 years ago

In terms of the layout, Having a thin layout is not going to work with many of the code samples, as anything deeper than about 30 characters is going to require scrolling or wordwrap is going to be a mess, stretching things out.

BobGneu commented 6 years ago

In looking at a few of the other similar sites ~ 740 - 800px of width seems to provide enough space for code examples, though its still pretty tight.

We can create a template with a header, side menu and footer. Given that we have full HTML access we can even position the footer and headers at the top and bottom of any given window. No need for JQuery or anything. Mock up the layout you would like to see and I'll give it a go.

ilexp commented 6 years ago

Repo renamed. Moved img and en into pages sub directory.

👍

Images can be referenced using markdown similar to the following

I think we should stick to the markdown way. I don't have a clear favorite among the variants, but would vaguely prefer relative inline paths.

In terms of the layout, Having a thin layout is not going to work with many of the code samples, as anything deeper than about 30 characters is going to require scrolling or wordwrap is going to be a mess, stretching things out.

In looking at a few of the other similar sites ~ 740 - 800px of width seems to provide enough space for code examples, though its still pretty tight.

We can create a template with a header, side menu and footer. Given that we have full HTML access we can even position the footer and headers at the top and bottom of any given window. No need for JQuery or anything. Mock up the layout you would like to see and I'll give it a go.

Do we have a proper css file to work with? Might as well go for a responsive design and use the full width up to a max value for big screens, and adjust the sidebar for small screens below a min width. Could turn it into a "site header" instead in those cases.

Let's do the transfer.

Great, let's do it. I think the easiest way would be to transfer it to me, and I'll forward it to the AdamsLair org.

BobGneu commented 6 years ago

Initiated the transfer to you.

ilexp commented 6 years ago

Transferred 👍 Here's the new repo link.

ToDo

Set up labels on the new docs repo, probably similar to the ones in the main Duality repo.
Transfer all remaining ToDo items into issues in the new docs repo, but keep this one open until first release.
Fix the remaining images and links on pages that were not yet cleaned up.
Decide on a good base theme / template to use and customize.
Integrate the base theme.
Figure out which special templates are required, for example for the home page
Adjust the default template with footer support.
Adjust the default template with sidebar support.
Just before release, pull all the latest Wiki pages into the new repo again to not miss recent changes. Make sure to fix images and links as done before.

BobGneu commented 6 years ago

Labels Created.

BobGneu commented 6 years ago

images ticket created.

https://github.com/AdamsLair/duality-docs/issues/2

BobGneu commented 6 years ago

Theme ticket created https://github.com/AdamsLair/duality-docs/issues/3

BobGneu commented 6 years ago

I summed up the template based notes together in AdamsLair/duality-docs#3 and opened up a ticket for pulling in the latest from the wiki. https://github.com/AdamsLair/duality-docs/issues/4

BobGneu commented 6 years ago

I elaborated as best I could, Please feel free to edit them further.

ilexp commented 6 years ago

Nice work on the issues! The template one is actually not what I meant, but still a good idea the way you read it. Updated ToDo overview:

ToDo

Adjust labels to match naming (where appropriate) and colors from the main repo and follow a consistent color scheme overall.
Fix the remaining images and links on pages that were not yet cleaned up.
Decide on a good base theme / template to use, then integrate it to start iterating on.
- As part of this, figure out which special jekyll page templates are required, for example for the home page.
- Adjust the default template with footer support.
- Adjust the default template with sidebar support.
Just before release, pull all the latest Wiki pages into the new repo again to not miss recent changes. Make sure to fix images and links as done before.

Note that we both had been added as collaborators accidentally during the transfer. I removed us again, so access rights are now again managed via teams, but this also means that you won't be able to adjust the labels, since you're no longer a project admin. I'll pick up that one.

ilexp commented 6 years ago

Progress

Adjusted labels to use similar color-based categorization as in the main repo, but removed and renamed labels where applicable.

Immediate ToDo

Fix the remaining images and links on pages that were not yet cleaned up.
Decide on a good base theme / template to use, then integrate it to start iterating on.
- As part of this, figure out which special jekyll page templates are required, for example for the home page.
- Adjust the default template with footer support.
- Adjust the default template with sidebar support.
Just before release, pull all the latest Wiki pages into the new repo again to not miss recent changes. Make sure to fix images and links as done before.

ilexp commented 6 years ago

Created a First Release milestone on the docs repo, which contains all issues that need to be addressed in order to release.

cc @AdamsLair/duality-contributors for everyone who would be interested in joining the docs transfer with a PR or two.

ilexp commented 6 years ago

Moved all remaining docs issues from the main repo to the docs repo.

ilexp commented 6 years ago

Closing this, as all open work have been moved to the new docs repo and first release milestone.

AdamsLair / duality