search

backdrop-ops / backdropcms.org

Issue tracker for the BackdropCMS.org website
https://backdropcms.org
25 stars 21 forks source link

Setup new docs.backdropcms.org website for *all* documentation #747

Closed ghost closed 3 years ago

ghost commented 3 years ago

The Idea

I made a comment about Backdrop's documentation a while ago. Now I'd like to expand upon it and make an official request for my suggestions...

Examples of other projects that do this are:

Benefits

The benefits of these suggestions include:

ghost commented 3 years ago

To give an idea of the re-structing changes I'm proposing, and to better see why we should consolidate all docs together, here's a list of all of the Backdrop documentation (and its current location) I could find (please do let me know if I missed any!):

Developer Notes

https://backdropcms.org/developers (BackdropCMS.org > Support > Developer Notes)

Contributors Guide

https://backdropcms.org/contributors-guide (BackdropCMS.org > About > Contributors Guide)

User Guide

https://backdropcms.org/user-guide (BackdropCMS.org > Support > User Guide)

Developer Documentation

https://api.backdropcms.org/ (API > Developer Documentation)

Backdrop API

https://api.backdropcms.org/api/backdrop/groups (API > Backdrop API)

Glossary

https://api.backdropcms.org/glossary (API > Glossary)

ghost commented 3 years ago

And here's a rough idea of the re-structuring for the new site I have in mind:

Getting Started

Installing

Updating/Upgrading

Hosting/Deploying

Migrating from Drupal

Introduction

Upgrading from D7

User Guide

Quick Start

Using Backdrop

Contributed Modules

Developing for Backdrop

Modules

Themes

Layouts

Security

Contributing

Issue Queue

BackdropCMS.org

API

Appendix/Reference

General

Coding/Documentation Standards

cellear commented 3 years ago

Can we add "Migrating from Wordpress" in there? :)

cellear commented 3 years ago

Also getting data in and out.

ghost commented 3 years ago

So this issue is primarily about setting up a new site for our documentation, and then consolidating all of our existing documentation there. If/when that happens, we'll have a new, separate repo for the Docs site where requests for new/improved documentation can be posted.

That just made me realise another benefit that I'll add above: When everything's in the one place, it's easier to see where we might have gaps in the documentation and so it'll be easier to add and improve it over time.

Also just remembered another example of where I've seen this done (added link above too): https://docs.tugboat.qa/

klonos commented 3 years ago

It should be "Migrate from WP", but "Update from Drupal 7"

klonos commented 3 years ago

I'm supportive of this initiative 👍 👍 👍

bugfolder commented 3 years ago

As a relatively new Backdrop user who is still discovering the little nuggets of useful information squirreled away in diverse places, I applaud this as well. 👍 👍 👍

docwilmot commented 3 years ago

This is a great idea

jenlampton commented 3 years ago

I'm still concerned that were starting with a proposed solution (a new website) without first identifying what the actual problem is. We've been through this several times before, and every time we do so, we end up realizing that all the problems we'd like to solve can be solved with the tools we already have.

Creating yet another website will mean that we will have yet another place for people to need to know about in order to find what they need. It will mean that people will need another log-in, and we'll have more sites to maintain.

If we have time to devote to documentation, it should be spent on writing and/or organizing the documentation, and not on building a new site or setting up a new platform. I know that we're all developers and would rather be doing that, but I don't think it's what the community needs us to be spending our time on right now.

ghost commented 3 years ago

I'm still concerned that were starting with a proposed solution (a new website) without first identifying what the actual problem is.

The problem is that our documentation is spread out between two separate websites. That's not ideal. It's disjointed, hard to find, and can't be all viewed at-a-glance. Its also harder to update and maintain docs on two separate sites.

Creating yet another website will mean that we will have yet another place for people to need to know about in order to find what they need. It will mean that people will need another log-in, and we'll have more sites to maintain.

Incorrect. I have really done my best to consider issues like this and mitigate them where possible, so read through what I propose above and you'll see that I mention that the ”API site can be merged into new Docs site (since it has its own sub-pages anyway) - so we don't have yet another site/domain/repo to maintain”.

In other words, I'm proposing that we move all docs that are on b.org to the API site, reorganise them, then change the URL from api.b.org to docs.b.org. The docs site will replace the API site.

Users won't need a login to read the docs, and those who want to contribute documentation likely already have an account on the API site since that's where half our documentation's lives currently. So no extra logins needed. In fact, currently users wanting to contribute need two logins: b.org and api.b.org. With this new site, they'll only need one: docs.b.org.

And there will be no more sites to maintain than we have now.

jenlampton commented 3 years ago

I really like the proposed re-organization of content, but I don't see any advantage to moving any of it to yet another website. If we were to leave the user-guide where it is, and put everything else on the API site, that would result in a massive improvement to our documentation, with significantly less dev work.

We worked with a designer and content strategist when we first built b.org, and we discovered that separating the user-guide from the back-end documentation is an improvement over having them both in the same place. The API helps people understand how to write code, and the user-guide helps people understand how to use the interface.

People who are trying to understand how to create content and learn what taxonomy terms are, do not want to read an API. There's no reason we should make them dig through developer documentation to find what page they need to be on to push a button. (This was one one of the biggest complaints about Drupal's documentation -- that it was written by developers for developers, and did not consider the perspective of the non-developers.)

There is an additional benefit to having a user-guide on the primary, promotional website. People are already on that website to evaluate whether they want to use Backdrop or not. Having a user-guide on that site is valuable in helping them make that decision.

All of the examples above where documentation is separated out into a different site are all developer tools. None of these projects have a user-interface (or a User Guide) that is intended for a different audience, and would be better off in a more user-friendly location.

I think we need to be careful not to make the same mistakes as the Drupal project. API documentation is not "advanced" documentation, and the User Guide is not "beginner" documentation. They are documenting different things.

The problem is that our documentation is spread out between two separate websites.

Wouldn't moving everything (other than the user guide) to the API site solve this same problem?

Its also harder to update and maintain docs on two separate sites.

This sounds nice, but I don't think this is a legitimate concern. Can you provide an example where a change would require something to be updated in both places?

read through what I propose above and you'll see that I mention that the ”API site can be merged into new Docs site

I saw that. But this is also an argument for keeping the API site. The curent API site is the api, plus a handful of documentation pages. If that's the end goal for the new docs site, then there's no reason to build anything new at all.

In other words, I'm proposing that we move all docs that are on b.org to the API site, reorganise them, then change the URL from api.b.org to docs.b.org. The docs site will replace the API site.

So why go through all the extra work of building a new site? Let's just reorg the docs, and change the URL if you think docs. is better than api.

ghost commented 3 years ago

So why go through all the extra work of building a new site? Let's just reorg the docs, and change the URL if you think docs. is better than api.

Absolutely.

My suggestion to 'build a new documentation website' was just a way of stating the need for a dedicated website (which we don't currently have) to store all documentation. I wasn't meaning to imply that the actual implementation needed to be literally building a new website from scratch. I fully support and agree that the implementation to achieve this goal should be re-purposing the API site. And yes I think an important step is renaming the API site/domain from 'API' to 'Docs'. I even had an initial thought to do a POC for this idea by making a PR against the API repo and using Tugboat to test/preview it), but now that I think about it there won't be much/any changes that would go into a PR, it'll mainly be content/DB changes... So I'm sorry if I used the wrong terminology above when referring to building a new website...

As for leaving the User Guide separate from the rest of the documentation, I don't support that idea and would be interested to hear from others on that subject. Having said that, if the only way to get this to happen is to agree to leave the User Guide on B.org by itself, then I'll yield on that point for now, and we can re-assess once the new documentation site is working and please have had a chance to see it in action.

jenlampton commented 3 years ago

And yes I think an important step is renaming the API site/domain from 'API' to 'Docs'.

This is a good place to start. We'll need to do some server magic to rewrite all he URLs when a request for api comes in. But I suspect @larsdesigns can probably figure out the nginx config to add for that...

I'm sorry if I used the wrong terminology above when referring to building a new website...

No need to apologize. I hear "build a new website" and my brain hears "more unnecessary work". Since the problem we're facing isn't technical so much as organizational, let's organize first :)

As for leaving the User Guide separate from the rest of the documentation, I don't support that idea and would be interested to hear from others on that subject.

I'd also like to hear from others, but preferably not any other developers.

So far the only calls I've heard to merge the User Guide with the developer docs have come from developers and not the people we intended to read the user guide. I'd like to hear from Content editors and Decision makers (though, those voices are historically the ones that are hardest for us to get) to see if they currently use the User Guide, and how they'd feel about having it merged with the API site.

bugfolder commented 3 years ago

Although I'm also a developer (sort of), I'm mostly a content editor. When I use the docs, if I'm looking up how to use something, I use the User Guide; when I'm looking up how to alter or augment something, I use the API docs.

As another example of ways of addressing this issue, CiviCRM has a whole slew of different guides that all live at https://docs.civicrm.org. While I would not hold up Civi as a paradigm of great documentation in all things, that is a nice organization of information.

ghost commented 3 years ago

CiviCRM has a whole slew of different guides that all live at https://docs.civicrm.org.

Thanks for that, hadn't seen it before. This is what I'm envisioning for Backdrop - a 'docs' site that has everything in one place. User guide, developer docs, upgrading, installing, etc. All on the one site, but all in their own sections. Users don't have to read developer guides, but at least they know where they are should they ever want to try writing some code or something down the track. And with the ability to see all documentation at a glance, you can better see where something's missing to better improve it.

olafgrabienski commented 3 years ago

This is what I'm envisioning for Backdrop - a 'docs' site that has everything in one place. User Guide, developer docs, upgrading, installing, etc. All on the one site, but all in their own sections.

In my opinion that would be an improvement. I see @jenlampton's point re separating User guide and Developer documentation. But there are things that belong in both (or even more) places, and if some of them are on different websites and others not, it's harder to find the information I need and to see how it is related.

An example: The installation of Backdrop is mentioned in the User Guide on https://backdropcms.org/user-guide/quick-start. The page links to https://backdropcms.org/installation, which according of the sidebar navigation is part of some "Developer notes" of the User Guide (without navigation items referencing to the actual User Guide, though). Finally, there is a link to "advanced instructions" which points to a page on the API site. In my opinion, it would be more easy to place and connect these instructions on one site.

ghost commented 3 years ago

Documentation is easy to find

Case in point: I just went looking for the documentation re. how to adopt an abandoned project. I knew I'd seen it somewhere before, so I first went to B.org, then to the Developer section there. Couldn't find it. Maybe it's in the Contributors Guide then... Nope. User guide? Also nope. Ok, must be on the API site then. Finally found it under: Developer Documentation > Contributing for Backdrop CMS > Contribute to a module, layout, or theme > Contrib application process (right at the bottom of the page).

That just made me realise another benefit of having all documentation on the one site: searchability! You currently can't search for a topic across multiple sites. But on one site, you could search and find everything about a particular topic.

ghost commented 3 years ago

A brief discussion about this in the meeting today resulted in:

My thoughts on a plan are to clone the API repo to a new 'docs' repo, then use this to install the site live on the B.org server under the 'docs' sub-domain with a copy of the (non-sanitised) API DB. Then start adding documentation content from B.org onto the new site.

Questions I have are:

I'd love to get @larsdesigns' feedback on all of this 😃

docwilmot commented 3 years ago

I think it makes a lot of sense to have all docs in one place and I again approve this message. Re developers vs non-developers, I dont think this is much of an issue. Prior to learning to write code, searching Drupal for ideas on how to do things, the most frustrating thing as a non-developer was having to jump all over the place and search incessantly to learn stuff. And even as a non-developer you find yourself searching through all sources; the idea is to solve your question or issue, not whether its an API doc or a User doc (I dont think I understood that "API docs" were different from "User docs" then either), as long as it helps. And what helps is having everything accessible, and a single site would be sensible.

larsdesigns commented 3 years ago

Redirects are possible...

ghost commented 3 years ago

Looks like we can try using BIEN to export content from B.org and import into the new site. I can even do the exporting from my local copy of B.org so as not to install extra modules on the live site unnecessarily.

jenlampton commented 3 years ago

Is there a better/easier way?

Yes.

It would be a lot less work to add the new docs content directly onto the api site. We don't need another site. We don't even need another repo.

I'd like to recommend the first step of this process be to change api. to docs., and set up the redirect.

How do we handle URL redirections? (e.g. api.backdropcms.org/... -> docs.backdropcms.org/...) Will the core Redirect functionality work, or do we need some sort of server-level redirection in place?

This will need to happen at the nginx level. We need to recieve all incoming traffic at api., and rewrite the URL with docs. instead, so the path remains unchanged. Core redirect will not work for this purpose.

Then, any content we want moved can be migrated (I recommend using feeds module since we'll be able to keep everything from the current nodes if we want it). Then, all the docs pages can be re-organized in their final destination.

If we want to leave all the new content unpublished until it's ready for prime-time we can do that :)

ghost commented 3 years ago

We don't need another site. We don't even need another repo.

I thought we did because the current repo is named api.backdropcms.org, but if we can just rename it and keep all the existing issues, code, etc., then that's obviously preferred 🙂

I'd like to recommend the first step of this process be to change api. to docs., and set up the redirect.

👍🏻 I will liaise with @larsdesigns to get this done.

jenlampton commented 3 years ago

but if we can just rename it

Yep, we can. :) Let's rename it the same time as we switch the subdomain and do the redirect.

I will liaise with @larsdesigns to get this done.

❤️

Re developers vs non-developers, I dont think this is much of an issue.

Also -- just for the record -- if everyone feels that moving the non-developer docs over to this site is the preferred way to go, I'll support it. As always, thank you all for continuing to bring up documentation issues and working hard to improve everyone's experience with Backdrop :)

larsdesigns commented 3 years ago

Completed the following:

  1. Created a clone of the api.backdropcms.org as docs.backdropcms.org including database and files.
  2. Created a docs user for the server and database.
  3. Created DNS A record for docs.backdrop.org (@BWPanda).
  4. Configured the docs site clone to run as docs.backdropcms.org (@BWPanda).
  5. Created SSL certificate for docs.backdropcms.org.
  6. Created nginx redirect for api.backdropcms.org to docs.backdropcms.org.
  7. Recorded the new ssh pasword in 1password.
  8. Recorded the sql password in the ~/mysql.txt file.

@BWPanda, is going to update content within docs.backdropcms.org.

Everything seems to be in working condition. Let me know if something comes up.

larsdesigns commented 3 years ago
  1. Created the cron jobs for docs.backdropcms.org.
  2. Disabled the cron jobs for api.backdropcms.org. It will take a little while but the sanitized backups should drop off for the api site.
jenlampton commented 3 years ago

I was going to rename the github repo, but it looks like someone's beaten me to it :) https://github.com/backdrop-ops/docs.backdropcms.org

ghost commented 3 years ago

Then, any content we want moved can be migrated (I recommend using feeds module since we'll be able to keep everything from the current nodes if we want it). Then, all the docs pages can be re-organized in their final destination.

@jenlampton What do you mean by "we'll be able to keep everything from the current nodes if we want it"...? Does Feeds let you do stuff that BIEN doesn't?

I'm planning to do the migration of content from my local B.org site to the live Docs site, but I've also heard comments that @jenlampton was planning to migrate the content. So as not to duplicate efforts, how best can we coordinate this? Divide and conquer? Be good to nail down the exact migration steps too so we're all on the same page...

jenlampton commented 3 years ago

@jenlampton What do you mean by "we'll be able to keep everything from the current nodes if we want it"...? Does Feeds let you do stuff that BIEN doesn't?

I would assume so, because it can do so much. But I don't know what BIEN does so I can't really say.

I'm currently using Feeds on 6 backdrop sites, and about 15 Drupal sites. It's running imports as frequently as every 15 minutes in production, pulling from YouTube, Zotero, Granicus, CSV files, other various JSON or XML feeds, and often from one of my sites to another (usually when I need the same content in both places).

If this experience would be helpful - let me know.

I'm planning to do the migration of content from my local B.org site to the live Docs site,

I was planning on doing it from the live b.org site to the live docs site, every 12 hours or so. Until we are ready to pull the plug on the old docs pages. (Then we don't need to worry about things getting out of sync)

but I've also heard comments that @jenlampton was planning to migrate the content. So as not to duplicate efforts, how best can we coordinate this? Divide and conquer?

I didn't want to do anything while you were away since I wasn't sure what your plan was. If you would prefer to plow forward without me, that's fine too. :)

Be good to nail down the exact migration steps too so we're all on the same page...

Yes, let's make a plan.

Do you already know what you want to keep from the old docs pages? All the same fields? Any of the node metadata like Author info, published status, published date? What about menu items, and/or book order? (I was thinking maybe you wouldn't want to keep the book and book order info...)

edit: There are some book pages that we are not moving over, it would also be good to get some solid logic on what makes a book page one we are moving vs one we are not.

ghost commented 3 years ago

I was planning on doing it from the live b.org site to the live docs site, every 12 hours or so. Until we are ready to pull the plug on the old docs pages. (Then we don't need to worry about things getting out of sync)

Ah, nice! I hadn't considered that. I was just going to do a once-off migration.

If this experience would be helpful - let me know.

Yes please. It sounds like you have a better grasp of what's required here @jenlampton (and undoubtedly more time than I have ATM), so please take this over from me in order to move this issue forward.

Do you already know what you want to keep from the old docs pages?

I hadn't considered exactly what would be migrated. I guess I just assumed everything would be copied over as-is, then any future thoughts about possible changes and improvements could be addressed in separate issues.

As for what to copy over, I think the list I added above should contain all the content that needs migrating.

ghost commented 3 years ago

Also, @larsdesigns and I were talking a week or so back about how to do redirects for individual pages/URLs from B.org to Docs, and after looking into it Idiscovered that Backdrop's core Redirect module/functionality allows redirecting from a Backdrop path to an external one. So my plan was to create redirects on B.org for all migrated content pointing to the same node on the Docs site.

jenlampton commented 3 years ago

@BWPanda the importer is set up on the docs site to pull in all "Documentation" nodes every 12 hours. There are currently 77 nodes that will get synced. I copied both the node type and the text format from b.org to docs.b.org, but I added a Legcacy NID field to the docs site. (That can be removed when we're done) You may want further adjustments -- but have a look and let me know if you need anything else :)

(Also -- feel free to create separate issues in the docs queue if needed)

ghost commented 3 years ago

I think we can close this now, since the new site is setup and working. I'll open separate issues in the Docs issue queue re. migrating content, etc. as needed.

ghost commented 3 years ago

Actually I'm going to re-open this as this seems like the best place to discuss the progress of migrating the documentation content over. Once it has been migrated and is the canonical source of documentation, then we can use the Docs issue queue for ongoing tickets.

ghost commented 3 years ago

So I posted this in Zulip, and will copy it here as an update for those interested/involved:

Ok, so I've made a start on this, see https://docs.backdropcms.org/documentation/getting-started (bottom of the right sidebar menu). I created blank (for now) book pages for the sections, and am then going through the migrated content and editing it to update the book settings and menu settings.

The problem is I need to do both - add it to the appropriate book/parent, and add it to the appropriate menu/parent. These two things seem very similar, and it feels like I'm duplicating work. So I'm going to stop here for now, and file an issue about making menus act more like books, then we'll only need to edit each node to update the menu settings, and won't have to worry about books anymore.

The problem is that we want to have links in the menu for docs pages (e.g. the right sidebar on the Docs site), but we also want to have those handy pager-type links at the bottom of each page to navigate back and forth easily, therefore we have to use the Book module too. But adding a docs node to a menu (and specifying a parent, weight, etc.) and then also adding the same node to a book (and specifying the parent, weight, etc.) seems like a duplication of work.

My suggestion therefore is to add functionality to menus that allow the display of a pager that navigates between links in that menu. Basically a combination of Book and Menu. I found a Drupal module that does essentially that: https://www.drupal.org/project/menu_pager

My questions now are:

ghost commented 3 years ago

Maybe I'll proceed with setting up nodes with menu links (so they appear in the right sidebar), but I won't add Book functionality (since hopefully that'll be replaced in future anyway).

Still open to feedback though.

jenlampton commented 3 years ago

I like the idea of adding a contrib module immediately, and working towards replacing book in core in 2.x.

On Mon, Mar 29, 2021, 8:05 PM Peter Anderson @.***> wrote:

Maybe I'll proceed with setting up nodes with menu links (so they appear in the right sidebar), but I won't add Book functionality (since hopefully that'll be replaced in future anyway).

Still open to feedback though.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/backdrop-ops/backdropcms.org/issues/747#issuecomment-809869904, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADBER2ZBZWWLIKTUY6GA7DTGE5XPANCNFSM4WE7QJWA .

ghost commented 3 years ago

Here you go: https://github.com/backdrop-contrib/menu_pager

I have it installed on my local copy of the Docs site. It works nicely, but is duplicated by the Book navigation on certain pages. We'll need to remove the Book module if possible, as long as it doesn't remove the content type, since that's what we're using for the documentation nodes...

jenlampton commented 3 years ago

I think it will remove the nodes. Can we add a setting to disable the book papger?

On Wed, Mar 31, 2021, 4:02 AM Peter Anderson @.***> wrote:

Here you go: https://github.com/backdrop-contrib/menu_pager

I have it installed on my local copy of the Docs site. It works nicely, but is duplicated by the Book navigation on certain pages. We'll need to remove the Book module if possible, as long as it doesn't remove the content type, since that's what we're using for the documentation nodes...

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/backdrop-ops/backdropcms.org/issues/747#issuecomment-810976795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADBER7KQULR2JC3HDHWV7LTGL6NDANCNFSM4WE7QJWA .

ghost commented 3 years ago

I uninstalled the Book module from my local site, and it leaves the content type in place, so all nodes, etc. still working nicely.

ghost commented 3 years ago

So I've update the live Docs site to use the new Menu Pager module. In addition to its defalt functionality of adding previous/next links based on the current page in the menu, I've added functionality similar to Book module to show child links for each parent page. I've then disabled (but not uninstalled, just in case) the Book module from the Docs site, so now all navigation is being handled by the main menu and Menu Pager module.

Now I think we can close this issue as 'complete' and use the Docs issue queue for ongoing bug fixes, feature requests, etc.