Refactor Ansible Core Documentation Into Modular Guides

[dharmabumstead]

Proposal: Refactoring the Ansible Core Docs

Author: Scott Butler <@dharmabumstead>

Date: 2017/10/15

• Status: New
• Proposal type: documentation
• Targeted Release: 2.5
• Associated PR: no PR; original brainstorming here: https://docs.google.com/a/redhat.com/document/d/1P8gmNGoXAuQ6NcrX_pVZ2kyiB2l2U3W3Eo0ObDGPV-g/edit?usp=sharing
• Estimated time to implement: TBD

Motivation

We need world-class, enterprise-ready documentation.

This proposal is to refactor the Ansible Core documentation from one giant set into targeted, modular guides. This will make it easy for the multiple audiences that Ansible serves to zero in on the information that is specific to their usage scenarios without wading through material that's not useful to them.

With the addition of Ansible Engine, we also have more Enterprise-focused customers purchasing Ansible for the first time. Enterprise customers are accustomed to seeing targeted guides (as is typical in the Red Hat documentation portfolio). As our customer/user focus changes over time, Enterprise-focused customers will demand polished, well-organized documentation that is easier use.

Most world-class software documentation - both open-sourced and closed-source - is published as targeted guides rather than a single monolithic document. Aside from being easier for the end user, there are tangible benefits to making the documentation pieces smaller and easier to work with when seeking help from community members. Here are some good examples:

Fedora -- https://docs.fedoraproject.org/f26/install-guide/index.html Docker - https://docs.docker.com/ (guides)
https://docs.docker.com/manuals/ (product manuals) Amazon Web Services - https://aws.amazon.com/documentation/ Python - https://docs.python.org/3/ Eucalyptus - https://docs.eucalyptus.com/eucalyptus/4.3/

Problems

Our current documentation is unwieldy and not organized as well as it could be. It's not as easy as it should be for a specific type of customer to zero in on what they are interested in without having to wade through things that they're not interested in.

It's also not easy to work on the documentation. With the exception of the work we've done in the past year on the community guide and dev guide, most of the docs are jumbled into one huge directory with little organization (as I write this, there are currently 1501 rst files in the /ansible/docs/docsite/rst directory, many of which are generated RST module files). These files can and should be organized more cleanly.

Solution proposal

Break the documentation out into targeted guides.

Move the documentation source files into directories that are organized by guide.

Here’s a high-level view of how I think the guides should be broken out:

1. "Start Here" Guide

2. Installation (& Configuration/Admin?) Guide

3. Users Guide

4. Developer Guide

5. Community Guide

6. Reference Guides & Appendices:

   • Module Reference
   • Configuration Settings
   • Glossary
   • Playbook reference (near future)
   • API Reference (future)

7. Release Notes

8. Roadmap

URLs for the new docs would just have an additional level. They could look something like this:

http://docs.ansible.com/ansible/latest/start-here/index.html http://docs.ansible.com/ansible/latest/install-guide/index.html http://docs.ansible.com/ansible/latest/user-guide/index.html http://docs.ansible.com/ansible/latest/developer-guide/index.html http://docs.ansible.com/ansible/latest/community-guide/index.html http://docs.ansible.com/ansible/latest/reference/modules/index.html http://docs.ansible.com/ansible/latest/reference/configuration/index.html http://docs.ansible.com/ansible/latest/reference/yaml/index.html http://docs.ansible.com/ansible/latest/reference/glossary/index.html http://docs.ansible.com/ansible/latest/reference/playbook/index.html http://docs.ansible.com/ansible/latest/reference/glossary/index.html http://docs.ansible.com/ansible/latest/release-notes/index.html http://docs.ansible.com/ansible/latest/roadmap/index.html

Doc structure for the rst directory would look similar to the URLs:

rst/start-here/ rst/install-guide/ rst/user-guide/ rst/developer-guide/ rst/community-guide/ rst/reference/modules (or maybe just rst/modules, since these are generated) rst/reference/configuration/ rst/reference/glossary/ rst/release-notes/ rst/roadmap/

---

Proposed Refactored Doc Structure (for illustration purposes; not a 100% complete outline)

1. Start Here Guide:

   • Intro to Ansible
   • Concepts
   • Quick Start Tutorials
   • Where to go next:
   • “I want to install Ansible”
   • “I want to manage Linux machines with Ansible”
   • “I want to manage Windows machines with Ansible”
   • “I want to manage networking with Ansible”
   • “I want to extend Ansible”
   • “.....”

2. Installation and Upgrade Guide:

   • Intro
   • Selecting a Version
   • Machine Requirements
   • Control Machine
   • Managed Nodes
   • Installing the Control Machine
   • Installing From Packages
   • Installing From Source
   • Porting Guides
   • Ansible 1.x to 2.0
   • Ansible 2.2 to 2.3
   • Ansible 2.3 to 2.4

3. Users Guide:

   • Intro
   • Working With Ansible Core
     • Working with the Command Line
       • ansible
       • ansible-playbook
       • ansible-vault
       • ansible-galaxy
       • ansible-console
       • ansible-config
       • ansible-doc
       • ansible-inventory
       • ansible-pull
     • Working With Inventory
     • Working with Modules
       • Intro
       • Module Return Values
       • Maintenance and Support
       • Link to Module Reference Guide
     • Working with Playbooks
       • Intro
       • Variables
       • Conditionals
       • Loops
       • Blocks
       • Lookups
       • Prompts
       • Tags
       • Strategies
       • Reusable Playbooks
         • Includes/Imports
         • Roles
       • Advanced Playbook Concepts
         • Privilege Escalation with become
         • Accelerated Mode
         • Asynchronous Actions and Polling
         • Delegation and Rolling Updates
         • Check Mode
         • Debugging
         • Error Handling
     • Working with Ansible Vault
   • Ansible Cloud
     • AWS
     • Azure
     • Rackspace
     • Google Cloud
     • CloudStack
   • Ansible Networking
     • Intro ...
   • Working with BSD
   • Working with Windows
   • Working with Networking
   • Best Practices

4. Developers Guide:

   • Intro
   • Developing Modules
     • ...
   • Developing Plugins
   • Developing Dynamic Inventory Sources
   • Developing the Ansible Core Engine
   • Testing
   • Release and Maintenance

5. Community Guide: Proposal(as current)

6. Reference & Appendices:

   • Module Reference
   • Configuration Settings
   • YAML Syntax
   • Glossary
   • Playbook reference (near future)
   • API Reference (future)

7. Release Notes

   • What's New
   • Resolved Issues
   • Known Issues

8. Roadmap

Dependencies (optional)

Explain any dependencies. This section is optional but could be helpful.

• Process for generating module docs would need to be updated to put RST in their own directory
• Build process would need to be updated
• We would need to do some redirects
• Core docs landing page would need to be revamped

Testing (optional)

We would need thorough testing to ensure we haven't missed any redirects. This can be mostly automated (Integrity link checker, etc).

Documentation (optional)

We should have a high level overview of where the docs live - the purpose of each guide, the logic behind how the guides are laid out, and where the source files should live.

Anything else?

[bcoca]

A few things:

• plugin docs already follow a subdir structure plugins/<plugin type> using the same code that generates modules with altered paths. So it should be very easy to change modules to do same.

  • 'working with playbook' section is missing a lot of plugins, I would suggest a plugins section that contains all of them.

  • 'accelerated mode' has been removed ... we should not have a section for it

  • 'diff' mode and 'lists' might merit their own section or a 'introspection' section that groups them with 'check mode'

  • 'playbook reference' is this the same as the playbook keywords docs?

[kustodian]

I'm interested where will the filters and tests be? I can't see them anywhere in the structure and they are probably the most used pages once you are in Ansible for a long time.

[dagwieers]

I have always had great respect for PHP's annotated manual that must have existed 20 years ago. Users could add comments to the documentation to clarify information or add examples.

Every few months people would go over the comments and would improve the documentation (cleaning the comments). This way the feedback is readily available to others, and the documentation becomes more relevant.

I know Github enables people to make PRs directly, but from a user POV an annotated manual is much more powerful to engage users, and provided feedback can converge over time.

[dagwieers]

Initially the Ansible documentation was a guide you could read from front to back, starting with the basics and building on new stuff learned, page by page.

[bcoca]

@kustodian most plugins http://docs.ansible.com/ansible/devel/plugins.html, tests and filters are still under development.

@dagwieers I do think that separating the 'intro' from 'manual' is useful, the structure proposed above seems to do this.

[gundalow]

I have always had great respect for PHP's annotated manual that must have existed 20 years ago. Users could add comments to the documentation to clarify information or add examples. Every few months people would go over the comments and would improve the documentation (cleaning the comments). This way the feedback is readily available to others, and the documentation becomes more relevant. I know Github enables people to make PRs directly, but from a user POV an annotated manual is much more powerful to engage users, and provided feedback can converge over time.

Definitely some interesting ideas here, though this should be a separate proposal to this one

[alikins]

Do urls for existing docs (incoming links) need to be preserved?

(My vote would be to redirect for the straightforward cases, but not to block documentation improvements if a simple redirect isn't practical).

[wenottingham]

Ideally the 'simple' cases for redirect would include the module index/docs.

[alikins]

(This is kind of a docs wishlist, only some of which could be considered part of a refactor. So this is partially off topic, but while it is in my head...)

• reference of all available jinja and ansible provided jinja filters and tests. ie, all of them in the same doc, so I don't have to jump back and forth between jinja and ansible docs

• module_utils API docs including version_added info

• different styling for doc 'notes'. Not sure why, but my brain tends to skip over the notes when reading the docs. I think partially because they often look like decorative horizontal rules instead of highlighted notes. I have no suggestions on what would be better though.

• Have the various playbook example snippets link to that example being used in situ in a real playbook. And ideally, playbooks that follow the best practices. That would be a ton of work but would be fantastic for learning how all the parts go together.

• A couple of playbook walk throughs of real playbooks with annotations and explanations.

• A walk though of building a playbook and inventory from scratch.

• Eradication of any glib uses of 'obviously' or 'simply' or 'trivially' or 'just'. For example, http://docs.ansible.com/ansible/devel/dev_guide/developing_inventory.html:

"Simple! We just create a script or program that can print JSON in the right format when fed the proper arguments. You can do this in any language."

• Fact documentation reference. What facts are commonly available, what do the data structures look like, what do they mean. This could arguably be something fact returning modules should document. (http://docs.ansible.com/ansible/devel/hpilo_facts_module.html being a good example). Maybe a reference list of the standard facts and a list of facts available from other modules?

• Some way to indicate variable precedence when discussing anything that can set/provide a variable. For example, when describing 'register', it would be great if there were a little snippet/widget indicating where it fit in. For ex:

(lower) [ role default | inventory file or script group vars | inventory group_vars/all | playbook group_vars/all | inventory group_vars/ | playbook group_vars/ | inventory file / script host vars | inventory host_vars/ | playbook host_vars/ | host facts | play vars | play vars_prompt | play vars_files | role vars | block vars | task vars | role param | include_param | include_vars | set_facts / registered_vars | extravars ] (higher)_

• Cross reference http://docs.ansible.com/ansible/devel/glossary.html entries to pages elaborating on them. For example, adding a link to http://docs.ansible.com/ansible/devel/intro_adhoc.html for the 'Ad Hoc' definition.

• Examples and cross references on http://docs.ansible.com/ansible/devel/playbooks_keywords.html

[dharmabumstead]

@alikins Great list - thanks!

Yes, cleaning up the wording and tone is a continuing area of focus.

[bcoca]

@alikins https://github.com/phred/ansible-examples/blob/master/pedantically_commented_playbook.yml

[alikins]

(more brainstormy bits)

• add a section with high level overview of 'What is Ansible' and make sure it has an entry in the nav tree. More or less the 'about' of http://docs.ansible.com/ansible/latest/index.html. I wouldn't mind seeing that info in more of a bullet list format, but breaking up the blob of text with a few highlighted keypoints would help.

  • ansible is used to automate systems in a way that is:
    • Repeatable
    • idempotent
    • predictable
    • ordered
    • described in a format
    • that can be stored in source control
    • read and understood by humans
    • read and understood by software
    • agentless
    • minimal requirements on systems to be managed
    • uses and extends tooling already widely used
    • ssh for transport and remote management
    • wide assortment of ways to provide system inventory
    • capable of managing a wide variety of platforms
    • intended to be usable with and by existing automation
    • targets non-interactive use
    • adaptable to existing systems without requiring rebuilding existing systems from scratch
    • adaptable to tools and processes that create and provision new systems
    • extensible with minimal requirements

• related to above, add a section about (for lack of a better term...) 'ansible philosophy' detailing what ansible is best for, what kind of problems to tackle, how to think about automating problems, the underlining intention of the 'best practices', etc. Some examples:

  • inventory enumerates systems and provides enough info for ansible to be able to manage them
  • system config info that can be parameterized should use playbook/role vars to provide values for the params

[dharmabumstead]

@alikins The 'Getting Started' section listed at the top of the outline is where the new 'front matter' will live. This would be a high level intro and overview that is intended to cover what Ansible is and does. All of the other guides will have fresh intro material that cover high level concepts and set expectations for who the audience is and a baseline for what is covered and not covered. As it is now, most of the docs dive directly into bare metal detail with no preamble, which can be disorienting and unfriendly.

[halberom]

Not sure if this is the right proposal - but it would be good to get the lookup plugin linked to/listed more prominently - especially as the new loop: structure uses them in a much more exposed fashion.

[bcoca]

@halberom way ahead of you http://docs.ansible.com/ansible/devel/plugins/lookup.html#plugin-list

[halberom]

I found that right after I commented (nice work) - still think it needs to be more prominent and/or linked to. Maybe also an entire plugin section on the left, given how plugin orientated Ansible is becoming.

[dharmabumstead]

Implemented, so closing.