define/improve documentation structure

[wholmgren]

We should consider implementing the documentation structure ideas described in this pycon 2017 talk and associated blog post:

https://www.youtube.com/watch?v=azf6yzuJt54

https://www.divio.com/en/blog/documentation/

My hope is that a more clearly defined documentation structure will help users identify how they can easily contribute to the documentation.

If people are interested and willing to commit the time to produce more how-to and tutorial content, I would volunteer to improve the documentation infrastructure and then help users get content into the documentation. Ideally, adding to the documentation would be as easy as adding a new jupyter notebook, markdown, or rst file to a source/howtos or source/tutorials directory. This would be a post-PVSC project.

[cwhanse]

@wholmgren what do you think about SunPy's documentation structure? Looking around at other projects to find a model for pvlib-python

[wholmgren]

I like it. I suggest we pursue that refactoring once #846 is merged.

[wholmgren]

I also like where MetPy is going here https://github.com/Unidata/MetPy/pull/1457

[kandersolar]

I just stumbled on this -- too late for this year I guess, but maybe next year: https://developers.google.com/season-of-docs

[mikofski]

There's also outreachy:

Outreachy internship projects may include programming, user experience, documentation, illustration, graphical design, or data science.

Altho I believe Google Season of Docs recruits professional tech writers who probably need considerably less oversight than an intern.

[kandersolar]

FYI the Divio link is now broken (but is still on archive.org). Here is an updated link that seems to have more detail than the original page did: https://documentation.divio.com/

I had a hard time understanding the differences between "Tutorials" and "How-To Guides" until I saw this table again:

	Most useful when we’re studying	Most useful when we’re coding
Practical steps	Tutorials	How-to guides
Theoretical knowledge	Explanation	Reference

So at a basic level, "how-to" guides are for people that are already writing code but want something to copy/paste for their specific task. Tutorials are for people who are looking to learn and the exact content doesn't need to be immediately applicable.

Anyway here is a first attempt at recategorizing our current docs with the Divio structure:

Divio category	Divio analogy	Current docs examples
Tutorials	Classroom lesson	Some of our gallery examples, sort of
How-To Guides	Cookbook recipe	Most of our current gallery examples, sort of
Explanation	High-level discussion	Most user's guide pages; maybe some gallery examples
Reference	Encyclopedia entry	API Reference pages, Variables & Symbols page

I propose this strategy to guide reworking our existing content and adding new content in the future:

• Tutorials: targets people who need an entry point to get started. Its goal is to provide the initial background and experience necessary to effectively use the rest of our documentation. I like the idea of having a range of complexity: start with very basic tutorials with minimal code, then progress to more complex "case studies" that develop intuition about using pvlib in general and provide examples of how to use pvlib for "complete" real-world tasks. These Tutorials could form the basis of future in-person tutorial sessions like the PVSC/PyData/PVPMC sessions.
• Example Gallery: targets people who were already working on pvlib-based code right before opening pvlib's docs. Its goal is to provide a library of straightforward code examples that users can copy/paste into their own scripts to achieve some small and specific goal. These pages are short and probably shouldn't have explanatory paragraphs (brief comments are ok). They also don't include abstractions like for loops or helper functions. If a gallery page feels like it should have explanation, or perform some comparison or analysis, it should be a case study in the Tutorials section. Calling get_solarposition with a DatetimeIndex fits in here; comparing solar position algorithms does not.
• User's Guide: targets people who want to understand the bigger context of what their code is doing. Its goal is to deepen high-level understanding, not necessarily show how to do something. The ModelChain and time/timezone pages are great examples of this. I think I'd argue that our current Intro Tutorial page also falls into this category (I'd rename it to Modeling Paradigms).
• API Reference: targets people that really want to nail down the details on a specific part of pvlib. Its goal is to thoroughly document the code without getting distracted by explanation.

With that strategy in mind, I see two main areas for improvement:

• The example gallery is currently an awkward category chimera; plot_partial_module_shading_simple is more Tutorial and I'm not sure whether plot_interval_transposition_error is more Tutorial or Explanation. Additionally, the examples that skew closer to How-To should be simplified. For example plot_ghi_transposition defines a helper function and demonstrates seasonal variation. I think the Divio approach would nix all that and just focus on getting from basic weather data to calling get_total_irradiance. Quantifying irradiance seasonality could be a Tutorial case study.
• I think we are currently very weak when it comes to complete beginners who need something simple and easy to grasp. I'm imagining something with minimal code, for example using a single day of data from the NIST ground system, use measured GTI and Tamb with sapm_cell and pvwatts_dc to predict DC power and compare with the real measured AC power. From there we could have a few other beginner Tutorial pages that build up to more complex (but still approachable) tasks like making a PVWatts emulator with ModelChain, then fetching a TMY and generating an 8760. Then have a separate "Advanced" section or something with the case studies. Perhaps a section for reproducing figures from scientific papers would be valuable.

I declare this wall of text complete and solicit your feedback.

[cwhanse]

+1 to the proposed strategy. The Intro Tutorial was intended for beginners, but I think our idea of a beginner was someone who already knew python and had some understanding of PV production modeling and needed to know how to use pvlib. I'd rather not try to teach PV power modeling by means of the pvlib docs, but we could copy the process figure from pvpmc.sandia.org if we think we need something.

I would like the API reference to become easier to use. Functions are not in alphabetical order, full paths are displayed (e.g., pvlib.clearsky.bird) and some methods are mixed in with base layer functions (pvlib.location.Location.get_clearsky) is in the "Clear sky" category.

[adriesse]

In terms of priorities, I would see API as number one and the rest as nice to have.

[wholmgren]

@kanderso-nrel thanks for the careful analysis. It's hard for me to see how we go from here to there without a single enormous PR from a hero (i.e. you :) ). But maybe as a first step we could rewrite a few examples and create two categories.

The API reference worked a lot better before we adopted the pydata sphinx theme. For example, see 0.8.0 API docs. I haven't kept up with it, but maybe a newer version of pydata sphinx theme would help with general readability too.

[kandersolar]

Agreed the new theme really makes a mess of the API reference. The problem is that, in the new theme, the left sidebar is for listing sub-pages and in-page sections go in the right sidebar, whereas the old theme put them both in the left sidebar. I will try to figure out some way to improve this.

It's hard for me to see how we go from here to there without a single enormous PR

Yeah. Maybe some alternative workflow would make sense here, like sending PRs to an intermediate (non-master) branch to allow smaller piecemeal reviews without making a mess of master in the meantime. I think more thought is needed about how specifically the docs would change, and then maybe a sensible implementation plan will emerge.