Release: 0.1.0 & 0.2.0

[zaneselvans]

On our call this morning, we made an attempt to cull issues from the 0.1.0 milestone and... we didn't remove very much. It also seemed like we lacked consensus on what level of done-ness is required for us to make a release. In the hopes of getting some more clarity on that and maybe input from folks who are using the software already (@gschivley, @joshdr83, @briannacote, @karldw, @udayvaradarajan) I created this issue for discussion leading up to our completion of the release!

Some of the larger questions included:

Documentation

• Includes issues #316, #317, #318, and #333
• We have two reservoirs of documentation right now: Python docstrings and a few markdown files checked into the repository.
• Some of the markdown files are "special" in that they are recognized by GitHub and automatically integrated into its presentation on the web (the software license, code of conduct, contributing guidelines, README, etc.).
• Other markdown files (e.g. the getting started guide) exist only inside the repository.
• At a minimum, we need a little bit of glue that ensures all the existing documentation content shows up in a readable / searchable way in the Sphinx documentation that's built on ReadTheDocs.
• Are there any additional pieces of documentation that we think are absolutely necessary? Will the software be usable by a normal person (who is willing to install a python package and use a Jupyter Notebook... okay, so not really "normal" but you know).

Testing and Data Validation

• Includes issues #275, #276, #277, #278, and #279
• What level of confidence do we need to have in the finished data products to feel comfortable encouraging people to use them?
• There will always be data quality issues, missing data, etc. We can't wait until it's perfect.
• At the same time, in most cases we do not yet have automated tests that ensure basic expectations are met, beyond simply checking that the data exists, reporting how large it is, and ensuring it satisfies the type and relational constraints of the database schema.
• However, based on our own past internal use of the data to do analysis (e.g. the MCOE calculations), we do have some expectations about what ought to be working, and generally what some of the data ought to look like: reasonable heat rates, capacity factors, fuel costs, etc.
• This means that we can relatively easily change something, mess up the output data, and not realize it until much later, when someone attempts to use the data. Up until now that's generally been us, but after a release, it seems more likely that it will also be other people.
• What do we already believe should to be working in the data?
• How can we codify those expectations into data validation tests?
• Are there any other things that we're not sure of about the data products that we think would be vital to check before publicizing it?

Post-ETL Functionality

• Currently we have an thin output and analysis layer that sits on top of the database, makes the data more immediately usable, and provides some interesting derived values. This especially includes the pudl.output subpackage, and the pudl.analysis.mcoe module.
• We will need to invest some time to ensure that this functionality continues to work after we move away from populating a database, and toward exporting datapackages.
• Does that work need to happen before we make a release? Are we comfortable releasing without those denormalized outputs and without the useful derived values working yet?
• However, currently the Testing and Data Validation (see above) also depend on the pudl.output subpackage. If we do not want to make this post-ETL functionality available and we do want to have the data validation, then additional work will be required to migrate the testing and data validation to using the datapackage output directly, rather than relying on the outputs.

[karldw]

Documentation

• It seems important to document how the data package is installed. What happens after someone types pip install packagename?
• The docs currently populated in readthedocs are only about database tables. Could those be removed, since people will not be using the database? I've always found incorrect docs worse than incomplete ones.
• It would be nice (but not essential for this release) to have user docs and developer docs. Things like the current gettings_started.md mostly apply to developers or contributors, but not people who just want to use the package.

Testing

I'm not sure CEMS should hold up a proper release, but if you want CEMS tests, #260 lists some that should exist but don't.

[zaneselvans]

Hey Karl, thanks for the feedback. Documentation is definitely a big weak point right now, and the doc builds aren't really working right yet. We're spending today doing nothing but docs -- first cleaning up the docstrings, and then hopefully moving on to gluing it all together with some better high level explanations. I certainly need to write up the new setup and testing process for other codey people.

[zaneselvans]

With our being behind schedule, and also releasing a final postgres based version of PUDL as v0.1.0, this has morphed into being two releases in quick succession, encompassing a lot more packaging and documentation and release mechanics than originally expected, but only approximately as much data content as we thought v0.1.0 would have. So we're going to adjust the versions / milestones accordingly.