`dcpy` - add statistics and/or data viz

[damonmcc]

questions for scoping

• which information/stats would be useful to have at the end of multiple data products?
  • as opposed to logic that's so product-specific it would better off in a product's folder
• how should the stats be preserved?
  • github action logs expire after 400 days and aren't "machine readable" aka can't be queried or shared easily
  • csvs and images exported with build outputs?
  • objects to be used in a page of the QAQC app?

[athursland]

Ideas for information/stats relevant to many data products

• Basic descriptive information about the dataset: the number of cols in a dataset + a list of those cols and their datatypes, the number of rows in the dataset, and summary statistics about any numeric columns
• If there are categorical columns, summary statistics about numeric columns when grouping by each of the category values. For example, final_category in db-checkbook or typecatego in db-cpdb
• If there are categorical columns, size of each group after groupby
• Length of time it took to run the most recent build, plus a timestamp of the most recent build
• Number of data sources for the data product
• Zonal descriptive information: similar to grouping by categorical values, but with a focus on borough, community district, other location-based zones

[athursland]

Notes from brainstorming:

• Functions that don't need to know anything about the specific data product
• Every data product outputs one table at the end, at least - a primary table and possibly some child/derivative tables
• Facilities, ZTL are good as reference
• This issue can serve as a continuous discussion + brainstorming thread, will open a more specific issue for adding basic descriptive stat fns