NYCPlanning / data-engineering-qaqc

streamlit app for data engineering
https://edm-data-engineering.nycplanningdigital.com
1 stars 0 forks source link

CPDB: AD feedback on QAQC page #187

Open AmandaDoyle opened 2 years ago

AmandaDoyle commented 2 years ago

Hi all - here is my feedback on the CPDB QAQC page going from top to bottom. Love how the text is dynamic.

Information reported in the Capital Commitment Plan published by the NYC Office of Management and Budget (OMB) three times per year is the foundation that CPDB is built from. Therefore, only capital projects that are in the Capital Commitment Plan are reflected in CPDB. Additional data sources are incorporated to map the capital projects.

CPDB enables Planners to better understand and communicate New York City’s capital project portfolio within and across particular agencies. While not comprehensive, CPDB's spatial data provides a broad understanding of what projects are taking place within a certain area, and is starting point to discovering opportunities for strategic neighborhood planning.


- [x] Update "ABOUT QAQC" to "About the QAQC Reports" and change the text below to the following:

The QAQC page is designed to highlight key measures that can indicate potential data issues in a CPDB build. These graphs report summary statistics at the agency level and there are 3 ways to filter and view the data (w/ additional variation at the graph level):

  1. Agency type: Sponsoring agency OR Managing agency
  2. Aggregation type: the total number of projects OR the total sum ($) of all commitments
  3. Category type: Include projects in all categories (fixed asset, lump sum, ITT, Vehicles & equipment) OR include only projects that are categorized as Fixed Asset

Additionally, there are basic geographic checks to facilitate the QAQC process of the disparate source data we receive from various city agencies. These checks are not meant to be comprehensive, rather they are intended to provide an indication if spatial data is outside of the NYC spatial boundaries or incorrect in some way.


- [x] Add and edit language under "Key CPDB QAQC terms:"
This graph visualizes the difference in the Number of Projects by Managing Agency between the current (aka latest) and the previous version of CPDB. While the underlying Capital Commitment Plan data changes between versions, any drastic changes between CPDB versions that are illustrated by this graph can indicate if there is a specific agency or source dataset to look into further that may have introduced these anomalies.  Anomalies include, but are not limited to, no projects being mapped for a given agency when there were mapped projects in the previous version, the number of projects doubling for an agency between versions, or the total sum of commitments halving for an agency between versions.  This chart also gives the viewer the flexibility to change between all projects by Number of Projects (both mapped and unmapped) along with an option to just view the mapped (geolocated) projects. Click the "Latest Version" and "Previous Version" labels in the legend to display the total Number of Projects for each.