CTHUB - Upload CVP Data

[shayjeff]

Describe the task Add the CVP (Commercial Vehicle Pilots Program) data file to CTHUB.

CVP dataset is now available in the Teams channel (External Zelda > Files > Data_2024 > CVP).

CVP_Analysis_2024-05-17.xlsm

1. Data tab contains CVP dataset and corresponding headers
2. Data_Tab_Format contains descriptions for each field:

• Column Name
• Description
• Type: field type
• Missing values: whether field can contain missing values or not.
• Values: list of possible values. If blank, values can be anything
• Multiple possible: whether several categories can be present in one cell. For instance, a single CVP project could be part of several economic regions or just one (if they are from the list of possible economic regions).
• Error: type of error that should come up
• Warning: type of warning that should come up
• Pre-processing: whether the field should be pre-processed or not.
• Next steps: notes for the future and for record keeping, these may or may not be necessary.

Purpose The purpose of this task is to enable the business area the ability to uploading CVP data by into CTHUB.

Acceptance Criteria

• [ ] CVP data file successfully uploaded to CTHUB.

Development Checklist:

• [ ] New tables to contain CVP data and region data (joined)
• [ ] add cvp to upload dropdown
• [ ] create service for upload
• [ ] keep in mind the region field will come in as a string but will need to be split up if it contains more than one, joined to the region table (get_or_create)

Additional context

The Economic Region field allows multiple values (e.g. Thompson-Okanagan, Nechako, Vancouver Island & Coast) in a single cell from the list of all economic regions.

In addition to that, they also have the Across BC option which appears redundant because you could select all of the regions instead. However, I was asked to keep this category in cases where it is not very clear where the project is taking place.

Eventually, we will have a map with all of the economic regions. Since Across BC is not a region, it would need to be converted into a list containing all of the regions (Vancouver Island/Coast, Mainland/Southwest, Thompson-Okanagan, Kootenay, Cariboo, North Coast, Nechako, Northeast) so that all regions would be displayed on the map. Perhaps, an additional field would need to be created so that Across BC is displayed in the dashboard dropdown but another field with all of the regions is shown on the map.

[katerinkus]

CVP dataset is now available in the Teams channel (External Zelda > Files > Data_2024 > CVP).

CVP_Analysis_2024-05-17.xlsm

1. Data tab contains CVP dataset and corresponding headers
2. Data_Tab_Format contains descriptions for each field:
   • Column Name
   • Description
   • Type: field type
   • Missing values: whether field can contain missing values or not.
   • Values: list of possible values. If blank, values can be anything
   • Multiple possible: whether several categories can be present in one cell. For instance, a single CVP project could be part of several economic regions or just one (if they are from the list of possible economic regions).
   • Error: type of error that should come up
   • Warning: type of warning that should come up
   • Pre-processing: whether the field should be pre-processed or not.
   • Next steps: notes for the future and for record keeping, these may or may not be necessary.

Notes

• If data type is character, it means it can be anything (string and number, just a number, etc.).
• Start and End columns should not contain missing values. However, if they do, users should get a warning not an error (i.e. they should still be able to upload the row with missing Start and End).
• I am currently waiting for Emmanuel (who created this dataset) to get back to me on Start and End because I am fairly certain that Metabase does not understand quarters because they are not an iso8601 date time standard. And if he wants to keep quarters, they should be in 2023-Q1 format for Pandas to understand them.

[emi-hi]

Please add your planning poker estimate with Zenhub @tim738745

[katerinkus]

Hi @tim738745 and @emi-hi! I finally got the a copy with dates instead of quarters. The final copy also also has a new column name. Instead of End Date it is Completion Date. The dataset is in the same folder on teams _CTHUB_/Data_2024/CVP/CVP_Analysis_2024-06-07.xlsm. I will delete the old file (dated 05-17) if that is okay with you.