-
Unfortunately, it looks like there can be some inconsistency with whether the city is included in the line below the subcontractor name. It may be more useful here to use the column widths to get arou…
-
**Khoi Tran's email and GitHub username:**
- khoi.tran@dot.ca.gov
- khoitran510
**Things to add Khoi to...**
- [x] GitHub @britt-allen via ITOps – [ticket link](https://airtable.com/app1Jm2SLJEU3ljBr…
-
I don't see any updates to the current data relay pipeline since sunday. Am I looking in the wrong, place, or is your Airflow down @pingpingxiu-DOT-ca-gov?
-
We need a way to calculate speed in order to calculate multiple other performance metrics which use speed as a variable. In order to do this, we are going to use a simplified calculation that will giv…
-
Now that we are starting to land some incremental models, we should rethink how our CI works. Right now, we are running a `dbt build` for every CI run. This won't work well for incremental models, sin…
-
The current data relay pipeline seems to be only loading VDS data for Caltrans district 7. It seems that the landing servers for D7 and the other districts are different, and we may be only grabbing d…
-
It looks like some extra digits are occasionally being picked up in the City field. Example: Identifier# 01-0C7104_10239, Bidder ID# 3
![image](https://github.com/nesaboz/caltrans_data_extraction/a…
-
Many downstream metrics (e.g., speed) are based on five-minute aggregations of the raw VDS data. Ideally, this would include imputations, but for the time being, we want to aggregate the un-imputed ra…
-
About 10% of our users visit the site on a mobile or tablet device. That might not sound like much, but it amounts to ~7,000 people this year. The quality of site display on smaller screens is pretty …
-
XML configuration files for districts need to be parsed into structured parquet files for load to the data warehouse. Dependent on #75