search

deployment-gap-model-education-fund / deployment-gap-model

ETL code for the Deployment Gap Model Education Fund
https://www.deploymentgap.fund/
MIT License
6 stars 2 forks source link

Update offshore wind data #345

Closed bendnorman closed 3 months ago

bendnorman commented 3 months ago

This PR updates the proprietary offshore wind data. To archive the raw data I just downloaded it directly from airtable and dropped it into the gs://dgm-archive. The modeling is mostly the same as the original data but I added an association table for port locations. There are a few other one to many relationships between projects and other attributes by decided to just focus on location columns for now.

TrentonBush commented 3 months ago

Also how did you export the raw CSVs? Did you use somebody else's Airtable view? I'd consider making our own like "Catalyst Export" so other people's changes don't impact us. Plus we can prep it how we want to make normalization easier, if needed.

bendnorman commented 3 months ago

I just downloaded a csv from the "Cleaned View". It seems like data between views stay synced the the columns and layout your choose are specific to the view. So is the benefit of creating our own view that the columns will stay the same between updates? In other words, if they change the "cleaned view" view layout, the "catalyst export" view's layout will stay the same?

TrentonBush commented 3 months ago

is the benefit of creating our own view that the columns will stay the same between updates?

Yep, exactly