catalyst-cooperative / pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
https://catalyst.coop/pudl
MIT License
468 stars 107 forks source link

Port existing ETL logic to Dagster #2246

Closed jdangerx closed 1 year ago

jdangerx commented 1 year ago

Scope

# Punchlist for merging into dev
* [ ] #2303 
* [ ] #1874 
* [ ] #2223 
* [ ] #2309 
* [ ] #2272 
* [ ] #2301 
* [ ] #1873 
* [ ] #2296 
* [ ] #2286 
* [ ] #2292
* [ ] #2300 
* [ ] #2322 
* [ ] #2318 
* [ ] #2352 
* [ ] #2368 
* [ ] #2294 
* [ ] #2295 
* [ ] #2306
* [ ] #2396 
* [ ] #1872 
* [ ] #2356 
* [ ] #2419 
* [x] Merge pre-Dagster dev into main
* [x] Tag a final pre-Dagster commit/release
* [x] Notify key outside contributors
* [x] Merge #2104 into `dev`
* [x] Do a demo / walkthrough internally
* [x] Merge `dev` into existing working branches
* [x] Publicize the change and hold office hours
bendnorman commented 1 year ago

I think this is the remaining work to get dagster-asset-etl merged into dev:

zaneselvans commented 1 year ago

I think this is basically done and could be closed. Maybe we should post an announcement in GitHub Discussions and do some Tweetes.