search

CityOfLosAngeles / aqueduct

A shared pipeline for building ETLs and batch jobs that we run at the City of LA for Data Science Projects. Built on Apache Airflow & Civis Platform
Apache License 2.0
21 stars 6 forks source link

RatStat DAG #143

Open brendanbailey opened 4 years ago

brendanbailey commented 4 years ago

Hey Hunter and Ian

We've launched a new version of RatStat, which should address some of the data entry issues we were facing in the past.

I think we're ready to create a dag to pull the data from AGOL into your data warehouse.

Hunter - I think you already created a dag, but can't seem to find it in the repo. I'll refactor the code, if you forward me the original script if that's helpful. Or I could just give you the feature id. Let me know what works on your end

hunterowens commented 4 years ago

woot!

since we never wanted the data to land, the ratstat dag is still hanging as a pr (#88)

I think we can easily change it to use hte new layers, lmk what those are and I will modify.

brendanbailey commented 4 years ago

Great - the id is 9e7caafab26b490e8ad7f02e5518758c

In previous versions the app tracked both bait traps and rat burrows, and also has associated tables with the layers. In this version it only tracks burrows, and there are no additional tables.

hunterowens commented 4 years ago

@brendanbailey syncing back in on this. Still need to be deployed?

brendanbailey commented 4 years ago

It does, but definitely not priority. I would prioritize the COVID work over this

hunterowens commented 4 years ago

ok, keeping on hold.