search

chriswhong / nyc-civic-jobs

A simple website for browsing NYC government job listings
MIT License
8 stars 1 forks source link

Automate data pipeline #7

Open chriswhong opened 5 years ago

chriswhong commented 5 years ago

Maybe a cron job that runs nightly, to replace the data...

However, the dataset only includes agencies that have current listings, and there is a hard-coded lookup to map the agencyname from the data to a displayname. When data is updated, there could be new agencies that won't get a good displayname. We'll need to take that into account.

chriswhong commented 5 years ago

This is currently underway via nyc-jobs-scraper. It's not automated yet.