nprapps / elections18-general

2018 midterm election back-end: Associated Press data ETL, database, admin panel, and JSON output; iteration upon 2016 GE work
MIT License
3 stars 1 forks source link

Decrease amount written to data-pipeline log in production #35

Open mileswwatkins opened 6 years ago

mileswwatkins commented 6 years ago

Not a worry for this year, but potentially for next: the production fetch_and_publish_results.log can get quite large over time; after a few weeks of running continuously during 2018 testing, it got up to 1.4 GB in size. Maybe handle logging levels more conservatively for production, where disk size is limited and this could create a hard-to-debug issue on election nights?