search

civic-jabber / data-ingest

Utilities for ingesting data pertaining to state and local political issues
MIT License
0 stars 0 forks source link

Speed up load time for config files #4

Open MthwRobinson opened 3 years ago

MthwRobinson commented 3 years ago

With the addition of news sources from every state, the load time for the YAML config files has become very slow. This (minorly) impacts the run time for the load_news function. More importantly, it's causing the tests to take a while. The task is to either:

  1. Improve the load time for the YAML files
  2. Migrate the news sources configs into Postgres or
  3. Mock the config loads in the tests