Closed denisalevi closed 8 years ago
The question is, do we really want a forecast that far in advance? It's going to blow up our database and run time for scraping data integration. Maybe we should only hand as many days to the DB as the site with the smallest forecasting period has (as only this is relevant for comparative analysis - or are there other analysis goals for which we would need that much data?). I think that would be 10, which also corresponds to the example dictionary in the wiki.
Let's discuss this in the next session. But yeah, if we restrict the number of days of prediction, this should probably happen on the DB side when integrating.
Accuweather downloads html_pages with data up to 90 days in the future. The
test_scraper_output.py
raises anAssertionError
if the daily dictionary keys are>=20
. Please fix that to>=90
or make an exception for accuweather. link to code