Add prediction_time to scraping dictionary

[clauslang]

From #23:

• On the highest hierarchy level of the dictionary, i.e. as neighbour to 'city', 'site', 'date', 'daily', 'hourly', the dictionary has to have another entry 'prediction_time', that denotes the time when the html file was downloaded as an int in the format YYYYMMDDHHMM. This should be easy to implement, since this timestamp is included in the file names.

Please drop a quick line here once this is done for your script so that we have an overview.

[clauslang]

Done for owm.

[akresnia]

done for timeanddate

[janfb]

done for wetter.com

[gkBCCN]

My commit closed the issue on GitHub accidentally. My commit only resolves this issue for wetter.de.

[denisalevi]

Just to clarify: We now have multiple files for the same date and city but e.g. downloaded with 12h difference. So I have to handle it myself, that the predictions e.g. at 5am and 5pm end up in different dictionaries that will then individually we loaded into the database (with the only difference to distinguish them being the 'prediction_time' value)?

[clauslang]

Yes. We might in the future decide that only one prediction per day is loaded into the database. I think the reasoning for downloading twice a day was mainly to be safe in case one of the downloads fails. (Even though it is possible that there is an interesting analysis case comparing predictions of the same day at different times for a subset of the providers.)

[clauslang]

@inesw finished it for wunderground

[denisalevi]

I do have the prediction time in my dictionaries, but currently data saved at different hours of the same day (e.g. 5am and 5 pm) end up in the same dictionary. I need to fix that, will do that later today.

[denisalevi]

done for accuweather