search

BCCN-Prog / weather_2016

For the BCCN 2016 advanced programming project
3 stars 1 forks source link

Server: data backup! #12

Closed daveh19 closed 8 years ago

daveh19 commented 8 years ago

Since the scrapers are still very much works in progress maybe we should have a cron job that backs up the data directories every night. It can just dump into a large backup folder somewhere that can be deleted if it gets too big.

erensezener commented 8 years ago

The scrapers don't modify old files. They save new files such that the file name contains the date. So I feel like files are fairly safe.

daveh19 commented 8 years ago

Good. I figured it was like this. But is there any chance that they can accidentally save under a wrong name? They should really test for file existence before saving. Maybe we can make that an issue under scrapers.

daveh19 commented 8 years ago

Issue created for scrapers, closing this.