Update Drought Data

[rafinkanisa]

As part of re-activating the ASDC to humanitarian organization, we'll have to update all of the outdated data in ASDC to the most recent one

TODO :

• [x] Create Script to update the drought data
• [x] Perform drought data update

[dodiws]

• test drought update script in local, using local db
• looking for ways to monitor celery task log to help on debugging since drought script takes long processing time and celery task sometime not working

[mishaker]

@dodiws please include the script you wrote to update the db here in this issue. thanks

[dodiws]

This commit add filter_woys and max_loop param on drought update function to better control update process.

from geodb.tasks import downloadtif_task
downloadtif_task.delay()

That script update latest 1 week drought data. It should be called using celery call but not working, still investigating.

[dodiws]

After tweak celery setting, drought update for latest 1 week can be called using:

./manage.py celery call geodb.tasks.downloadtif_task

The task will be carried on asynchronously by celery worker process therefore ssh connection can be closed. Monitoring the log for progress:

tail -f /home/ubuntu/celery_worker.log

Updating about 1 year of drought data in progress.

[rafinkanisa]

Thank you @dodiws Will keep the issue in the progress stage until the data update is finished

[dodiws]

Finished updating 46 weeks of drought data, the next weekly update will be done manually every Monday because new data is published every Sunday from the data source.