Create and optimize Analysis queue handling to avoid db deadlock

kartoza / geocris-inasafe-fba

Next generation of InaSAFE that includes support for forecast based action

MIT License

1 stars 1 forks source link

Create and optimize Analysis queue handling to avoid db deadlock #26

Open lucernae opened 4 years ago

lucernae commented 4 years ago

Our previous solution that uses cronjob to queue analysis seems to be deadlocking. We decided to extract the logic and put it outside the db as Django backend/celery job.

lucernae commented 4 years ago

Add integration tests with the following requirements:

Should be able to be run manually (command can be executed in a Makefile)
Consists of creating 30 hazard_event_queue in subsequent date
Analysis queue should be able to handle the queue even with high frequency (1 minutes of periodic check even when the analysis has not been completed yet)
Record average of analysis processing time and total time.