search

oss-aspen / 8Knot

Dash app in development to serve open source community visualizations using GitHub data from Augur. Hosted app: https://eightknot.osci.io
MIT License
49 stars 60 forks source link

have analysis workers retry rather than wait on query workers #591

Open JamesKunstle opened 9 months ago

JamesKunstle commented 9 months ago

Currently, analysis workers just poll-wait on whether the data that is required has been cached. This consumes a worker thread for an indeterminate amount of time.

Tasks in celery can retry as many times as they like, provided that they have the 'bind' mode enabled. This makes the first argument to the task 'self' and allows the task to use 'self.retry' in application logic.

We could use this to minimize pool exhaustion- if the data isn't available for an analysis worker, just have it retry.

JamesKunstle commented 7 months ago

https://github.com/plotly/dash/issues/2725

JamesKunstle commented 5 months ago

This requirement is blocked waiting on the Plotly Dash maintainers. Essentially, they need to support the issue that I opened (above) that allows tasks to be retried rather than requiring that they poll forever. There's been some interest in this feature but no movement yet. The maintainers said that they would work on it if we paid for that development, which could be an option in the future.

cdolfi commented 2 months ago

@JamesKunstle If I am remembering correctly the necessary change was implemented by plotly?