search

matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.88k stars 2.65k forks source link

Concurrency lock for scheduled tasks #17868

Closed sgiehl closed 9 months ago

sgiehl commented 3 years ago

Currently a scheduled task can be triggered multiple times in parallel. As this can cause serious issues for tasks that manipulate data, we should provide some kind of concurrency lock for scheduled tasks to prevent that by default.

Simplest solution might be to create a Concurrency/Lock for each Task that is running. We could maybe use a default TTL of one hour, but allow each task to provide a custom TTL for that, so a longer running task can define a longer one. This approach for sure wouldn't prevent concurrency for tasks that might run longer than the defined TTL. But it would at least prevent tasks being triggered multiple times.

To really avoid concurrency guess we would need to implement a more complex solution that really checks if a Task might be running in another process, but that might not be needed for now.

Note: there should also be a possibility to allow a task to be executed in parallel (maybe by setting the TTL to null or 0)

tsteur commented 11 months ago

We've had an issue where we received the same alert email 8 times. refs https://github.com/matomo-org/plugin-CustomAlerts/issues/155

refs https://github.com/matomo-org/matomo/issues/19287