matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.71k stars 2.62k forks source link

Create re-usable interprocess locking utility #12712

Open diosmosis opened 6 years ago

diosmosis commented 6 years ago

Dealing with concurrency occurs a lot when working on Matomo, but there are no re-usable, proven implementations. We should add a locking mechanism that satisfies the following conditions:

tsteur commented 6 years ago

@diosmosis just looking at https://github.com/matomo-org/matomo/blob/3.4.0/core/CronArchive/SharedSiteIds.php#L127 I notice the runExclusive is actually not that exclusive because it is only exclusive per server but when different servers try to execute, this would be still allowed even though they shouldn't.

For this we would as well need a lock in the DB eg a Memory table. However, by pid wouldn't work here (like we did in the other plugin) as it may be executed from various servers and the pid wouldn't be existing on another server. It is nothing of big importance right now but needs to be looked at when doing this change.

diosmosis commented 6 years ago

That's why I called it an 'InterprocessLock', won't work between servers. I think the queued tracking locking mechanism you wrote for the mysql backend would work for this use case? Could put that into a 'InterServerLock' class or something w/ a better name.

tsteur commented 6 years ago

Yes I think that one should work across servers.

tsteur commented 6 years ago

If someone needs this again please ping me and I can maybe provide some code for a LocalLock (lock per server) and a GlobalLock (lock across servers)