search

acquia / blt

Acquia's toolset for automating Drupal 8 and 9 development, testing, and deployment.
https://docs.acquia.com/blt/
GNU General Public License v2.0
442 stars 394 forks source link

Memcache locking and stampede protection #3050

Closed danepowell closed 5 years ago

danepowell commented 6 years ago

I think we should discuss the memcache locking and stampede protection that BLT provides by default.

For at least some customers with certain traffic patterns (lots of concurrent auth users), we have evidence that memcache locking and stampede protection can have a significant negative performance impact.

At the very least, we should make it easier for these customers to disable locking and stampede protection, and document how to do this. Stampede protection is easy to disable by just setting $settings['memcache']['stampede_protection'] = FALSE;, but locking I'm not 100% sure how to remove, since you not only have to remove the configuration in acquia/blt/settings/memcache.yml, but probably have to modify $settings['bootstrap_container_definition'] as well.

Going one step further: should we disable locking and stampede protection for all customers by default? Do we have any data on whether most users will benefit from those settings or not?

lcatlett commented 6 years ago

I think memcache settings being in BLT should only be considered a workaround since it will soon be the default/baseline memcache config on Acquia Cloud, in docs, and updated in the the memcache module README. There are some updates in progress regarding locking that are considered release blockers; benchmarks as well as tests for both stampede protection and locking are part of unblocking that work so more data should be available soon.

The new Acquia / formerly BLT settings are intended to cover 80-90% of the customer use case so any changes might be a better fit for Acquia docs if the settings are unique to Acquia or just the memcache module for general recommendations. There is now a code snippit section maintained in Acquia docs so maybe that could be pulled into blt to make things easier (eg https://github.com/acquia/docs/blob/master/docs/_acquia-code-examples/acquia-cloud/cloud-memcache-d8.php). I remember git submodules being a pain, so i'm not sure of the simplest way to pull in files from a private repo.

bkosborne commented 5 years ago

I was about to comment that BLT should not enable stampede protection without also having core use memcache for the lock backends, because that was (at least at some point) the recommendation from the memcache module. However, I see that Memcache module has actually removed the stampede protection entirely due to bugs.

So I guess BLT should remove the line that enables stampede protection, because it does nothing.