zodb / relstorage

A backend for ZODB that stores pickles in a relational database.
Other
54 stars 46 forks source link

Reduce gevent switching during locked parts of TPC. #357

Closed jamadden closed 4 years ago

jamadden commented 4 years ago

Those two items let us implement no-switch mode fully for queries as well, for both the load and store connections.

Also, avoid potentially blocking on the cache locks during this time.

Fixes #339.

Awaiting some benchmarking...

jamadden commented 4 years ago

Benchmarks look good. Adding non-conflicting objects is not impacted. Using 60 greenlets to update 1,000 conflicting objects looks very nice:

60 greenlets/1,000 conlficts Time (ms) CPU Usage
This branch 284 75%
Master (default commit-timeout) Failed Failed
Master (60s commit-timeout) 719 50%

The default settings for master couldn't finish this test: many greenlets died by timing out to lock rows. With an extended timeout, master could complete the test, taking 719ms and never using more than 50% CPU. With these changes, however, we cut that time down to just 284ms and make better use of the CPU.