Shopify / ghostferry

The swiss army knife of live data migrations
https://shopify.github.io/ghostferry
MIT License
693 stars 65 forks source link

Potentially "overlocking" in cursor? #341

Open SpencerMalone opened 1 year ago

SpencerMalone commented 1 year ago

We get a lot of issues caused by... Error 1213 (40001): Deadlock found when trying to get lock; try restarting transaction when mass migrating data, and I was trying to understand why the select locks are so aggressive.

I'm guessing that the locking in... https://github.com/Shopify/ghostferry/blob/master/cursor.go#L174-L176 is an implementation of around https://github.com/Shopify/ghostferry/blob/master/tlaplus/ghostferry.tla#L332-L354 in the design spec, and I was wondering if there was any reason we couldn't use a FOR SHARE / LOCK IN SHARE MODE in the cursor to reduce the write lock churn when migrating lots of pieces of data at once? This issue is half bug / half feature request / half question, but the IRC seemed pretty empty when I poked at it, so I figured I'd broach the subject here.