treeverse / lakeFS

lakeFS - Data version control for your data lake | Git for data
https://docs.lakefs.io
Apache License 2.0
4.46k stars 359 forks source link

Make update object metadata API atomic #8264

Closed arielshaqed closed 1 month ago

arielshaqed commented 1 month ago

Protect updating object metadata on an existing entry to be safe against concurrent modifications, deletions, and commits and merges to the branch. This is a bit tricky because it needs to work for both committed and staged objects.

Fixes #8262 - a race that would give an odd result if update object metadata managed to lose against concurrent delete and uncommitted GC, or potentially merges into the branch.

github-actions[bot] commented 1 month ago

E2E Test Results - Quickstart

11 passed
github-actions[bot] commented 1 month ago

E2E Test Results - DynamoDB Local - Local Block Adapter

13 passed
github-actions[bot] commented 1 month ago

🎊 PR Preview bb6c44428e67ed789e6fca8a960063a4bb4fd7e0 has been successfully built and deployed to https://treeverse-lakeFS-preview-pr-8264.surge.sh

🕐 Build time: 0.01s

🤖 By surge-preview

arielshaqed commented 1 month ago

Reviewers: I know that tests fail. Please review anyway, we will likely need >1 round -- and these will be easy enough to fix, and not in the areas modified in this PR.

arielshaqed commented 1 month ago

LookED good to Nir 😭

N-o-Z commented 1 month ago

LookED good to Nir 😭

I'll review it today 😃

arielshaqed commented 1 month ago

ObMeme: scumbag steve meme: Writes fix for rare race condition.  Is late to every meeting.