search

scalableminds / webknossos

Visualize, share and annotate your large 3D images online
https://webknossos.org
GNU Affero General Public License v3.0
124 stars 23 forks source link

Atomic Transaction management / Mutex for committing annotation updates #7307

Open fm3 opened 1 year ago

fm3 commented 1 year ago

The annotation update route has nice transaction management, but it relies on the TracingService.handleUpdateGroup method being atomic (so that the version increase caused by committed updates is already available when the next requests arrives).

While an unlikely case, it is theoretically possible to construct a scheduling scenario where multiple conflicting updates are committed with the same version number. In the fossilDB, one will overwrite the other, potentially creating invalid annotation states.

We should guard against this by creating some kind of mutex for this saving (per tracing id) or using the transaction management from rocksDB, and exposing that via the fossilDB api

philippotto commented 1 year ago

Is this a new problem or did it exist before?

fm3 commented 1 year ago

It already existed and we discussed it already previously, but @frcroth raised this concern again during the review of #7290.

Also, committing volume buckets has become much slower due to the segment index, which somewhat increases the risk of this happening.