SharedCode / sop

Horizontally Scaleable Objects Persistence
http://sop.codeplex.com
MIT License
13 stars 4 forks source link

Known Issue: Partially completed commit can leave unused Node(s) behind #34

Closed grecinto closed 7 months ago

grecinto commented 8 months ago

Discussed in https://github.com/SharedCode/sop/discussions/8

Originally posted by **grecinto** January 21, 2024 During application crash and SOP transaction(s) are in the middle of "commit", temporary Node(s) used by these transaction(s) can be left orphaned. These Nodes' references are recorded in the registry as "inactive" ID(s). They can be cleaned up by SOP on succeeding transaction(s) involving their "active" Node(s). BUT in order not to impact I/O or performance of the system, they were decided to be left as such. It is anticipated that occurrence of this kind of partially completed transactions will be very scarce to zero, thus, it will not impact storage efficiency in time. The other approach known, is to issue a "delete" on these Node(s) part of future transactions, thus, doing an auto-cleanup. However, these are expensive operations and potentially be doing nothing, as most leftover unused Node(s) will be serviced by the "Delete Service" or the SOP commit's cleanup phase. (see here where the auto-cleanup for this use-case was being done, was commented out: https://github.com/SharedCode/sop/blob/c6d8a1716b1ab7550df7e1d57503fdb7e041f00f/in_red_ck/node_repository.go#L288) If in case your setup produces many occurrences of this kind of transactions, pls. feel free to report it(together with repro steps, as usual). Or, if you have a suggestion how to better handle this scenario, and how to introduce a cleanup task for such kind of Node(s), pls. do suggest for discussion. Thanks & kind regards, - SOP maintainer
grecinto commented 8 months ago

Being addressed & latest code changes for this are in "value data segment" branch.

grecinto commented 7 months ago

Fixed in latest checkin in master branch.