search

nspcc-dev / neofs-node

NeoFS is a decentralized distributed object storage integrated with the Neo blockchain
https://fs.neo.org
GNU General Public License v3.0
31 stars 38 forks source link

Improve GC routine #2866

Open carpawell opened 3 weeks ago

carpawell commented 3 weeks ago

Is your feature request related to a problem? Please describe.

GC may be more intelligent. Currently, it only works with clear wording like "a user deleted an object" or "epoch N happened, so every object whose expiration is <= N-1 should expire too". But sometimes real garbage may appear in the storages (bad migration, shard error, bugs, configuration changes, missed meta change notification etc), and some background routine may try to monitor it.

Describe the solution you'd like

If some object without a container, lost object part, or some other object that should not exist is found, it should be GC-ed. A background routine fits well for this.

Describe alternatives you've considered

Some explicit external command like vacuum in the control service.

Additional context

https://github.com/nspcc-dev/neofs-node/issues/2858 happened and there is a fix for it :https://github.com/nspcc-dev/neofs-node/pull/2863. However, it does not solve the problem fully: if some parts are stored and no parent header is found, it will never be GC-ed now. It should be solved carefully. Situations, when a node believes that there is no object in the network, should be proved.

roman-khimov commented 3 weeks ago

We have replicator that goes over the object list all the time, it can notice inconsistencies, it can invalidate objects (trying to replicate them is an error anyway).