nspcc-dev / neofs-node

NeoFS is a decentralized distributed object storage integrated with the Neo blockchain
https://fs.neo.org
GNU General Public License v3.0
32 stars 38 forks source link

Revise metabase's indexes #2989

Open carpawell opened 2 days ago

carpawell commented 2 days ago

Is your feature request related to a problem? Please describe.

I'm always frustrated when I receive questions about metabase I cannot answer. Sometimes such questions make sense. Also, it is a fact that most of the indexes were created many years ago and then just extended when needed. We have added LOCKs, reworked big objects, added TREE service, and decided to drop it but our indexes are the same. The last straw was the requirement to unmarshal an object when trying to delete it, and if is not possible, it is a deadlock: #2971 (you cannot clear the full object info if you do not unmarshal it, you just do not have reverse OID -> attributes index)

Describe the solution you'd like

Just look at indexes, try to see the whole picture, and see if we can drop some, or if we need to add some. E.g. it seems like currently, GT/LT search filters do take a lot of resources.

Describe alternatives you've considered

In fact, it works now nothing can be done but it feels like a matter of time.

Additional context

Current state: https://github.com/nspcc-dev/neofs-node/blob/master/pkg/local_object_storage/metabase/VERSION.md.

roman-khimov commented 2 days ago

you cannot clear the full object info if you do not unmarshal it, you just do not have reverse OID -> attributes index

Well, it's a problem. Functionally, performance-wise. Other things can be less important, like #2757.