search

nspcc-dev / neofs-node

NeoFS is a decentralized distributed object storage integrated with the Neo blockchain
https://fs.neo.org
GNU General Public License v3.0
31 stars 38 forks source link

Rework lists stored as values in metabase #2808

Open roman-khimov opened 2 months ago

roman-khimov commented 2 months ago

Is your feature request related to a problem? Please describe.

I'm always frustrated when I see object lists stored in the metabase as values. This means encoding/decoding with associated memory bloat for huge lists. Not a problem if we have tens/hundreds of values, can be a problem with hundred thousands of values.

Describe the solution you'd like

Rework them into a set of keys. Can be a batch, can be a key per item, depending on specific case (needs to be evaluated).

Describe alternatives you've considered

Leaving things as is, works for small lists.

Additional context

2807.

carpawell commented 2 months ago

Additional context https://github.com/nspcc-dev/neofs-node/pull/2807.

Wrong context? This PR just did how it is done for every list in the metabase now. This is not the only case.

Rework them into a set of keys. Can be a batch, can be a key per item

What does it mean? I can think of bbolt bucket. But I am also interested in what you meant.

roman-khimov commented 2 months ago

Wrong context? This PR just did how it is done for every list in the metabase now. This is not the only case.

I know. That's why it's an additional issue.

carpawell commented 2 months ago

I just meant that a side dev can see this issue and fix just a single list since that is the context of the issue.