nspcc-dev / neofs-node

NeoFS is a decentralized distributed object storage integrated with the Neo blockchain
https://fs.neo.org
GNU General Public License v3.0
32 stars 38 forks source link

Speed up split object streaming #2791

Open carpawell opened 6 months ago

carpawell commented 6 months ago

Is your feature request related to a problem? Please describe.

Node can try to collect more child objects at once for streaming to a client, not one by one.

Describe the solution you'd like

Think and try some alternatives to this code. At least keeping 2 objects may be more effective. While streaming one object to a client, the second one is being fetched. This way, with the same network speed b/w nodes and clients, a client should not wait for a child object to be received from the next-door node. On the other hand, memory consumption will increase too.

Also, streaming objects directly looks fast, and memory independent. Changes to this code can be considered too. But we should track the progress and not stream any byte twice.

Describe alternatives you've considered

Keep it as is. At least, it works and we do not have yet issues about slow object assembling.