search

nspcc-dev / neofs-sdk-go

Go implementation of NeoFS SDK
Apache License 2.0
6 stars 14 forks source link

Support custom memory allocators in object slicer #544

Open cthulhu-rider opened 10 months ago

cthulhu-rider commented 10 months ago

object slicer allocates byte buffers of certain size using Go built-in make function. From one side, it's normal default behavior for the library functionality. At the same time, some applications may desire to manage memory allocations deeper in order to improve app's performance

it's proposed to define app-level memory allocator (interface) and support it by the slicer as option. If set, this allocator is be used for processing, otherwise, slicer defaults to make

additionally, this approach will allow to get rid of https://github.com/nspcc-dev/neofs-sdk-go/blob/1db2fbf3c1ad047d1ab3dd74d7c3a930856cf0f0/object/slicer/options.go#L58-L60 that requires buffer to be of size MaxObjectSize (64MB).

roman-khimov commented 10 months ago

Usually if you need a custom allocator you're doing something wrong, it'll be complicated and buggy. Pools/buffers should be sufficient for this.

cthulhu-rider commented 10 months ago

Pools/buffers should be sufficient for this

these are custom allocators to me