Mixed and sparse tensor attributes are not releasing memory back to the OS when downsized

[geirst]

In mixed and sparse tensor attributes we are heap allocating the memory needed for each tensor value. These allocations are managed by vespamalloc. When such tensor attribute is downsized (e.g. when migrating data as part of adding more node resources) the memory is released by the attribute, but vespamalloc is not releasing the excess memory back to the OS. It is kept for later reuse. This is a problem if we have reached feed blocked and are adding more node resources to reduce the memory usage on each node. Currently, a restart of the content node is required to release the excess memory back to the OS.

A solution for this is to allocate the cell data for a tensor value in a datastore, which has memory management of large mmapped buffers. This requires a rewrite of StreamedValueStore used by SerializedFastValueAttribute (representing the tensor attribute). Example mixed tensor:

field my_mixed_tensor type tensor<float>(x{}, y[256]) {
    indexing: attribute
}

[geirst]

This is fixed as part of solving https://github.com/vespa-engine/vespa/issues/21962. The new TensorBufferStore stores the tensor data in an underlying ArrayStore (datastore), with separate buffer types for different number of subspaces. For tensors with 1000 is more subspaces the buffer data is heap allocated.