paleolimbot / nanoarrow

Helpers for Arrow C Data & Arrow C Stream interfaces
5 stars 0 forks source link

Implement growable buffers #9

Closed paleolimbot closed 2 years ago

paleolimbot commented 2 years ago

This PR implements a struct ArrowBuffer, which is a mutable buffer with a reference to its allocator. It is loosely based on Arrow's Buffer and TensorFlows's C API TF_Buffer ( https://github.com/tensorflow/tensorflow/blob/master/tensorflow/c/c_api.h#L102-L130 ). Because it holds a reference to its allocator, it can be grown, shrunk, and freed. The anticipated use is:

// stack allocate
struct ArrowBuffer buffer;
ArrowBufferInit(&buffer);

// write to buffer->data, maybe using helpers
ArrowBufferWriteChecked(&buffer, "1234567890", 10)
ArrowBufferWriteChecked(&buffer, "12", 2)

// shrink to final size
ArrowBufferReallocate(&buffer, 12);

// transfer responsibility to another allocator
// (I'm anticipating that internally allocated struct ArrowArrays will have exactly
// one allocator for all their buffers...most of the time this call will do nothing
// because the allocators will be the same)
ArrowBufferSetAllocator(&buffer, ArrowBufferAllocatorDefault());

// release
ArrowBufferRelease(&buffer);
codecov-commenter commented 2 years ago

Codecov Report

Merging #9 (bfcfc38) into main (3cb6063) will increase coverage by 0.47%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main       #9      +/-   ##
==========================================
+ Coverage   93.40%   93.87%   +0.47%     
==========================================
  Files           5        6       +1     
  Lines         637      686      +49     
  Branches       21       23       +2     
==========================================
+ Hits          595      644      +49     
  Misses         27       27              
  Partials       15       15              
Impacted Files Coverage Δ
src/nanoarrow/buffer.c 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3cb6063...bfcfc38. Read the comment docs.