paleolimbot / nanoarrow

Helpers for Arrow C Data & Arrow C Stream interfaces
4 stars 0 forks source link

Implement bitmap helpers #14

Open paleolimbot opened 2 years ago

paleolimbot commented 2 years ago

Dealing with bitmaps is hard and we should provide some tools to do the finicky hard-to-get-right math:

I think we also need a builder dedicated to bitmaps since building them up incrementally (without allocating if there are no null) is much of the use case. Something like

struct ArrowBitmapBuilder { struct ArrowBuffer buffer; int64_t n_values; uint8_t pending_values[64]; int8_t pending_values_offset; }
ArrowBitmapBuilderAppendInt8(struct ArrowBitmapBuilder* bitmap_builder, const int8_t* values, int64_t n_values);
ArrowBitmapBuilderAppendInt32(struct ArrowBitmapBuilder* bitmap_builder, const int32_t* values, int64_t n_values);
ArrowBitmapBuilderReset(struct ArrowBitmapBuilder* bitmap_builder);

(because char and int arrays are common ways that these things are stored)

lidavidm commented 2 years ago

CountNull needs an explicit length right?