search

353solutions / carrow

Go wrapper for Apache Arrow C++
https://arrow.apache.org/
BSD 3-Clause "New" or "Revised" License
15 stars 0 forks source link

Buffered Append #41

Closed tebeka closed 4 years ago

tebeka commented 5 years ago

Currently we have Append calling the C append function on every value. I suggest we'll have a "buffered append" where append will add to existing Go array and only when this array is full will call the right AppendValues C++ method.

tebeka commented 5 years ago

For some reason I don't see a speedup when doing this. @yonidavidson - Can you run the benchmark and see what it does for you?

Run go test -v -run '^$' -bench . in #42, then comment out the special Finish for int array builder and run again.

tebeka commented 5 years ago

The benchmark was wrong! The buffered code clocks 12.6ns/op vs 92.9ns/op for the unbuffered one, ~7 times faster.

Will add tests and buffer the rest of the types.