influxdata / flux

Flux is a lightweight scripting language for querying databases (like InfluxDB) and working with data. It's part of InfluxDB 1.7 and 2.0, but can be run independently of those.
https://influxdata.com
MIT License
769 stars 153 forks source link

feat(array): less leaky string array #5483

Closed mhilton closed 6 months ago

mhilton commented 6 months ago

Change the behviour of the string array back to the old behaviour where accessing the Value function returns a string that is backed by the arrow memory buffer. This avoids data allocations to memory outside of the memory allocator.

The implementation of array.String has been simplified somewhat as part of the new behaviour.

There are a number of places where correct behviour relies on copies of the data being made. To avoid having to fix all of these in the same PR a temporary ValueCopy function has been added to maintain the old semantics. This is being used everywhere the Value function was previously, except for cases where the value is obviously immediately processed, then discarded.

The cases where the VisitCopy function is being used will be address one at a time until we can avoid significant levels of unaccounted memory.

Checklist

Dear Author :wave:, the following checks should be completed (or explicitly dismissed) before merging.

Dear Reviewer(s) :wave:, you are responsible (among others) for ensuring the completeness and quality of the above before approval.