search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.48k stars 3.52k forks source link

[C#] `Column(string)` method in `RecordBatch` is linear to the number of columns #44501

Open vthemelis opened 4 days ago

vthemelis commented 4 days ago

Describe the enhancement requested

It looks like a column lookup by name is linear to the number of columns. This is not intuitive and can easily lead to performance regressions. Would it be possible to add a lookup to convert this into an O(1) operation?

Component(s)

C#

CurtHagenlocher commented 4 days ago

Column names are not actually required to be unique, which is why Schema.Fields is marked deprecated. If you know that the column names in your data are unique, you could still use it. Otherwise, we'd need something like a mapping of a string onto what's possibly a list of field positions.