search

lancedb / lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..
https://lancedb.github.io/lance/
Apache License 2.0
3.65k stars 195 forks source link

feat: pyarrow FixedSizeListArray __repr__ does not recognize extension types, such as bf16 #1751

Open eddyxu opened 7 months ago

eddyxu commented 7 months ago

For example, 


from lance.arrow import bfloat16_array
import pyarrow as pa

arr = bfloat16_array([1.1, 2.1, 3.4])
fsl = pa.FixedSizeListArray.from_arrays(arr, 3)

print(fsl)

we got

[
  [
    8D3F,
    0640,
    5A40
  ]
]
wjones127 commented 7 months ago

See also https://github.com/apache/arrow/issues/36648