Unsigned indices in dictionaries give error

lancedb / lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

https://lancedb.github.io/lance/

Apache License 2.0

3.96k stars 220 forks source link

Unsigned indices in dictionaries give error #142

Closed changhiskhan closed 2 years ago

changhiskhan commented 2 years ago

repro using oxford_pet:

ds.column("class").to_numpy()

Raises exception:

ArrowTypeError: Converting unsigned dictionary indices to pandas not yet supported, index type: uint8

changhiskhan commented 2 years ago

this won't be supported until we make a corresponding pandas extension dtype