search

elixir-explorer / explorer

Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir
https://hexdocs.pm/explorer
MIT License
1.12k stars 123 forks source link

fix - cannot encode value Binary to term #995

Closed ceyhunkerti closed 1 month ago

ceyhunkerti commented 1 month ago

fixes https://github.com/elixir-explorer/explorer/issues/994

import polars as pl
df = pl.DataFrame({"image": [b'\x01\x02\x03']})
df.write_parquet("ok.parquet")
df = pl.DataFrame({"image": [{"bytes": b'\x01\x02\x03'}]})
df.write_parquet("bad.parquet")
iex(1)> Explorer.DataFrame.from_parquet!("bad.parquet")
#Explorer.DataFrame<
  Polars[1 x 1]
  image struct[1] [%{"bytes" => [1, 2, 3]}]
>
josevalim commented 1 month ago

Looks great! Can you please add a test as well? :) You should be able to write the parquet file using a binary series.

philss commented 1 month ago

Thanks for the PR, @ceyhunkerti! I made a fix for that in #996 returning the binary series , so I'm closing this one.