elixir-explorer / explorer

Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir
https://hexdocs.pm/explorer
MIT License
1.13k stars 123 forks source link

Panic for list of lists with `nil` entries #976

Closed maennchen closed 2 months ago

maennchen commented 2 months ago

Code

Explorer.DataFrame.new(
  [
    %{addresses: nil},
    %{
      addresses: [
        %{lines: ["Nadarzyn Mszczonowska 24"]},
        %{lines: ["Nadarzyn Mszczonowska 24"]}
      ]
    },
    %{addresses: [%{lines: nil}, %{lines: nil}]}
  ],
  dtypes: [{"addresses", {:list, {:struct, [{"lines", {:list, :string}}]}}}]
)

Expected

Either an error explaining that the lists can't be nil or the following:

#Explorer.DataFrame<
  Polars[3 x 1]
  addresses list[struct[1]] [
    [],
    [%{"lines" => ["Nadarzyn Mszczonowska 24"]}, %{"lines" => [...]}],
    [%{"lines" => []}, %{lines: []}]
  ]
>

Actual

thread '<unnamed>' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-core-0.42.0/src/named_from.rs:170:56:
called `Result::unwrap()` on an `Err` value: ComputeError(ErrString("dtypes don't match, got struct[1], expected: struct[1]"))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
** (ArgumentError) cannot create series "addresses": Erlang error: :nif_panicked
    (explorer 0.9.2) lib/explorer/polars_backend/data_frame.ex:578: Explorer.PolarsBackend.DataFrame.series_from_list!/3
    (explorer 0.9.2) lib/explorer/polars_backend/data_frame.ex:529: anonymous fn/3 in Explorer.PolarsBackend.DataFrame.from_tabular/2
    (elixir 1.17.2) lib/enum.ex:1703: Enum."-map/2-lists^map/1-1-"/2
    (explorer 0.9.2) lib/explorer/polars_backend/data_frame.ex:523: Explorer.PolarsBackend.DataFrame.from_tabular/2
    temp/trigger.exs:35: (file)

Runtime

Potentially related to #857

billylanchantin commented 2 months ago

Thanks for the report, @maennchen! We just merged a fix for this issue to main. Feel free to test it out now or wait until the next release.