search

elixir-explorer / explorer

Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir
https://hexdocs.pm/explorer
MIT License
1.12k stars 123 forks source link

Error using is_finite and is_infinite within mutate #906

Closed lostbean closed 6 months ago

lostbean commented 6 months ago

I'm trying to use is_finite but it's failing when used from within mutate or mutate_with. Replacing with is_nan or using the series directly gives no error and works as expected.

Failing:

iex(16)> CvmExplorer.monthly_stats(df) |> Explorer.DataFrame.mutate_with(&[b: Explorer.Series.is_finite(&1["value_share_diff_pct"])])
** (ErlangError) Erlang error: :nif_not_loaded
    :erlang.nif_error(:nif_not_loaded)
    (explorer 0.8.2) lib/explorer/polars_backend/native.ex:458: Explorer.PolarsBackend.Native.err/0
    (explorer 0.8.2) lib/explorer/polars_backend/lazy_frame.ex:465: anonymous fn/3 in Explorer.PolarsBackend.LazyFrame.mutate_with/3
    (elixir 1.15.7) lib/enum.ex:2510: Enum."-reduce/3-lists^foldl/2-0-"/3
    (explorer 0.8.2) lib/explorer/polars_backend/lazy_frame.ex:463: Explorer.PolarsBackend.LazyFrame.mutate_with/3
    (explorer 0.8.2) lib/explorer/polars_backend/data_frame.ex:647: Explorer.PolarsBackend.DataFrame.mutate_with/3
    iex:16: (file)

Working:

iex(16)> CvmExplorer.monthly_stats(df) |> Explorer.DataFrame.mutate_with(&[b: Explorer.Series.is_nan(&1["value_share_diff_pct"])])
#Explorer.DataFrame<
  Polars[254466 x 9]
....
>
iex(17)> (Explorer.Series.is_finite(CvmExplorer.monthly_stats(df)["value_share_diff_pct"]))                    #Explorer.Series<
  Polars[254466]
  boolean [true, true, true, true, true, true, true, true, true, true, true,
   true, true, true, true, true, true, true, true, true, true, true, true, true,
   true, true, true, true, true, true, true, true, true, true, true, true, true,
   true, true, true, true, true, true, true, true, true, true, true, true, true,
   ...]
>
lkarthee commented 6 months ago

It has been fixed in main branch by Jose - https://github.com/elixir-explorer/explorer/commit/b59caa22184805736db55ffd2ba6cac737433af5 . It is not released yet. You can use main until release.