JuliaData / DataFrames.jl

In-memory tabular data in Julia
https://dataframes.juliadata.org/stable/
Other
1.71k stars 360 forks source link

CartesianIndex error in Julia 1.11 #3412

Closed drizk1 closed 5 months ago

drizk1 commented 5 months ago

Hello, dataframes.jl is wonderful.

The TidierData.jl unnest_longer function is now causing an error on the 1.11 nightly run. which I would like to address

Is there different way to write to the line below to avoid the error and accommodate this behavior change for both 1.8 and 1.11?

Thank you

excerpt of code line causing problem and error:


    if !keep_empty
      df_copy = filter(row -> !any(ismissing, [row[col] for col in column_symbols]), df_copy) 
       ### this line above is where the error is originating.
    end

ERROR: MethodError: no method matching DataFrames.SubDataFrame(::DataFrame, ::CartesianIndex{1}, ::Colon)    
    Closest candidates are:
      DataFrames.SubDataFrame(::DataFrame, !Matched::Colon, ::Any)
       @ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/subdataframe/subdataframe.jl:86
      DataFrames.SubDataFrame(::DataFrame, !Matched::AbstractVector{Bool}, ::Any)
       @ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/subdataframe/subdataframe.jl:98
      DataFrames.SubDataFrame(::DataFrame, !Matched::AbstractVector{Int64}, ::Any)
       @ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/subdataframe/subdataframe.jl:80

    Stacktrace:
      [1] copyto!
        @ ./broadcast.jl:0 [inlined]
      [2] copyto!
        @ Base.Broadcast ./broadcast.jl:924 [inlined]
      [3] copy
        @ Base.Broadcast ./broadcast.jl:896 [inlined]
      [4] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, DataFrames.var"#103#104"{TidierData.var"#77#81"{Vector{String}}}, Tuple{DataFrames.DataFrameRows{DataFrame}}})
        @ Base.Broadcast ./broadcast.jl:871
      [5] _filter_helper(f::Function, cols::DataFrames.DataFrameRows{DataFrame})
        @ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/abstractdataframe/abstractdataframe.jl:1216
      [6] #filter#92
        @ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/abstractdataframe/abstractdataframe.jl:1189 [inlined]
      [7] filter(f::Function, df::DataFrame)
        @ DataFrames ~/.julia/packages/DataFrames/58MUJ/src/abstractdataframe/abstractdataframe.jl:1188
      [8] unnest_longer(df::DataFrame, cols::Vector{Symbol}; indices_include::Nothing, keep_empty::Bool)
        @ TidierData ~/work/TidierData.jl/TidierData.jl/src/nests.jl:117
      [9] macro expansion
        @ ~/work/TidierData.jl/TidierData.jl/src/nests.jl:158 [inlined]
     [10] top-level scope
      @ none:3
bkamins commented 5 months ago

Fixed in https://github.com/JuliaData/DataFrames.jl/pull/3413

drizk1 commented 5 months ago

Incredible how fast that was. Thank you so much!

bkamins commented 5 months ago

For the time being, you can manually add the:

Base.getindex(itr::DataFrames.DataFrameRows, i::CartesianIndex{1}) = itr[i[1]]

line in your code to make things work on nightly.

drizk1 commented 5 months ago

Thank you! I actually just reran the workflow w no changes, and it worked w the original code.