empty ByRow test failure in Julia_1.11.0-DEV.1238

[George9000]

Under test/select.jl, several of the empty ByRow tests fail between lines 1579 and 1610.

Errors

``` empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1579 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1579 =# [1, "1"] end)) => AsTable)) == DataFrame() Evaluated: 0×2 DataFrame Row │ ref size │ GenericM… Tuple… ─────┴─────────────────── == 0×0 DataFrame Stacktrace: [1] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined] [2] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1579 [inlined] [3] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [4] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1580 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1580 =# [1, "1"] end)) => [:p, :q])) Expected: ArgumentError No exception thrown Stacktrace: [1] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1580 [inlined] [2] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [3] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1579 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1579 =# [1, "1"] end)) => AsTable)) == DataFrame() Evaluated: 0×2 DataFrame Row │ ref size │ GenericM… Tuple… ─────┴─────────────────── == 0×0 DataFrame Stacktrace: [1] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined] [2] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1579 [inlined] [3] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [4] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1580 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1580 =# [1, "1"] end)) => [:p, :q])) Expected: ArgumentError No exception thrown Stacktrace: [1] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1580 [inlined] [2] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [3] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1609 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1609 =# [1, "1"] end)) => AsTable)) == DataFrame() Evaluated: 0×2 DataFrame Row │ ref size │ GenericM… Tuple… ─────┴─────────────────── == 0×0 DataFrame Stacktrace: [1] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined] [2] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1609 [inlined] [3] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [4] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1610 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1610 =# [1, "1"] end)) => [:p, :q])) Expected: ArgumentError No exception thrown Stacktrace: [1] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1610 [inlined] [2] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [3] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1609 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1609 =# [1, "1"] end)) => AsTable)) == DataFrame() Evaluated: 0×2 DataFrame Row │ ref size │ GenericM… Tuple… ─────┴─────────────────── == 0×0 DataFrame Stacktrace: [1] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined] [2] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1609 [inlined] [3] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [4] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 empty ByRow: Test Failed at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1610 Expression: select(df, [] => (ByRow((()->begin #= /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1610 =# [1, "1"] end)) => [:p, :q])) Expected: ArgumentError No exception thrown Stacktrace: [1] macro expansion @ ~/Documents/julia/dev/DataFrames/test/select.jl:1610 [inlined] [2] macro expansion @ ~/applications/julia11/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1598 [inlined] [3] top-level scope @ ~/Documents/julia/dev/DataFrames/test/select.jl:1555 Test Summary: | Pass Fail Total Time empty ByRow | 46 8 54 1.5s FAILED: select.jl LoadError: Some tests did not pass: 46 passed, 8 failed, 0 errored, 0 broken. in expression starting at /Users/foo/Documents/julia/dev/DataFrames/test/select.jl:1 ```

versioninfo and full test results

[bkamins]

Thank you for reporting. I will look into it.

[bkamins]

I have investigated the issue. It is related to Tables.jl not DataFrames.jl.

The problematic line is:

elseif isconcretetype(WT) && fieldcount(WT) > 0

in fallbacks.jl (line 216).

The change in Julia that causes this is as follows.

On Julia 1.10:

julia> fieldcount(Vector{Any})
0

julia> fieldnames(Vector{Any})
()

On Julia Version 1.11.0-DEV.1346:

julia> fieldcount(Vector{Any})
2

julia> fieldnames(Vector{Any})
(:ref, :size)

I have asked on Slack if the change is intentional or not.

[bkamins]

CC @quinnj - as we need to decide what to do with this.

[bkamins]

The change leading to this problem is https://github.com/JuliaLang/julia/pull/51319

[bkamins]

In general probably we shoudl special case AbstractArray (and document it. As currently we have (Julia 1.10):

julia> Tables.columns(typeof(@view [1,2][:])[])
Tables.CopiedColumns{@NamedTuple{parent::Vector{Vector{Int64}}, indices::Vector{Tuple{Base.Slice{Base.OneTo{Int64}}}}, offset1::Vector{Int64}, stride1::Vector{Int64}}} with 0 rows, 4 columns, and schema:
 :parent   Vector{Int64} (alias for Array{Int64, 1})
 :indices  Tuple{Base.Slice{Base.OneTo{Int64}}}
 :offset1  Int64
 :stride1  Int64

julia> DataFrame(Tables.columns([@view [1,2][:]]))
1×4 DataFrame
 Row │ parent  indices                       offset1  stride1
     │ Array…  Tuple…                        Int64    Int64
─────┼────────────────────────────────────────────────────────
   1 │ [1, 2]  (Base.Slice(Base.OneTo(2)),)        0        1

julia> DataFrame(Tables.columns([[1,2]])) # this is expected
ERROR: ArgumentError: 'Vector{Vector{Int64}}' iterates 'Vector{Int64}' values, which doesn't satisfy the Tables.jl `AbstractRow` interface

[bkamins]

Also, a basic case that is probably unwanted:

julia> Tables.columntable(Vector{Any}[])
(ref = MemoryRef{Any}[], size = Tuple{Int64}[])