JuliaData / TableOperations.jl

Common table operations on Tables.jl interface implementations
Other
46 stars 9 forks source link

`TableOperations.transform()` returns lazy table with no schema #32

Open EssamWisam opened 1 year ago

EssamWisam commented 1 year ago

I'm attempting to perform transformations on a table and subsequently convert it into a matrix.

X = DataFrame(rand(10, 2), :auto)             # also tried a matrix table
Xcat = X |> TableOperations.transform(Dict(0=> x -> x)) |> Tables.matrix

However, I get the error

type Nothing has no field types

Stacktrace:
 [1] getproperty(x::Nothing, f::Symbol)
   @ Base [./Base.jl:38](https://file+.vscode-resource.vscode-cdn.net/Users/essam/Documents/GitHub/Imbalance.jl/Base.jl:38)
 [2] matrix(table::TableOperations.Transforms{true, Tables.MatrixTable{Matrix{Float64}}, Dict{Int64, var"#15#16"}}; transpose::Bool)
   @ Tables [~/.julia/packages/Tables/AcRIE/src/matrix.jl:87](https://file+.vscode-resource.vscode-cdn.net/Users/essam/Documents/GitHub/Imbalance.jl/~/.julia/packages/Tables/AcRIE/src/matrix.jl:87)
 [3] matrix(table::TableOperations.Transforms{true, Tables.MatrixTable{Matrix{Float64}}, Dict{Int64, var"#15#16"}})
   @ Tables [~/.julia/packages/Tables/AcRIE/src/matrix.jl:84](https://file+.vscode-resource.vscode-cdn.net/Users/essam/Documents/GitHub/Imbalance.jl/~/.julia/packages/Tables/AcRIE/src/matrix.jl:84)
 [4] |>(x::TableOperations.Transforms{true, Tables.MatrixTable{Matrix{Float64}}, Dict{Int64, var"#15#16"}}, f::typeof(Tables.matrix))
   @ Base [./operators.jl:911](https://file+.vscode-resource.vscode-cdn.net/Users/essam/Documents/GitHub/Imbalance.jl/operators.jl:911)
 [5] top-level scope

The underlying reason as far as I understand is that the following test passes

Xcat = X |> TableOperations.transform(Dict(0=> x -> x)) 
@test isnothing(Tables.schema(Xcat))

Note that we don't run into the same issue if we use TableTransforms.select(...). Also, on the other hands, converting to a columntable works but I am not sure how efficient that operation is if I were to follow it with a matrix conversion.