xiaodaigh / JDF.jl

Julia DataFrames serialization format
MIT License
88 stars 9 forks source link

type_compress!() no longer works #56

Closed claforte closed 3 years ago

claforte commented 3 years ago

Hi, first of all, congratulations on the birth of your daughter, I hope your family is all safe and healthy. And thanks for this great JDF package. :-)

Using JDF 0.3.0 and 0.2.2, I can't get type_compress! to work. I'm running some of my old code that I'm 90% sure used to work. Everytime I call type type_compress! on a DataFrame I get this error message:

UndefVarError: compress not defined
 Stacktrace:
   [1] type_compress(::CategoricalArrays.CategoricalArray{Int64,1,UInt16,Int64,CategoricalArrays.CategoricalValue{Int64,UInt16},Union{}}) at E:\develop\julia\depot\packages\JDF\JhtJ5\src\type_compress.jl:182
   [2] type_compress!(::DataFrame; compress_float::Bool, verbose::Bool) at E:\develop\julia\depot\packages\JDF\JhtJ5\src\type_compress.jl:24
   [3] type_compress! at E:\develop\julia\depot\packages\JDF\JhtJ5\src\type_compress.jl:19 [inlined]
   [4] load_join_csvs(::Array{FilePathsBase.WindowsPath,1}; add_row::Bool, join_on::Array{String,1}, ignore_cols::Array{String,1}, enable_compress::Bool) at e:\develop\julia\depot\dev\Mperf\src\load_pal.jl:52

I'm not a Julia expert, maybe I'm doing something dumb, but I tried to run your package's Tests (using dev JDF, ] test). There were no error, but then I noticed that most (all?) of the tests that use type_compress! aren't run by default (I guess they take too long and they require access to your C:\data folder).

Thank you!

xiaodaigh commented 3 years ago

Let me check what's happening. Can't remember if it was moved to Dataconvenience.jl or not.

claforte commented 3 years ago

Thanks! I fixed the problem in my local fork with CategoricalArrays.compress instead of just compress. I think it might be confused by Blosc.compress?