xiaodaigh / JDF.jl

Julia DataFrames serialization format
MIT License
88 stars 9 forks source link

DateTime vector with missing entries #72

Closed chipkent closed 2 years ago

chipkent commented 2 years ago

Issue #62 indicated a problem with Date entries. Unfortunately, DateTime is still broken as of v0.4.4.

The problem was first reported at: https://github.com/chipkent/DataFrameTools.jl/issues/14

The problem can be reproduced by running docker run -it julia:1.6:

using Pkg
Pkg.add("DataFrames")
using DataFrames
Pkg.add("Dates")
using Dates

df = DataFrame()
df[!, :test] = [DateTime(2000,1,1,1,1,1), missing]
df

Pkg.add("JDF")
using JDF
JDF.savejdf("test.jdf", df)

Error:

   nested task error: Abstract type AbstractTime does not have a definite size.
    Stacktrace:
      [1] sizeof
        @ ./essentials.jl:455 [inlined]
      [2] compress!(dest::Vector{UInt8}, src::Ptr{Dates.AbstractTime}, src_size::Int64)
        @ Blosc ~/.julia/packages/Blosc/vjmKP/src/Blosc.jl:58
      [3] compress(src::Ptr{Dates.AbstractTime}, src_size::Int64; kws::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
        @ Blosc ~/.julia/packages/Blosc/vjmKP/src/Blosc.jl:86
      [4] compress(src::Ptr{Dates.AbstractTime}, src_size::Int64)
        @ Blosc ~/.julia/packages/Blosc/vjmKP/src/Blosc.jl:85
      [5] compress(src::Vector{Dates.AbstractTime}; kws::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
        @ Blosc ~/.julia/packages/Blosc/vjmKP/src/Blosc.jl:92
      [6] compress(src::Vector{Dates.AbstractTime})
        @ Blosc ~/.julia/packages/Blosc/vjmKP/src/Blosc.jl:91
      [7] compress_then_write(b::Vector{Dates.AbstractTime}, io::BufferedStreams.BufferedOutputStream{IOStream})
        @ JDF ~/.julia/packages/JDF/SMfQY/src/compress_then_write.jl:8
      [8] compress_then_write(b::Vector{Union{Missing, DateTime}}, io::BufferedStreams.BufferedOutputStream{IOStream})
        @ JDF ~/.julia/packages/JDF/SMfQY/src/type-writer-loader/Missing.jl:13
      [9] macro expansion
        @ ~/.julia/packages/JDF/SMfQY/src/savejdf.jl:75 [inlined]
     [10] (::JDF.var"#57#60"{String, DataFrame, Symbol})()
        @ JDF ./threadingconstructs.jl:169