xiaodaigh / JDF.jl

Julia DataFrames serialization format
MIT License
88 stars 9 forks source link

Date Vector with missing entries #62

Closed kafisatz closed 3 years ago

kafisatz commented 3 years ago

I run into the issue below with missing Date entries. Is this expected on JDF v0.4.0 ?


julia> col = [Date(1999,1,1),missing,today()]
3-element Array{Union{Missing, Date},1}:
 1999-01-01
 missing
 2021-04-14

julia> df = DataFrame(d = col)
3×1 DataFrame
 Row │ d
     │ Date?      
─────┼────────────
   1 │ 1999-01-01
   2 │ missing    
   3 │ 2021-04-14

julia> JDF.save(raw"c:\temp\fifi", df)
ERROR: TaskFailedException:
Abstract type AbstractTime does not have a definite size.
Stacktrace:
 [1] sizeof at .\essentials.jl:449 [inlined]
 [2] compress!(::Array{UInt8,1}, ::Ptr{Dates.AbstractTime}, ::Int64) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\Blosc\Six7M\src\Blosc.jl:58
 [3] compress(::Ptr{Dates.AbstractTime}, ::Int64; kws::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\Blosc\Six7M\src\Blosc.jl:86
 [4] compress(::Ptr{Dates.AbstractTime}, ::Int64) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\Blosc\Six7M\src\Blosc.jl:85
 [5] compress(::Array{Dates.AbstractTime,1}; kws::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\Blosc\Six7M\src\Blosc.jl:92
 [6] compress(::Array{Dates.AbstractTime,1}) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\Blosc\Six7M\src\Blosc.jl:91
 [7] compress_then_write(::Array{Dates.AbstractTime,1}, ::BufferedStreams.BufferedOutputStream{IOStream}) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\JDF\jOO9d\src\compress_then_write.jl:8
 [8] compress_then_write(::Array{Union{Missing, Date},1}, ::BufferedStreams.BufferedOutputStream{IOStream}) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\JDF\jOO9d\src\type-writer-loader\Missing.jl:13
 [9] macro expansion at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\JDF\jOO9d\src\savejdf.jl:73 [inlined]
 [10] (::JDF.var"#61#64"{String,DataFrame,Symbol})() at .\threadingconstructs.jl:169
Stacktrace:
 [1] wait at .\task.jl:267 [inlined]
 [2] fetch(::Task) at .\task.jl:282
 [3] _broadcast_getindex_evalf at .\broadcast.jl:648 [inlined]
 [4] _broadcast_getindex at .\broadcast.jl:621 [inlined]
 [5] getindex at .\broadcast.jl:575 [inlined]
 [6] copy at .\broadcast.jl:876 [inlined]
 [7] materialize at .\broadcast.jl:837 [inlined]
 [8] save(::String, ::DataFrame; verbose::Bool) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\JDF\jOO9d\src\savejdf.jl:80
 [9] save(::String, ::DataFrame) at C:\Users\bernhard.konig.ROOT_MILLIMAN\.julia\packages\JDF\jOO9d\src\savejdf.jl:50
 [10] top-level scope at REPL[49]:1

julia>
xiaodaigh commented 3 years ago

Thanks for the report. This is unexpected. Let me debug this.

xiaodaigh commented 3 years ago

Thanks for the bug report. It's been fixed in 0.4.1. Will be available shortly.