search

GordStephen / TimeAxisArrays.jl

Convenience wrapper around AxisArrays.jl for working with time series data.
http://gordstephen.github.io/TimeAxisArrays.jl/latest/
Other
0 stars 2 forks source link

Relax categorical constraint on higher dimensions #5

Open GordStephen opened 7 years ago

GordStephen commented 7 years ago

For spatiotemporal datasets (e.g. https://github.com/JuliaStats/TimeSeries.jl/issues/237#issuecomment-286512076) it would be useful to use the time analysis operations here with additional dimensional (not categorical) axes.

Balinus commented 7 years ago

How can this be done? I could have a look, but I'm not sure where to look.

GordStephen commented 7 years ago

Right now the TimeAxisArray type is really just defined as an alias for an AxisArray with one specifically-named dimensional axis and an arbitrary number of categorical axes. I think it should just be be a matter of changing the alias to remove the categorical axis requirement. There might be other functionality that depends on that requirement though - I can't remember of the top of my head.

Probably also worth noting that the type alias code could likely be cleaned up to leverage the new type system changes in 0.6.

Balinus commented 7 years ago

ok, I'll try some modification this week if I have some free time.

I haven't looked yet at the changes for 0.6 though. I should do that soon for my own package.

Balinus commented 7 years ago

By removing the categorical axis requirement, the construction of the TimeAxisArray work fine.

new aliases

typealias TimeAxisArray{T,N,D,Ax} AxisArray{T,N,D,Ax}
typealias RegularTimeAxisArray{T,N,D,Ax} AxisArray{T,N,D,Ax}

Construction

julia> using TimeAxisArrays
WARNING: Method definition show(IO, AxisArrays.AxisArray{T<:Any, N<:Any, D<:Any, Ax<:Any}) in module TimeAxisArrays at /home/proy/.julia/v0.5/TimeAxisArrays/src/io.jl:91 overwritten at /home/proy/.julia/v0.5/TimeAxisArrays/src/io.jl:98.

julia> d = Date(2003,1,1):Date(2005,12,31)
2003-01-01:1 day:2005-12-31

julia> data = randn(1096, 2, 2);
axisdata = TimeAxisArray(data, Axis{:Timestamp}(d), Axis{:lon}(101:102), Axis{:lat}(1:2))
3-dimensional AxisArray{Float64,3,...} with axes:
    :Timestamp, 2003-01-01:1 day:2005-12-31
    :lon, 101:102
    :lat, 1:2
And data, a 1096×2×2 Array{Float64,3}:
[:, :, 1] =
 -0.585504    0.667164 
 -0.503965   -0.824205 
 -0.220978   -1.8132   
 -0.616088   -0.038606 
  1.89973    -0.0678481
  0.36855    -0.685603 
  1.05448    -0.724773 
  0.072895   -0.550007 
 -0.0847654  -1.61771  
  1.13098     1.13699  
  1.54483     0.353656 
 -0.33052    -1.91523  
 -2.15136     0.208129 
  1.15898    -0.816925 
  0.520118   -0.0618857
  0.650814   -1.05039  
  0.86143    -0.402294 
  0.351038   -0.902535 
 -0.706607   -0.825477 
  ⋮                    
  1.06088    -1.59086  
 -0.761472    0.752608 
 -1.32284    -1.24447  
 -0.564596   -0.554048 
 -0.103787    0.885441 
  1.19524    -1.35354  
  1.47775     0.0715582
 -0.131102   -0.217329 
 -0.1272      0.83098  
  1.5499     -0.185216 
 -0.142151    0.0147438
  0.716681    2.56115  
  1.26275     0.447962 
  2.04805    -0.550029 
  1.7486     -0.26556  
 -1.79568     0.228135 
 -1.36954     0.462533 
  1.18338     1.26268  

[:, :, 2] =
  2.672        -0.979687  
 -0.200491     -1.19122   
  0.755967      0.251535  
  0.258377      0.447305  
 -0.155009     -0.33376   
 -0.730596     -2.96795   
 -1.68473      -0.0752483 
 -0.535254     -0.157006  
  0.0235949     0.55986   
  1.61409       1.25917   
  0.0385427     1.01333   
  1.87137       0.0419743 
 -0.97219      -1.59647   
  0.854699      2.33751   
  0.427118     -0.911462  
  1.35077       1.80795   
  1.10683      -0.311698  
 -0.713548     -0.188556  
  0.902827     -0.00435683
  ⋮                       
  0.42973      -0.19868   
 -1.73914      -1.08767   
 -0.438811     -1.5991    
  0.237952     -1.24714   
 -0.748307     -1.7729    
  0.157859      0.625821  
  0.667784     -1.82726   
  0.121         1.42114   
 -0.0281587     0.769741  
  0.987771     -0.600496  
 -2.07365       0.19058   
  0.399811      0.940137  
  1.66791      -0.223442  
 -0.0440884     0.0506878 
 -0.000518237   2.41138   
  1.06038      -1.35042   
 -2.29661      -1.32036   
 -0.650597      1.20926

However, I can split it, but the show method fail. For example:

julia> y = split(axisdata, Dates.week)
157-element Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}:
Error showing value of type Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}:
------ MethodError --------------------- Stacktrace (most recent call last)

 [1] — _start() at client.jl:363

 [2] — run_repl(::Base.REPL.LineEditREPL, ::Base.##950#951) at REPL.jl:188

 [3] — run_frontend(::Base.REPL.LineEditREPL, ::Base.REPL.REPLBackendRef) at REPL.jl:903

 [4] — run_interface(::Base.Terminals.TTYTerminal, ::Base.LineEdit.ModalInterface) at LineEdit.jl:1579

 [5] — (::Base.REPL.##22#23{Bool,Base.REPL.##33#42{Base.REPL.LineEditREPL,Base.REPL.REPLHistoryProvider},Base.REPL.LineEditREPL,Base.LineEdit.Prompt})(::Base.LineEdit.MIState, ::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Bool) at REPL.jl:652

 [6] — print_response(::Base.REPL.LineEditREPL, ::Any, ::Void, ::Bool, ::Bool) at REPL.jl:139

 [7] — print_response(::Base.Terminals.TTYTerminal, ::Any, ::Void, ::Bool, ::Bool, ::Void) at REPL.jl:154

 [8] — display(::Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}) at multimedia.jl:143

 [9] — display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}) at REPL.jl:135

 [10] — display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::MIME{Symbol("text/plain")}, ::Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}) at output_prompt_overwrite.jl:8

 [11] — #showarray#342(::Bool, ::Function, ::IOContext{Base.Terminals.TTYTerminal}, ::Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}, ::Bool) at show.jl:1618

 [12] — print_matrix(::IOContext{Base.Terminals.TTYTerminal}, ::Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}, ::String, ::String, ::String) at show.jl:1379

 [13] — print_matrix(::IOContext{Base.Terminals.TTYTerminal}, ::Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}, ::String, ::String, ::String, ::String, ::String, ::String, ::Int64, ::Int64) at show.jl:1407

 [14] — alignment(::IOContext{Base.Terminals.TTYTerminal}, ::Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}, ::Array{Int64,1}, ::Array{Int64,1}, ::Int64, ::Int64, ::Int64) at show.jl:1278

 [15] — (::Base.#kw##sprint)(::Array{Any,1}, ::Base.#sprint, ::Int64, ::Function, ::AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}}, ::Vararg{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},N}) at <missing>:0

 [16] — #sprint#316(::IOContext{Base.Terminals.TTYTerminal}, ::Function, ::Int64, ::Function, ::AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}}, ::Vararg{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},N}) at io.jl:35

 [17] — show(::IOContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}}) at io.jl:98

 [18] — textrep(::IOContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}}, ::String) at io.jl:80

 [19] — setindex!(::Array{Array{Symbol,1},1}, ::UnitRange{Int64}, ::Int64) at array.jl:415

 [20] — copy!(::Base.LinearFast, ::Array{Symbol,1}, ::Base.LinearFast, ::UnitRange{Int64}) at abstractarray.jl:567

MethodError: Cannot `convert` an object of type Int64 to an object of type Symbol
This may have arisen from a call to the constructor Symbol(...),
since type constructors fall back to convert methods.

julia> typeof(y)
Array{AxisArrays.AxisArray{Float64,3,Array{Float64,3},Tuple{AxisArrays.Axis{:Timestamp,StepRange{Date,Base.Dates.Day}},AxisArrays.Axis{:lon,UnitRange{Int64}},AxisArrays.Axis{:lat,UnitRange{Int64}}}},1}

julia> y[1]
3-dimensional AxisArray{Float64,3,...} with axes:
    :Timestamp, 2003-01-01:1 day:2003-12-31
    :lon, 101:102
    :lat, 1:2
And data, a 365×2×2 Array{Float64,3}:
[:, :, 1] =
 -0.585504    0.667164 
 -0.503965   -0.824205 
 -0.220978   -1.8132   
 -0.616088   -0.038606 
  1.89973    -0.0678481
  0.36855    -0.685603 
  1.05448    -0.724773 
  0.072895   -0.550007 
 -0.0847654  -1.61771  
  1.13098     1.13699  
  1.54483     0.353656 
 -0.33052    -1.91523  
 -2.15136     0.208129 
  1.15898    -0.816925 
  0.520118   -0.0618857
  0.650814   -1.05039  
  0.86143    -0.402294 
  0.351038   -0.902535 
 -0.706607   -0.825477 
  ⋮                    
  0.0503006   0.225358 
  2.73277     0.0703782
  0.199723   -0.956005 
 -0.706286   -1.9662   
 -0.200159    0.703824 
 -0.813457    0.663858 
  0.0169925  -0.0297694
  0.619558    1.22465  
  0.334638   -1.19285  
 -1.86165     0.526604 
  0.940619    0.25232  
  0.0820115   1.76448  
 -0.99047    -0.668108 
  0.470751   -0.475072 
  0.230683   -0.280583 
 -0.695754    0.0897808
 -0.176153    0.688429 
  0.0823042   1.41379  

[:, :, 2] =
  2.672      -0.979687  
 -0.200491   -1.19122   
  0.755967    0.251535  
  0.258377    0.447305  
 -0.155009   -0.33376   
 -0.730596   -2.96795   
 -1.68473    -0.0752483 
 -0.535254   -0.157006  
  0.0235949   0.55986   
  1.61409     1.25917   
  0.0385427   1.01333   
  1.87137     0.0419743 
 -0.97219    -1.59647   
  0.854699    2.33751   
  0.427118   -0.911462  
  1.35077     1.80795   
  1.10683    -0.311698  
 -0.713548   -0.188556  
  0.902827   -0.00435683
  ⋮                     
  0.0734221  -1.49767   
 -0.617505    0.00709292
  1.22179    -0.233612  
  0.604006    1.5325    
 -1.77933    -0.351747  
 -0.307627   -0.594366  
 -0.511383   -1.20962   
 -0.209367   -1.10514   
  2.8217      1.94352   
 -1.11674     1.46416   
  1.56061    -0.978916  
  0.65698     0.656333  
 -0.722179   -1.36383   
 -0.617324   -1.61078   
 -0.300879    0.462141  
  1.08235    -1.11485   
 -0.172335    0.0249433 
  0.927751   -0.57671
GordStephen commented 7 years ago

That makes sense, the show method is expecting Symbol-valued axes on the higher dimensions but is getting Ints instead (since the lat and lon dimensions are ranges).

One thing to note, under that new definition a TimeAxisArray is indistinguishable from a normal AxisArray (hence the method redefinition warning) - try this definition instead:

typealias TimeAxisArray{T,N,D,Ax<:Tuple{TimeAxis,Vararg{Axis}}} AxisArray{T,N,D,Ax}
typealias RegularTimeAxisArray{T,N,D,Ax<:Tuple{RegularTimeAxis,Vararg{Axis}}} AxisArray{T,N,D,Ax}