search

Nosferican / Econometrics.jl

Econometrics in Julia
https://nosferican.github.io/Econometrics.jl/dev
ISC License
69 stars 19 forks source link

show(m::TableRegressionModel{EconometricModel}) fails #8

Closed AsafManela closed 5 years ago

AsafManela commented 5 years ago

Great package! I tried to fit an EconometricModel with a dataframe and a formula and the model fits, but showing the model fails.

Here's a mwe:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.1 (2019-05-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(v1.1) pkg> activate /tmp

(tmp) pkg> add Econometrics
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
  Updating `/tmp/Project.toml`
  [4d6a76a9] + Econometrics v0.1.0
  Updating `/tmp/Manifest.toml`
  [7d9fca2a] + Arpack v0.3.1
  [4fba245c] + ArrayInterface v0.1.1
  [9e28174c] + BinDeps v0.8.10
  [b99e7846] + BinaryProvider v0.5.6
  [49dc2e85] + Calculus v0.5.0
  [324d7699] + CategoricalArrays v0.5.5
  [bbf7d656] + CommonSubexpressions v0.2.0
  [34da2185] + Compat v2.1.0
  [9a962f9c] + DataAPI v1.0.0
  [a93c6f00] + DataFrames v0.19.0
  [864edb3b] + DataStructures v0.17.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [01453d9d] + DiffEqDiffTools v0.14.0
  [163ba53b] + DiffResults v0.0.4
  [b552c78f] + DiffRules v0.0.10
  [31c24e10] + Distributions v0.21.0
  [4d6a76a9] + Econometrics v0.1.0
  [1a297f60] + FillArrays v0.6.3
  [f6369f11] + ForwardDiff v0.10.3
  [41ab1584] + InvertedIndices v1.0.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [682c06a0] + JSON v0.21.0
  [d3d80556] + LineSearches v7.0.1
  [e1d29d7a] + Missings v0.4.1
  [d41bc354] + NLSolversBase v7.3.1
  [77ba4419] + NaNMath v0.3.2
  [6fe1bfb0] + OffsetArrays v0.11.1
  [429524aa] + Optim v0.18.1
  [bac558e1] + OrderedCollections v1.1.0
  [90014a1f] + PDMats v0.9.7
  [d96e819e] + Parameters v0.10.3
  [69de0a69] + Parsers v0.3.6
  [2dfb63ee] + PooledArrays v0.5.2
  [85a6dd25] + PositiveFactorizations v0.2.2
  [1fd47b50] + QuadGK v2.0.3
  [3cdcf5f2] + RecipesBase v0.7.0
  [731186ca] + RecursiveArrayTools v0.20.0
  [189a3867] + Reexport v0.2.0
  [ae029012] + Requires v0.5.2
  [79098fc4] + Rmath v0.5.0
  [1277b4bf] + ShiftedArrays v0.5.0
  [a2af1166] + SortingAlgorithms v0.3.1
  [276daf66] + SpecialFunctions v0.7.2
  [90137ffa] + StaticArrays v0.11.0
  [2913bbd2] + StatsBase v0.31.0
  [4c63d2b9] + StatsFuns v0.8.0
  [3eaba693] + StatsModels v0.6.2
  [3783bdb8] + TableTraits v1.0.0
  [bd369af6] + Tables v0.2.9
  [30578b45] + URIParser v0.4.0
  [2a0f44e3] + Base64 
  [ade2ca70] + Dates 
  [8bb1440f] + DelimitedFiles 
  [8ba89e20] + Distributed 
  [9fa8497b] + Future 
  [b77e0a4c] + InteractiveUtils 
  [76f85450] + LibGit2 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [a63ad114] + Mmap 
  [44cfe95a] + Pkg 
  [de0858da] + Printf 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [1a1011a3] + SharedArrays 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [4607b0f0] + SuiteSparse 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode 

julia> using DataFrames, Econometrics

julia> data = DataFrame(X=[1,2,3], y=[4,5,2])
3×2 DataFrame
│ Row │ X     │ y     │
│     │ Int64 │ Int64 │
├─────┼───────┼───────┤
│ 1   │ 1     │ 4     │
│ 2   │ 2     │ 5     │
│ 3   │ 3     │ 2     │

julia> fit(EconometricModel, @formula(y ~ X), data)
Continuous Response Model
Number of observations: 3
Null Loglikelihood: -4.92
Loglikelihood: -4.73
R-squared: 0.4286
LR Test: 0.38 ∼ χ²(1) ⟹  Pr > χ² = 0.5360
Error showing value of type EconometricModel{Econometrics.ContinuousResponse,StatsModels.FormulaTerm{StatsModels.Term,StatsModels.Term},Array{Int64,1},StatsBase.FrequencyWeights{Float64,Float64,StatsBase.FrequencyWeights{Float64,Float64,StatsBase.FrequencyWeights{Float64,Float64,FillArrays.Ones{Float64,1,Tuple{Base.OneTo{Int64}}}}}},Array{Float64,1},Tuple{String,Array{String,1}},Econometrics.VCE}:
ERROR: MethodError: no method matching iterate(::StatsModels.Term)
Closest candidates are:
  iterate(::Core.SimpleVector) at essentials.jl:568
  iterate(::Core.SimpleVector, ::Any) at essentials.jl:568
  iterate(::ExponentialBackOff) at error.jl:199
  ...
Stacktrace:
 [1] mapfoldl_impl(::Function, ::Function, ::NamedTuple{,Tuple{}}, ::StatsModels.Term) at ./reduce.jl:55
 [2] #mapfoldl#187(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}, ::Function, ::Function, ::Function, ::StatsModels.Term) at ./reduce.jl:72
 [3] mapfoldl(::Function, ::Function, ::StatsModels.Term) at ./reduce.jl:72
 [4] #mapreduce#191(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}, ::Function, ::Function, ::Function, ::StatsModels.Term) at ./reduce.jl:205
 [5] mapreduce(::Function, ::Function, ::StatsModels.Term) at ./reduce.jl:205
 [6] show(::IOContext{REPL.Terminals.TTYTerminal}, ::EconometricModel{Econometrics.ContinuousResponse,StatsModels.FormulaTerm{StatsModels.Term,StatsModels.Term},Array{Int64,1},StatsBase.FrequencyWeights{Float64,Float64,StatsBase.FrequencyWeights{Float64,Float64,StatsBase.FrequencyWeights{Float64,Float64,FillArrays.Ones{Float64,1,Tuple{Base.OneTo{Int64}}}}}},Array{Float64,1},Tuple{String,Array{String,1}},Econometrics.VCE}) at /home/amanela/.julia/packages/Econometrics/bbes2/src/main.jl:83
 [7] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::EconometricModel{Econometrics.ContinuousResponse,StatsModels.FormulaTerm{StatsModels.Term,StatsModels.Term},Array{Int64,1},StatsBase.FrequencyWeights{Float64,Float64,StatsBase.FrequencyWeights{Float64,Float64,StatsBase.FrequencyWeights{Float64,Float64,FillArrays.Ones{Float64,1,Tuple{Base.OneTo{Int64}}}}}},Array{Float64,1},Tuple{String,Array{String,1}},Econometrics.VCE}) at ./sysimg.jl:219
 [8] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:131
 [9] display(::REPL.REPLDisplay, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:135
 [10] display(::Any) at ./multimedia.jl:287
 [11] #invokelatest#1 at ./essentials.jl:742 [inlined]
 [12] invokelatest at ./essentials.jl:741 [inlined]
 [13] print_response(::IO, ::Any, ::Any, ::Bool, ::Bool, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:155
 [14] print_response(::REPL.AbstractREPL, ::Any, ::Any, ::Bool, ::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:140
 [15] (::getfield(REPL, Symbol("#do_respond#38")){Bool,getfield(REPL, Symbol("##48#57")){REPL.LineEditREPL},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:714
 [16] #invokelatest#1 at ./essentials.jl:742 [inlined]
 [17] invokelatest at ./essentials.jl:741 [inlined]
 [18] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2273
 [19] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:1035
 [20] run_repl(::REPL.AbstractREPL, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:192
 [21] (::getfield(Base, Symbol("##734#736")){Bool,Bool,Bool,Bool})(::Module) at ./client.jl:362
 [22] #invokelatest#1 at ./essentials.jl:742 [inlined]
 [23] invokelatest at ./essentials.jl:741 [inlined]
 [24] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:346
 [25] exec_options(::Base.JLOptions) at ./client.jl:284
 [26] _start() at ./client.jl:436

julia>
cpfiffer commented 5 years ago

I've had this before with StatsModels, where sometimes rhs is iterable and sometimes it is a singleton. I wonder if this should be fixed upstream in the long term by requesting that StatsModels always enforce some kind of iterable?

cpfiffer commented 5 years ago

As a side note, the problem comes from running regressions with single regressors. An example of the error is the mapreduce in this line:

https://github.com/Nosferican/Econometrics.jl/blob/e20246153f923dd5d2eae467e7ed19c0981eb056/src/main.jl#L48

Nosferican commented 5 years ago

Thanks so much for using the software and reporting the bug to improve the package. It is indeed the issue with different return types in StatsModel the issue. I thought I had caught those issues, but it seems those are correctly handled through decompose, but I missed the special handling later in the pipeline for some show methods. I worked on a fix, but I want to make sure it works for the other models (add a couple more tests). I should have the PR by tomorrow hopefully if not by the latest on Friday for JuliaCon hackaton. Thanks again.