search

jmboehm / GLFixedEffectModels.jl

Fast estimation of generalized linear models with high dimensional categorical variables in Julia
Other
33 stars 6 forks source link

Precompile issue in v0.5? #46

Closed Gkreindler closed 1 year ago

Gkreindler commented 1 year ago

I get an error while trying to precompile v0.5:

[ Info: Precompiling GLFixedEffectModels [bafb0ae5-e5f5-5100-81b6-6a55d777c812]
ERROR: LoadError: UndefVarError: omitsintercept not defined
Stacktrace:
 [1] nlreg(df::Any, formula::StatsModels.FormulaTerm, distribution::Distributions.Binomial{Float64}, link::GLM.LogitLink, vcov::StatsBase.CovarianceEstimator; weights::Union{Nothing, Symbol}, subset::Union{Nothing, AbstractVector}, start::Union{Nothing, AbstractVector}, maxiter_center::Int64, maxiter::Int64, contrasts::Dict{Symbol, Any}, dof_add::Int64, save::Vector{Symbol}, method::Symbol, drop_singletons::Bool, double_precision::Bool, dev_tol::Float64, rho_tol::Float64, step_tol::Float64, center_tol::Float64, separation::Vector{Symbol}, separation_mu_lbound::Float64, separation_mu_ubound::Float64, separation_ReLU_tol::Float64, separation_ReLU_maxiter::Int64, vcovformula::Union{Nothing, Expr, Symbol}, subsetformula::Union{Nothing, Expr, Symbol}, verbose::Bool)
   @ GLFixedEffectModels C:\Users\Gabriel Kreindler\.julia\packages\GLFixedEffectModels\U2YVx\src\fit.jl:93
 [2] nlreg (repeats 2 times)
   @ C:\Users\Gabriel Kreindler\.julia\packages\GLFixedEffectModels\U2YVx\src\fit.jl:40 [inlined]
 [3] top-level scope
   @ C:\Users\Gabriel Kreindler\.julia\packages\GLFixedEffectModels\U2YVx\src\GLFixedEffectModels.jl:61
 [4] include
   @ .\Base.jl:419 [inlined]
 [5] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base .\loading.jl:1554
 [6] top-level scope
   @ stdin:1
in expression starting at C:\Users\Gabriel Kreindler\.julia\packages\GLFixedEffectModels\U2YVx\src\GLFixedEffectModels.jl:2
in expression starting at stdin:1
ERROR: Failed to precompile GLFixedEffectModels [bafb0ae5-e5f5-5100-81b6-6a55d777c812] to C:\Users\Gabriel Kreindler\.julia\compiled\v1.8\GLFixedEffectModels\jl_D048.tmp.
Stacktrace:
  [1] error(s::String)
    @ Base .\error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base .\loading.jl:1705
  [3] compilecache
    @ .\loading.jl:1649 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base .\loading.jl:1337
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base .\loading.jl:1200
  [6] macro expansion
    @ .\loading.jl:1180 [inlined]
  [7] macro expansion
    @ .\lock.jl:223 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base .\loading.jl:1144
  [9] eval
    @ .\boot.jl:368 [inlined]
 [10] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base .\loading.jl:1428
 [11] include_string(m::Module, txt::String, fname::String)
    @ Base .\loading.jl:1438
 [12] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base .\essentials.jl:729
 [13] invokelatest(::Any, ::Any, ::Vararg{Any})
    @ Base .\essentials.jl:726
 [14] inlineeval(m::Module, code::String, code_line::Int64, code_column::Int64, file::String; softscope::Bool)
    @ VSCodeServer c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\eval.jl:233
 [15] (::VSCodeServer.var"#66#70"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\eval.jl:157
 [16] withpath(f::VSCodeServer.var"#66#70"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams}, path::String)
    @ VSCodeServer c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\repl.jl:249
 [17] (::VSCodeServer.var"#65#69"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\eval.jl:155
 [18] hideprompt(f::VSCodeServer.var"#65#69"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})
    @ VSCodeServer c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\repl.jl:38
 [19] (::VSCodeServer.var"#64#68"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\eval.jl:126
 [20] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging .\logging.jl:511
 [21] with_logger
    @ .\logging.jl:623 [inlined]
 [22] (::VSCodeServer.var"#63#67"{VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\eval.jl:225
 [23] #invokelatest#2
    @ .\essentials.jl:729 [inlined]
 [24] invokelatest(::Any)
    @ Base .\essentials.jl:726
 [25] macro expansion
    @ c:\Users\Gabriel Kreindler\.vscode\extensions\julialang.language-julia-1.47.2\scripts\packages\VSCodeServer\src\eval.jl:34 [inlined]
 [26] (::VSCodeServer.var"#61#62")()
    @ VSCodeServer .\task.jl:484

I just run

using Pkg
Pkg.activate(MY_ENV_PATH)
using GLFixedEffectModels

I have Julia 1.8.1 on Windows 11, just ran ]update, and everything works ok with v0.4 (using ]add FLFixedEffectModels@0.4)

jmboehm commented 1 year ago

Thanks. I can't replicate this right now (on Mac), but it may be due to a missing dependency. Which version of StatsModels (if any) do you have installed?

Gkreindler commented 1 year ago

Thank you. I do not have StatsModels! Adding it and running using StatsModels before trying to precompile GLFixedEffectModels still doesn't work.

jmboehm commented 1 year ago

Could you try the following?

] add https://github.com/jmboehm/GLFixedEffectModels.jl#precompile-and-speed
Gkreindler commented 1 year ago

I was able to uninstall the current version, install the commit you shared, and close and re-open Julia, but it still gives the same error (below).

could the issue be the following?

julia> using StatsModels
julia> StatsModels.omitsintercept
omitsintercept (generic function with 2 methods)

julia> omitsintercept
ERROR: UndefVarError: omitsintercept not defined
[ Info: Precompiling GLFixedEffectModels [bafb0ae5-e5f5-5100-81b6-6a55d777c812]
ERROR: LoadError: UndefVarError: omitsintercept not defined
Stacktrace:
 [1] nlreg(df::Any, formula::StatsModels.FormulaTerm, distribution::Distributions.Binomial{Float64}, link::GLM.LogitLink, vcov::StatsBase.CovarianceEstimator; weights::Union{Nothing, Symbol}, subset::Union{Nothing, AbstractVector}, start::Union{Nothing, AbstractVector}, maxiter_center::Int64, maxiter::Int64, contrasts::Dict{Symbol, Any}, dof_add::Int64, save::Vector{Symbol}, method::Symbol, drop_singletons::Bool, double_precision::Bool, dev_tol::Float64, rho_tol::Float64, step_tol::Float64, center_tol::Float64, separation::Vector{Symbol}, separation_mu_lbound::Float64, separation_mu_ubound::Float64, separation_ReLU_tol::Float64, separation_ReLU_maxiter::Int64, vcovformula::Union{Nothing, Expr, Symbol}, subsetformula::Union{Nothing, Expr, Symbol}, verbose::Bool)
   @ GLFixedEffectModels C:\Users\Gabriel Kreindler\.julia\packages\GLFixedEffectModels\rCDDJ\src\fit.jl:93
jmboehm commented 1 year ago

Thanks. It's strange: when I do

using StatsModels

the omitsintercept function gets exported and I can use it without referring to the StatsModels module. (This is with StatsModels 0.7.2.)

I've updated the precompile-and-speed branch with code that explicitly has StatsModels.omitsintercept. Let me know if it works now. If not, tbh i'm running out of ideas...

Gkreindler commented 1 year ago

Ah, ok, I had StatsModels v0.6.33, and this issue

(tj_env_gk) pkg> status --outdated
Status `C:\git-repos\jakarta-optimal-bus-nets\onemodel-versions\v7\tj_env_gk\Project.toml`
⌃ [324d7699] CategoricalArrays v0.9.7 (<v0.10.8)
⌃ [a93c6f00] DataFrames v1.3.6 (<v1.5.0)
⌃ [9d5cd8c9] FixedEffectModels v1.8.0 (<v1.9.2)
⌅ [2913bbd2] StatsBase v0.33.21 (<v0.34.0): FixedEffectModels, FixedEffects, GLFixedEffectModels, LsqFit, Optim, RegressionTables, StatsModels, Vcov
⌅ [3eaba693] StatsModels v0.6.33 (<v0.7.2): FixedEffectModels, RegressionTables

Removing RegressionTables makes everything work (with simply ] add GLFixedEffectModels. Adding RegressionTables again gives this:

⌅ [3eaba693] ↓ StatsModels v0.7.2 ⇒ v0.6.33

(and GLFixedEffectModels again fails to precompile)

So perhaps that may be the issue? Or I may be doing something wrong.

Thank you for all your help!!

jmboehm commented 1 year ago

Thanks-- for me it works even with StatsModels 0.6.33, so this is hard to debug for me. But it sounds like this is a (temporary) fix, so I'm closing this (please reopen if you think it's not). RegressionTables should soon be updated to work with StatsModels 0.7.

I've tagged a new version, 0.5.1, which should be merged into the General registry soon, so you can switch back to that one.