FixedEffects / FixedEffectModels.jl

Fast Estimation of Linear Models with IV and High Dimensional Categorical Variables
Other
227 stars 46 forks source link

GPU: ERROR: MethodError: no method matching similar #135

Closed zozotintin closed 3 years ago

zozotintin commented 3 years ago

Problem

When I tried the GPU example in the home page, it creates the following error

julia> reg(df, @formula(Sales ~ NDI + fe(State) + fe(Year)), method = :gpu, double_precision = false)
ERROR: MethodError: no method matching similar(::Type{CuArray{Float32,1}}, ::Type{Float32}, ::Tuple{Int64})
Closest candidates are:
  similar(::Array, ::Type, ::Tuple{Vararg{Int64,N}}) where N at array.jl:380
  similar(::SubArray, ::Type, ::Tuple{Vararg{Int64,N}} where N) at subarray.jl:65
  similar(::Base.ReshapedArray, ::Type, ::Tuple{Vararg{Int64,N}} where N) at reshapedarray.jl:207
  ...
Stacktrace:
 [1] convert(::Type{CuArray{Float32,1}}, ::StatsBase.UnitWeights{Int64}) at /home/poyu/.julia/packages/GPUArrays/ZxsKE/src/host/construction.jl:69
 [2] FixedEffectSolverGPU at /home/poyu/.julia/packages/FixedEffects/W0Q2Y/src/FixedEffectSolvers/FixedEffectSolverGPU.jl:109 [inlined]
 [3] FixedEffects.AbstractFixedEffectSolver{Float32}(::Array{FixedEffects.FixedEffect,1}, ::StatsBase.UnitWeights{Int64}, ::Type{Val{:gpu}}, ::Int64) at /home/poyu/.julia/packages/FixedEffects/W0Q2Y/src/FixedEffectSolvers/FixedEffectSolverGPU.jl:130
 [4] reg(::Any, ::FormulaTerm, ::StatsBase.CovarianceEstimator; contrasts::Dict{Symbol,Any}, weights::Nothing, save::Symbol, method::Symbol, nthreads::Int64, double_precision::Bool, tol::Float64, maxiter::Int64, drop_singletons::Bool, progress_bar::Bool, dof_add::Int64, subset::Nothing) at /home/poyu/.julia/packages/FixedEffectModels/c2OD9/src/fit.jl:150
 [5] top-level scope at REPL[5]:1

I am not sure whether this is related to #110 or not. I did using CUDA before using FixedEffectModels but the error persists.

Version Info

julia> Pkg.status("FixedEffectModels")
Status `~/.julia/environments/v1.5/Project.toml`
  [9d5cd8c9] FixedEffectModels v1.3.0

julia> Pkg.status("CUDA")
Status `~/.julia/environments/v1.5/Project.toml`
  [052768ef] CUDA v2.1.0

julia> Pkg.status("RDatasets")
Status `~/.julia/environments/v1.5/Project.toml`
  [ce6b1742] RDatasets v0.6.10
julia> versioninfo()
Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, cascadelake)
$ nvidia-smi 
Sun Nov  8 17:26:01 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.100      Driver Version: 440.100      CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro P400         Off  | 00000000:2D:00.0 Off |                  N/A |
| 34%   41C    P8    N/A /  N/A |    754MiB /  1975MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0   1227901      C   ...u/Documents/tools/julia-1.5.2/bin/julia    25MiB |
|    0   3720070      G   /opt/teamviewer/tv_bin/TeamViewer              7MiB |
|    0   3807834      G   rstudio                                      104MiB |
|    0   3956699      G   /usr/lib/xorg/Xorg                           163MiB |
|    0   3958448      G   /usr/lib/xorg/Xorg                           188MiB |
|    0   3958580      G   /usr/bin/gnome-shell                         176MiB |
|    0   3959048      G   ...quest-channel-token=7434953905993259253    67MiB |
+-----------------------------------------------------------------------------+
matthieugomez commented 3 years ago

Thanks for the report. GPU is hard to test for me. Could you try with this fork of FixedEffects and report to me if it solves the problem? https://github.com/matthieugomez/FixedEffects.jl (it should return v2.0.1 after doing ]status FixedEffects)

zozotintin commented 3 years ago

Hi. I am sorry that between the time I reported the bug and the time you proposed the fix, the CUDA driver on the workstation died, and I don't have the root privilege to fix the driver. I am sorry the I am not able to help with testing at this moment.

matthieugomez commented 3 years ago

Would you be able to have another look now?

dwinkler1 commented 3 years ago

I encountered the same error with the GPU, upgraded to the new version and now getting this error

julia> reg(df, @formula(Sales ~ NDI + fe(State) + fe(Year)), method=:gpu);
ERROR: type FixedEffect has no field m
Stacktrace:
[1] getproperty(::FixedEffects.FixedEffect{CuArray{UInt32,1},CuArray{Float64,1}}, ::Symbol) at ./Base.jl:33
[2] update_weights!(::FixedEffects.FixedEffectSolverGPU{Float64}, ::StatsBase.UnitWeights{Int64}) at
/home/daniel/.julia/packages/FixedEffects/bI2LS/src/FixedEffectSolvers/FixedEffectSolverGPU.jl:137
[3] FixedEffects.AbstractFixedEffectSolver{Float64}(::Array{FixedEffects.FixedEffect,1}, ::StatsBase.UnitWeights{Int64}, ::Type{Val{:gpu}}, ::Int64) at
/home/daniel/.julia/packages/FixedEffects/bI2LS/src/FixedEffectSolvers/FixedEffectSolverGPU.jl:130
[4] reg(::Any, ::FormulaTerm, ::StatsBase.CovarianceEstimator; contrasts::Dict{Symbol,Any}, weights::Nothing, save::Symbol, method::Symbol, nthreads::Int64,
double_precision::Bool, tol::Float64, maxiter::Int64, drop_singletons::Bool, progress_bar::Bool, dof_add::Int64, subset::Nothing) at
/home/daniel/.julia/packages/FixedEffectModels/0aerL/src/fit.jl:150
[5] top-level scope at REPL[36]:1
pkg> st FixedEffects
Status `~/Documents/Projects/Playlist Network Analysis/code/EmbeddingNetwork/Project.toml`
[c8885935] FixedEffects v2.0.1 `https://github.com/matthieugomez/FixedEffects.jl.git#master`
dwinkler1 commented 3 years ago

This fixes it for me:

Pull request

matthieugomez commented 3 years ago

Closing the issue --- it should now work with the master version of FixedEffects.jl. Thanks @danielw2904

dwinkler1 commented 3 years ago

Sorry it needs to be reopened I am getting the original error again. Possibly because I forked @matthieugomez 's repo rather than the official one? I think this needs to be merged as well diff

dwinkler1 commented 3 years ago

See pr