search

panlanfeng / KernelEstimator.jl

The julia package for nonparametric density estimate and regression
Other
26 stars 6 forks source link

Multivariant regression? #6

Open arturgower opened 7 years ago

arturgower commented 7 years ago

Hi there, nice package. Any plans/capabilities for multiple regression, several independent variables, or multivariant regression (more than one dependent variable) ?

panlanfeng commented 7 years ago

There is a multivariate regression function localconstant that allows multiple independent variables (or covariates, the X). Right now I don't have plan to add function dealing with multiple response variables but I am open to any suggestions. And you are very welcome to open pull requests.

arturgower commented 7 years ago

Thanks Pan, great I'll test out the multiple independent variable capabilities. Afraid I can't help develop much at the moment as am new to nonparametric regression.

arturgower commented 7 years ago

Any tips how to adjust the bias/variance trade off for localconstant(x,y)? Also guessing that passing localconstant(x,y; eval=newx), will predict the labels for newx right?

Also ran into this:

julia>n=50
julia>x = 2pi*rand(n,2); y = sin(x[:,1]).*cos(x[:,2]) + 0.2*randn(n);
julia>localconstant(x,y)
ERROR: MethodError: no method matching bwlocalconstant(::Array{Float64,2}, ::Array{Float64,1}, ::Array{KernelEstimator.#gaussiankernel,1})
Closest candidates are:
  bwlocalconstant(::AbstractArray{T<:Real,2}, ::AbstractArray{T<:Real,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/bandwidth.jl:316
  bwlocalconstant(::AbstractArray{T<:Real,2}, ::AbstractArray{T<:Real,1}, ::Array{Function,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/bandwidth.jl:316
  bwlocalconstant(::AbstractArray{T<:Real,1}, ::AbstractArray{T<:Real,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/bandwidth.jl:214
  ...
 in localconstant(::Array{Float64,2}, ::Array{Float64,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/regression.jl:98

which I worked around by changing line 96 of regression.jl from kernel::Array{Function, 1} = [gaussiankernel for i in 1:size(xdata)[2]] to kernel::Array{Function, 1} = Array{Function}([gaussiankernel for i in 1:size(xdata)[2]])

panlanfeng commented 7 years ago

localconstant will choose bandwidth via cross validation, which is the process of bias and variance trade off.

Yes the function will predict the y on newx.

Thanks for catching this bug. It is fixed in the master branch.

UserQuestions commented 6 years ago

Is there a plan to extend to multivariate local linear regression? My understanding is that such regressions have substantially better properties than local constant ones. I also think that there is a similar implementation in https://github.com/mcreel/Econometrics.jl

panlanfeng commented 6 years ago

Thank you for interested in the package. Local linear regression is much more complicated and computationally intensive but it is on the plan. As for the Econometrics package, I briefly browsed their code and I highly suspect their implementation is not real nonparametric local linear regression.