search

TuringLang / docs

Documentation and tutorials for the Turing language
https://turinglang.org/docs/
MIT License
231 stars 99 forks source link

Providing easily executable examples #20

Closed arnim closed 5 years ago

arnim commented 5 years ago

Hi together,

thanks for Turing and these great tutorials. One issue I've encountered while playing around with it was to get it actually run.

While this might be an "it works on my machine" thing it might help if there was something where people could run Turing models with one click and not experience any headaches.

I've cloned your repository and added REQUIREMENTS in the ./binder dir https://github.com/arnim/TuringTutorials https://notebooks.gesis.org/binder/v2/gh/arnim/TuringTutorials/master

However, I still get many errors. Am I using wrong versions of the dependencies? Or is there anything else I'm missing?

The requirements are in https://github.com/arnim/TuringTutorials/blob/master/binder/REQUIRE Maybe you have an idea.

Thanks in advance.

cpfiffer commented 5 years ago

Can you be any more specific about where you are getting errors and what those errors are?

arnim commented 5 years ago

1_GaussianMixtureModel.ipynb Out[6] MethodError: ForwardDiff.Dual https://notebooks.gesis.org/binder/v2/gh/arnim/TuringTutorials/master?filepath=1_GaussianMixtureModel.ipynb

2_LogisticRegression.ipynb Out[4] BoundsError: attempt to access "invalid columns 1:500 selected" https://notebooks.gesis.org/binder/v2/gh/arnim/TuringTutorials/master?filepath=2_LogisticRegression.ipynb

4_BayesHmm.ipynb Out[5] MethodError: no method matching parse(::SubString{String}) https://notebooks.gesis.org/binder/v2/gh/arnim/TuringTutorials/master?filepath=4_BayesHmm.ipynb

/5_LinearRegression.ipynb Out[3] BoundsError: attempt to access "invalid columns 1:22 selected" https://notebooks.gesis.org/binder/v2/gh/arnim/TuringTutorials/master?filepath=/5_LinearRegression.ipynb

Each of the errors can be reproduced by the respective links.

cpfiffer commented 5 years ago

Thank you! I know what the issue is for the logistic and linear regression tutorials, that should be easy to fix.

As for the mixture model tutorial, I'm not sure - looks like it's throwing an AD error around the MVNormal distribution in the model. @trappmartin or @mohamed82008, do you have ideas what's going on there?

The Bayes HMM model may be related to a recent change in the MCMCChain package, but I'll know more soon enough.

Thank you for consolidating these!

mohamed82008 commented 5 years ago

Unfortunately, I cannot reproduce the Gaussian mixture error on my machine. If you can try the above tutorial in a normal Julia environment with a Manifest.toml, this will be more reproducible on other machines. Just send the contents of the environment's Manifest.toml file so we can have the same setup as you and then hopefully see what's wrong.

arnim commented 5 years ago

@cpfiffer @mohamed82008 thanks for your feedback. The Manifest.toml is in the container linked above at /srv/julia/pkg/environments/v1.0/Manifest.toml and generated from https://github.com/arnim/TuringTutorials/blob/master/binder/REQUIRE

If there is something wrong, maybe you can provide a better suited REQUIRE file. Doing so would allow potential users to execute your tutorials with 1-click in the cloud and play around with them online.

cpfiffer commented 5 years ago

It's not an issue with the REQUIRE file. Two of these bugs are due to changes in the DataFrames interface, one of them (the mixture model tutorial) is due to a poorly implemented MvNormal function at Distributions.jl, and the last is currently being fixed on the main Turing branch.

The combination of the fact that these tutorials are new and the fact that there are many changes happening both on the Turing side and in the wider Julia ecosystem are making these less plug and play. I appreciate your patience here - we're always trying to make these accessible to everyone, and these are indeed problems I would like to see resolved.

Speaking of the MvNormal issue on the mixture model tutorial, @trappmartin (or maybe @willtebbutt, I don't think I quite heard your suggestion on the call the other day), do you have any quick fixes for this AD error? I'm also interested in why @mohamed82008 can't reproduce this --- I can produce this on my machine (Julia 1.0) in both :forward_diff and :reverse_diff.

Here's a copy of the full stacktrace:

[Gibbs] Sampling...  1%  ETA: 0:05:28

MethodError: no method matching Flux.Tracker.TrackedReal{Float64}(::Float64)
Closest candidates are:
  Flux.Tracker.TrackedReal{Float64}(::T<:Number) where T<:Number at boot.jl:725
  Flux.Tracker.TrackedReal{Float64}(::Any, !Matched::Any) where T<:Real at /home/cameron/.julia/packages/Flux/U8AZD/src/tracker/lib/real.jl:2
  Flux.Tracker.TrackedReal{Float64}(!Matched::Complex) where T<:Real at complex.jl:37
  ...

Stacktrace:
 [1] MvNormal(::Array{Flux.Tracker.TrackedReal{Float64},1}, ::Float64) at /home/cameron/.julia/packages/Distributions/WHjOk/src/multivariate/mvnormal.jl:220
 [2] (::getfield(Main, Symbol("###inner_function#365#11")))(::Turing.Core.VarReplay.VarInfo, ::Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}, ::Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}) at ./logging.jl:310
 [3] #call#3 at /home/cameron/.julia/dev/Turing/src/Turing.jl:57 [inlined]
 [4] Model at /home/cameron/.julia/dev/Turing/src/Turing.jl:57 [inlined]
 [5] runmodel!(::Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}, ::Turing.Core.VarReplay.VarInfo, ::Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}) at /home/cameron/.julia/dev/Turing/src/core/VarReplay.jl:110
 [6] (::getfield(Turing.Core, Symbol("#f#24")){Turing.Core.VarReplay.VarInfo,Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}},Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}})(::TrackedArray{…,Array{Float64,1}}) at /home/cameron/.julia/dev/Turing/src/core/ad.jl:149
 [7] #13 at /home/cameron/.julia/packages/Flux/U8AZD/src/tracker/back.jl:164 [inlined]
 [8] forward at /home/cameron/.julia/packages/Flux/U8AZD/src/tracker/back.jl:151 [inlined]
 [9] forward(::Function, ::Array{Real,1}) at /home/cameron/.julia/packages/Flux/U8AZD/src/tracker/back.jl:164
 [10] gradient_reverse(::Array{Real,1}, ::Turing.Core.VarReplay.VarInfo, ::Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}, ::Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}) at /home/cameron/.julia/dev/Turing/src/core/ad.jl:153
 [11] gradient at /home/cameron/.julia/dev/Turing/src/core/ad.jl:81 [inlined]
 [12] #2 at /home/cameron/.julia/dev/Turing/src/inference/support/hmc_core.jl:10 [inlined]
 [13] #_leapfrog#26(::getfield(Turing.Inference, Symbol("##6#7")){Turing.Core.VarReplay.VarInfo,Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}}, ::getfield(Turing.Inference, Symbol("##8#9")){Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}}, ::Function, ::Array{Real,1}, ::Array{Float64,1}, ::Int64, ::Float64, ::getfield(Turing.Inference, Symbol("##2#3")){Turing.Core.VarReplay.VarInfo,Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}},Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}}) at /home/cameron/.julia/dev/Turing/src/inference/support/hmc_core.jl:145
 [14] (::getfield(Turing.Inference, Symbol("#kw##_leapfrog")))(::NamedTuple{(:rev_func, :log_func),Tuple{getfield(Turing.Inference, Symbol("##6#7")){Turing.Core.VarReplay.VarInfo,Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}},getfield(Turing.Inference, Symbol("##8#9")){Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}}}}, ::typeof(Turing.Inference._leapfrog), ::Array{Real,1}, ::Array{Float64,1}, ::Int64, ::Float64, ::Function) at ./none:0
 [15] macro expansion at ./logging.jl:311 [inlined]
 [16] #_hmc_step#27(::Function, ::Function, ::Function, ::Array{Real,1}, ::Float64, ::getfield(Turing.Inference, Symbol("##4#5")){Turing.Core.VarReplay.VarInfo,Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}},Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}}, ::Function, ::getfield(Turing.Inference, Symbol("##16#17")), ::Int64, ::Float64, ::getfield(Turing.Inference, Symbol("##14#15")){Int64}) at /home/cameron/.julia/dev/Turing/src/inference/support/hmc_core.jl:186
 [17] (::getfield(Turing.Inference, Symbol("#kw##_hmc_step")))(::NamedTuple{(:rev_func, :log_func),Tuple{getfield(Turing.Inference, Symbol("##6#7")){Turing.Core.VarReplay.VarInfo,Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}},getfield(Turing.Inference, Symbol("##8#9")){Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}}}}, ::typeof(Turing.Inference._hmc_step), ::Array{Real,1}, ::Float64, ::Function, ::Function, ::Function, ::Int64, ::Float64, ::getfield(Turing.Inference, Symbol("##14#15")){Int64}) at ./none:0
 [18] #hmc_step#35 at /home/cameron/.julia/dev/Turing/src/inference/hmc.jl:74 [inlined]
 [19] (::getfield(Turing.Inference, Symbol("#kw##hmc_step")))(::NamedTuple{(:rev_func, :log_func),Tuple{getfield(Turing.Inference, Symbol("##6#7")){Turing.Core.VarReplay.VarInfo,Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}},getfield(Turing.Inference, Symbol("##8#9")){Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}}}}, ::typeof(Turing.Inference.hmc_step), ::Array{Real,1}, ::Float64, ::Function, ::Function, ::Function, ::Float64, ::HMC{Turing.Core.FluxTrackerAD,Symbol}, ::getfield(Turing.Inference, Symbol("##14#15")){Int64}) at ./none:0
 [20] macro expansion at ./logging.jl:322 [inlined]
 [21] step(::Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}, ::Turing.Sampler{HMC{Turing.Core.FluxTrackerAD,Symbol}}, ::Turing.Core.VarReplay.VarInfo, ::Val{false}) at /home/cameron/.julia/dev/Turing/src/inference/hmc.jl:212
 [22] #sample#45(::Bool, ::Nothing, ::Int64, ::Function, ::Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}, ::Gibbs{Tuple{PG{Symbol,typeof(Turing.Utilities.resample_systematic)},HMC{Turing.Core.FluxTrackerAD,Symbol}}}) at /home/cameron/.julia/dev/Turing/src/inference/gibbs.jl:114
 [23] sample(::Turing.Model{Tuple{:μ1,:μ2,:k},Tuple{:x},getfield(Main, Symbol("###inner_function#365#11")),NamedTuple{(:x,),Tuple{Array{Float64,2}}}}, ::Gibbs{Tuple{PG{Symbol,typeof(Turing.Utilities.resample_systematic)},HMC{Turing.Core.FluxTrackerAD,Symbol}}}) at /home/cameron/.julia/dev/Turing/src/inference/gibbs.jl:76
 [24] top-level scope at In[8]:2
mohamed82008 commented 5 years ago

With the Manifest file, I can reproduce it now. I will look into it.

mohamed82008 commented 5 years ago

This error is caused by a type piracy in Flux https://github.com/FluxML/Flux.jl/blob/master/src/tracker/lib/real.jl#L106 which causes ambiguity when converting a dual to another dual. MWE:

using ForwardDiff
using Flux.Tracker
ForwardDiff.Dual{Nothing, Real, 1}(ForwardDiff.Dual(1.0))
mohamed82008 commented 5 years ago

I'm also interested in why @mohamed82008 can't reproduce this

@cpfiffer I think I was using an old Flux.jl for some reason.

mohamed82008 commented 5 years ago

The reversediff issue is a different one though. It is caused by the lack of a converter from Float64 to TrackedReal which is required in https://github.com/JuliaStats/Distributions.jl/blob/master/src/multivariate/mvnormal.jl#L220. I opened another issue in Flux.

mohamed82008 commented 5 years ago

@arnim @cpfiffer The Gaussian mixture example should now work with Flux master, using both :forward_diff and :reverse_diff.

cpfiffer commented 5 years ago

@arnim I've just pushed updated tutorials to address the problems you've highlighted. We were able to fix a couple of very annoying bugs, thanks to you! Hopefully these will work better for you.

For anyone else who stumbles across this issue, make sure you update Flux (v0.7.2+) and Turing (to v0.6.7+) to solve the problems noted above.

arnim commented 5 years ago

Thanks @cpfiffer I've updated the requirements to the latest version of Flux and the GMM runs now smoothly.

Logistic Regression and Linear Regression still seem to have some some issues. In the case of Logistic Regression in In [4]

BoundsError: attempt to access "invalid columns 1:500 selected"

Stacktrace:
 [1] Type at /srv/julia/pkg/packages/DataFrames/IKMvt/src/other/index.jl:280 [inlined]
 [2] Type at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:41 [inlined]
 [3] Type at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:43 [inlined]
 [4] view at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:87 [inlined]
 [5] view at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:82 [inlined]
 [6] datasubset(::DataFrame, ::UnitRange{Int64}, ::LearnBase.ObsDim.Undefined) at /srv/julia/pkg/packages/MLDataUtils/Onazx/src/datapattern.jl:10
 [7] splitobs at /srv/julia/pkg/packages/MLDataPattern/mX21p/src/splitobs.jl:121 [inlined]
 [8] #splitobs#59 at /srv/julia/pkg/packages/MLDataPattern/mX21p/src/splitobs.jl:113 [inlined]
 [9] (::getfield(MLDataPattern, Symbol("#kw##splitobs")))(::NamedTuple{(:at,),Tuple{Float64}}, ::typeof(splitobs), ::DataFrame) at ./none:0
 [10] top-level scope at In[4]:1

In the case of Linear Regression in In [3]

BoundsError: attempt to access "invalid columns 1:22 selected"

Stacktrace:
 [1] Type at /srv/julia/pkg/packages/DataFrames/IKMvt/src/other/index.jl:280 [inlined]
 [2] Type at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:41 [inlined]
 [3] Type at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:43 [inlined]
 [4] view at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:87 [inlined]
 [5] view at /srv/julia/pkg/packages/DataFrames/IKMvt/src/subdataframe/subdataframe.jl:82 [inlined]
 [6] datasubset(::DataFrame, ::UnitRange{Int64}, ::LearnBase.ObsDim.Undefined) at /srv/julia/pkg/packages/MLDataUtils/Onazx/src/datapattern.jl:10
 [7] splitobs at /srv/julia/pkg/packages/MLDataPattern/mX21p/src/splitobs.jl:121 [inlined]
 [8] #splitobs#59 at /srv/julia/pkg/packages/MLDataPattern/mX21p/src/splitobs.jl:113 [inlined]
 [9] (::getfield(MLDataPattern, Symbol("#kw##splitobs")))(::NamedTuple{(:at,),Tuple{Float64}}, ::typeof(splitobs), ::DataFrame) at ./none:0
 [10] top-level scope at In[3]:1

The errors can be seen in

Logistic Regression: https://notebooks.gesis.org/binder/v2/gh/arnim/TuringTutorials/master?filepath=2_LogisticRegression.ipynb

Linear Regression: https://notebooks.gesis.org/binder/v2/gh/arnim/TuringTutorials/master?filepath=/5_LinearRegression.ipynb

The Manifest.toml is in the container linked above at /srv/julia/pkg/environments/v1.0/Manifest.toml and generated from https://github.com/arnim/TuringTutorials/blob/master/binder/REQUIRE

Maybe there is still something wrong with the REQUIRE file. Having a working REQUIRE file would allow potential users to execute your tutorials with 1-click in the cloud and play around with them online.

cpfiffer commented 5 years ago

You need to update the notebooks on Binder. They have changed to work around those errors. As I mentioned, none of this is an issue with the REQUIRE file.

cpfiffer commented 5 years ago

I think I understand that you're asking for us to add a REQUIRE file to the repo, which I can do. I'll add it shortly.

arnim commented 5 years ago

@cpfiffer Having a REQUIRE file to the repo would be great ;)

cpfiffer commented 5 years ago

I've added a REQUIRE file to the repo which should work for all the tutorials (hopefully). I've also attached a Binder tag to the repo in case anyone wants to fiddle around with the notebooks quickly.

arnim commented 5 years ago

@cpfiffer nice - THX ;)