search

TuringLang / ParetoSmooth.jl

An implementation of PSIS algorithms in Julia.
http://turinglang.org/ParetoSmooth.jl/
MIT License
19 stars 12 forks source link

Make MCMCDiagnosticTools an optional dependency #42

Open sethaxen opened 3 years ago

sethaxen commented 3 years ago

PSIS is generally useful for diagnosing importance sampling, but importance weights are not always generated with MCMC. Since MCMCDiagnosticTools is a somewhat heavy dependency, it's a shame it makes this package less lightweight for non-MCMC applications.

We could instead make MCMCDT a conditional dependency using Requires. I propose something like the following methods:

psis(log_ratios; kwargs...): computes PSIS assuming r_eff=1 psis(log_ratios, r_eff; kwargs...): computes PSIS using the user-provided r_eff. psis(log_ratios; ess_method::MCMCDiagnosticTools.AbstractESSMethod, kwargs...): defined if MCMCDiagnosticTools.jl is loaded using Requires; computes r_eff using the provided ESS method and calls psis(log_ratios, r_eff; kwargs...)

devmotion commented 3 years ago

Since MCMCDiagnosticTools is a somewhat heavy dependency, it's a shame it makes this package less lightweight for non-MCMC applications.

Even without the MCMCDiagnosticTools dependency this package is far from being lightweight: https://github.com/TuringLang/ParetoSmooth.jl/blob/425404f47d679e8859c83c8b288c55fe8fbf7f36/Project.toml#L7-L24 Maybe this could/should be changed but since I haven't used ParetoSmooth and don't plan to in the near future, others should discuss and decide this.

sethaxen commented 3 years ago

Yes, this is one of several dependency-related issues, see #44

devmotion commented 3 years ago

Since MCMCDiagnosticTools is a somewhat heavy dependency, it's a shame it makes this package less lightweight for non-MCMC applications.

Actually, I think it is not a heavy dependency: https://github.com/TuringLang/ParetoSmooth.jl/issues/44#issuecomment-924419064

sethaxen commented 3 years ago

Hm, I seem to recall it taking longer to load in the past. I wonder if invalidations were recently fixed in a dependency.