This PR improves the Pareto k-value warnings by suppressing the original warnings from the loo package and throwing customized warnings instead (thank you, @n-kall, for the suggestion).
While implementing this, I realized that if the projected draws used for the performance evaluation in the validate_search = FALSE case have different weights, then loo::sis() needs to be used instead of loo::psis() (because the weights would need to be taken into account when performing the Pareto smoothing). Hence, this fallback to loo::sis() is implemented here as well (and apart from that, we now use loo::sis() explicitly if loo::psis() would not perform the Pareto smoothing due to a small number of draws, but this is merely a safety measure, see the inline comments added in the code).
In order to differentiate between different warnings in the unit tests, other warnings are now thrown conditionally on their own "hidden" global options.
Some enhancements for capture.output() usage in fit_cumul() and fit_cumul_mlvl() are included as well (found when starting to use capture.output() in loo_varsel()).
Also added is a unit test that the warning for full Gaussian multilevel models added in #426 is thrown correctly (this test required the differentiation between different warnings, so it is included here).
This PR improves the Pareto k-value warnings by suppressing the original warnings from the loo package and throwing customized warnings instead (thank you, @n-kall, for the suggestion).
While implementing this, I realized that if the projected draws used for the performance evaluation in the
validate_search = FALSEcase have different weights, thenloo::sis()needs to be used instead ofloo::psis()(because the weights would need to be taken into account when performing the Pareto smoothing). Hence, this fallback toloo::sis()is implemented here as well (and apart from that, we now useloo::sis()explicitly ifloo::psis()would not perform the Pareto smoothing due to a small number of draws, but this is merely a safety measure, see the inline comments added in the code).In order to differentiate between different warnings in the unit tests, other warnings are now thrown conditionally on their own "hidden" global options.
Some enhancements for
capture.output()usage infit_cumul()andfit_cumul_mlvl()are included as well (found when starting to usecapture.output()inloo_varsel()).Also added is a unit test that the warning for full Gaussian multilevel models added in #426 is thrown correctly (this test required the differentiation between different warnings, so it is included here).