Feature request: progress indicator

lindeloev commented 5 years ago

loo has been running for four hours now on seven cores. I have no idea when it will finish, so intermittent progress reports would be great (e.g., every 10%). Specifically, I'm using brms::LOO(brmsfit) which I believe calls loo in the background.

avehtari commented 5 years ago

1) Can you report devtools::session_info("loo") If you are using Windows, check if this known but obscure problem in Windows multi core is affecting you https://discourse.mc-stan.org/t/problem-parallelizing-rstanarm-loo/6433/10 .

2) We could give quite good estimate of the total time after the first loo fold, and report if the total estimated time is more than a minute.

lindeloev commented 5 years ago

Indeed, I fell victim to the multi-core bug. Now it completes in around 4 minutes on one core (setting options(mc.cores = NULL)). Than you so much!

Yes, it would be nice-to-have an estimate of long completion times.

Output of devtools::session_info("loo"):

- Session info ---------------------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.5.1 (2018-07-02)
 os       Windows >= 8 x64            
 system   x86_64, mingw32             
 ui       RStudio                     
 language (EN)                        
 collate  English_United States.1252  
 ctype    English_United States.1252  
 tz       Europe/Paris                
 date     2018-12-30                  

- Packages -------------------------------------------------------------------------------------------------------------------------
 package     * version date       lib source        
 loo           2.0.0   2018-04-11 [1] CRAN (R 3.5.1)
 matrixStats   0.54.0  2018-07-23 [1] CRAN (R 3.5.1)

jgabry commented 5 years ago

I really like the idea of a progress bar (an optional progress bar, since it would add a bit of overhead, although not a ton).

I really don’t know much of what’s available for parallel progress bars other than the pbapply package, so I’ll have to look into it. And if anyone has suggestions please chime in.

avehtari commented 5 years ago

Seems easy to test with pbapply. Should define min_time.

stan-dev / loo

Feature request: progress indicator #91