abess-team / abess

Fast Best-Subset Selection Library
https://abess.readthedocs.io/
Other
474 stars 41 forks source link

Cross validation is slower in version 0.4.5 than in 0.4.0 #409

Closed brtang63 closed 1 year ago

brtang63 commented 2 years ago

Describe the bug

In my experiments, after updating abess from 0.4.0 to 0.4.5, I found the cv procedure get slower in some cases. The following code provides an example.

Code for Reproduction

library(microbenchmark)
library(abess)
n <- 3000
p <- 500
support.size <- 10

sim_once <- function(seed) {
  dataset <- generate.data(n, p, support.size, family = "binomial", seed = seed)

  time_cv <- microbenchmark(
    abess_fit <- abess(dataset[["x"]], dataset[["y"]], family = "binomial", tune.type = "cv", nfolds = 10),
    times = 1
  ) [["time"]] / 10^9

  time_cv
}

# average time
time <- sapply(1:5, sim_once)
mean(time)

image

bbayukari commented 2 years ago

I run your code with these two versions program on my computer, and the results are:

0.4.0 0.4.5
201.4724 220.9249
262.5131 253.5498
238.9165 266.0888
259.1678 246.1789
233.7465 238.2309

v0.4.5 seems slower by 2.5% than v0.4.0, but I don't find any changes which may cause this.

There is no such colossal performance difference in my repeated experiments as your results. They are generally about 10%.

Thanks for your feedback, and we will continue to focus on this question.