search

tidymodels / tidymodels.org

Source of tidymodels.org
https://www.tidymodels.org/
19 stars 15 forks source link

tune-text changer speed-up #27

Closed EmilHvitfeldt closed 6 months ago

EmilHvitfeldt commented 1 year ago

This page takes TOO long to render.

EmilHvitfeldt commented 6 months ago

total runtime: 1451.634 seconds

# A tibble: 53 × 3
   file                                                  time percentage
   <chr>                                                <dbl>      <dbl>
 1 learn/work/tune-text/index.qmd                      807.     0.556   
 2 start/case-study/index.qmd                          127.     0.0873  
 3 learn/work/bayes-opt/index.qmd                       77.9    0.0537  
 4 start/recipes/index.qmd                              63.9    0.0440  
 5 learn/models/time-series/index.qmd                   39.2    0.0270  
 6 learn/work/nested-resampling/index.qmd               35.3    0.0243  
 7 learn/models/conformal-regression/index.qmd          22.8    0.0157  
 8 start/resampling/index.qmd                           19.6    0.0135  
 9 learn/statistics/survival-metrics/index.qmd          19.5    0.0134  
10 learn/models/parsnip-ranger-glmnet/index.qmd         16.7    0.0115  
11 learn/statistics/survival-metrics-details/index.qmd  13.5    0.00927 
12 start/models/index.qmd                               13.4    0.00925 
13 learn/models/calibration/index.qmd                   11.8    0.00812 
14 learn/statistics/infer/index.qmd                     11.7    0.00803 
15 learn/models/sub-sampling/index.qmd                  11.2    0.00771 
16 start/tuning/index.qmd                               10.1    0.00697 
17 learn/work/fairness-readmission/index.qmd             9.92   0.00684 
18 learn/work/tune-svm/index.qmd                         9.83   0.00677 
19 learn/models/pls/index.qmd                            9.78   0.00673 
20 learn/statistics/xtabs/index.qmd                      8.31   0.00573 
21 learn/models/parsnip-nnet/index.qmd                   7.23   0.00498 
22 learn/models/coefficients/index.qmd                   7.03   0.00484 
23 learn/statistics/bootstrap/index.qmd                  6.72   0.00463 
24 learn/develop/models/index.qmd                        6.47   0.00445 
25 learn/develop/broom/index.qmd                         6.06   0.00418 
26 learn/statistics/k-means/index.qmd                    5.83   0.00402 
27 learn/work/fairness-detectors/index.qmd               5.5    0.00379 
28 learn/develop/recipes/index.qmd                       5.05   0.00348 
29 learn/work/case-weights/index.qmd                     5.01   0.00345 
30 learn/statistics/tidy-analysis/index.qmd              4.91   0.00338 
31 learn/develop/metrics/index.qmd                       4.90   0.00338 
32 learn/develop/parameters/index.qmd                    4.70   0.00324 
33 find/all/index.qmd                                    4.60   0.00317 
34 find/recipes/index.qmd                                4.02   0.00277 
35 find/broom/index.qmd                                  4.01   0.00276 
36 find/parsnip/index.qmd                                3.84   0.00265 
37 learn/index.qmd                                       2.19   0.00151 
38 books/tmwr/index.qmd                                  1.91   0.00132 
39 books/moderndive/index.qmd                            1.91   0.00131 
40 books/tidytext/index.qmd                              1.90   0.00131 
41 books/smltar/index.qmd                                1.85   0.00127 
42 books/fes/index.qmd                                   1.79   0.00123 
43 index.qmd                                             1.62   0.00112 
44 books/index.qmd                                       1.59   0.00110 
45 packages/index.qmd                                    1.52   0.00105 
46 start/index.qmd                                       1.51   0.00104 
47 contribute/index.qmd                                  1.49   0.00103 
48 404.qmd                                               1.44   0.000990
49 help/index.qmd                                        1.42   0.000978
50 about/index.qmd                                       1.42   0.000978
51 find/index.qmd                                        1.42   0.000977
52 about/davis/index.qmd                                 1.40   0.000963
53 about/max/index.qmd                                   1.38   0.000950
EmilHvitfeldt commented 6 months ago

made with following script

qmds <- fs::dir_ls(recurse = TRUE, regexp = "qmd$")

timer <- function(x) {
  time <- system.time(
    system(paste("quarto render", x))
  )
  time[["elapsed"]]
}

times <- purrr::map_dbl(qmds, timer)

library(tidyverse)

tibble(
  file = names(times),
  time = times
) |>
  mutate(
  percentage = time / sum(time)
) |>
  arrange(desc(time)) |>
  print(n = Inf)
EmilHvitfeldt commented 6 months ago

I got the runtime down to around 250 seconds with the changes in https://github.com/tidymodels/tidymodels.org/commit/ba221dc49003c2ea5a7d11a676165f199e2ac46f.

The remaining speed up will come from improvements in {textrecipes}