mllg / batchtools

Tools for computation on batch systems
https://mllg.github.io/batchtools/
GNU Lesser General Public License v3.0
169 stars 51 forks source link

Bugfix in waitForFile #276

Closed stuvet closed 2 years ago

stuvet commented 2 years ago

Background

Reprex

library(rbenchmark)
library(data.table)

fn <- fs::file_create(tempfile())
fn_short <- basename(fn)
path <- fs::path_dir(fn)

fs::file_exists(fn)
#> /tmp/RtmpZqNq5M/file1f8ca1b4f66d4 
#>                              TRUE

Existing strategy

fn_short %chin% list.files(path, all.files = TRUE)
#> [1] TRUE
fn %chin% list.files(path, all.files = TRUE)
#> [1] FALSE

Proposed strategy

basename(fn_short) %chin% list.files(path, all.files = TRUE)
#> [1] TRUE
basename(fn) %chin% list.files(path, all.files = TRUE)
#> [1] TRUE

Benchmarks

benchmark(
  fs::file_exists(fn),
  basename(fn) %chin% list.files(path, all.files = TRUE), 
  fs::path_file(fn) %chin% list.files(path, all.files = TRUE), 
  fn %chin% list.files(path, all.files = TRUE, full.names = TRUE), # Fails if fn is already a basename
  replications = 10000
)
#>                                                              test replications elapsed relative user.self sys.self user.child sys.child
#> 2          basename(fn) %chin% list.files(path, all.files = TRUE)        10000  0.121    1.090     0.083    0.036          0         0
#> 4 fn %chin% list.files(path, all.files = TRUE, full.names = TRUE)        10000  0.111    1.000     0.060    0.049          0         0
#> 1                                             fs::file_exists(fn)        10000  8.339   75.126     6.419    1.425          0         0
#> 3     fs::path_file(fn) %chin% list.files(path, all.files = TRUE)        10000  0.319    2.874     0.266    0.050          0         0

Created on 2021-07-24 by the reprex package (v2.0.0)

mllg commented 2 years ago

LGTM, thanks!