write_csv freezes/fails when writing to many files in a short amount of time.

[nzaidel]

readr::write_csv is not completing when I try to write 100 csv files in a loop. It writes some of the files and then locks up requiring a restart or R. The number of files it successfully writes varies each time I try and run.

In summary, the code I am trying to run works with small files or a small loop, but it fails with moderate sized files or loops because something is getting overwhelmed when I try an write out a lot of files in a short amount of time.

I have been trying to utilize the Sys.sleep function to alleviate but it has not been succesfull.

base R write.csv does work successfully when I run this loop.

Thank you for your attention!

library(readr)
library(purrr)
library(tibble)
library(strigr)
sessionInfo()

test_function <- function(outname){

  df <- tibble::tibble(a=1:10000, b = sample(letters, 10000, replace = TRUE))

  df %>% readr::write_csv(stringr::str_c("C:/Users/nathaniel.zaidel/test/out_", outname, ".csv"))
}

purrr::walk(1:100, test_function)

sessionInfo() R version 4.3.1 (2023-06-16 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8

time zone: America/New_York tzcode source: internal

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] tibble_3.2.1 purrr_1.0.2 readr_2.1.4

loaded via a namespace (and not attached): [1] utf8_1.2.4 R6_2.5.1 tzdb_0.4.0 magrittr_2.0.3 glue_1.6.2
[6] pkgconfig_2.0.3 lifecycle_1.0.4 cli_3.6.1 fansi_1.0.5 vctrs_0.6.4
[11] compiler_4.3.1 rstudioapi_0.15.0 tools_4.3.1 hms_1.1.3 pillar_1.9.0
[16] rlang_1.1.2