Open asfimport opened 2 years ago
Jonathan Keane / @jonkeane:
One way to do this would be to add something like the below to the file r/tests/testthat/helper-arrow.R
(which is where we had previously disabled those until ARROW-8379 was merged.
if (tolower(Sys.info()[["sysname"]]) == "windows") {
options(arrow.use_threads = TRUE)
}
Neal Richardson / @nealrichardson: I think what I would do is replace the .onLoad block with something that sets the cpu_count and io_thread_count to some fraction of Ncpus, so we'd have some parallelism but hopefully in a safe way.
Jonathan Keane / @jonkeane: That also works — for testing in CI, I would recommend using the max cpu_count and io_thread_count we can to have the highest chance of running into deadlocks (unless we conclusively prove that having fewer eliminates the deadlocks totally).
Dewey Dunnington / @paleolimbot: Another failure with a similar failure is here: https://github.com/apache/arrow/runs/4084014846?check_suite_focus=true#step:17:667
Sam Albers / @boshek:
Though I don't have time right now to distill into a minimal reprex (in fact it is sort of the opposite; sorry!), the targets
pipeline in this repo reliably (at least on my machine) experiences an issue with multithreading on Windows to extent that I did have to add this in to stop it from simply hanging on Windows. I'm not sure how helpful this is but up to this point I have to reliably reproduce this issue.
Followup to ARROW-8379, which set use_threads = FALSE on Windows. See discussion about adding more controls, disabling threading in some places and not others, etc. We want to do this soon after release so that we have a few months to see how things behave on CI before releasing again.
Collecting some CI hangs after ARROW-8379
https://github.com/apache/arrow/pull/11290/checks?check_run_id=3767787034
Reporter: Neal Richardson / @nealrichardson
Note: This issue was originally created as ARROW-14159. Please see the migration documentation for further details.