search

r-lib / testthat

An R 📦 to make testing 😀
https://testthat.r-lib.org
Other
888 stars 318 forks source link

Parallel testing in Azure DevOps Pipelines #1976

Open kummerer94 opened 2 months ago

kummerer94 commented 2 months ago

Hi,

first of all: thank you for your work on this package. It has made testing code in R so much more pleasant.

We upgraded to testthat edition 3 a few days ago to use the parallel test runner. To make this work, we followed the tutorial and locally (Windows) everything works fine.

Once we changed our Azure DevOps pipeline under Linux to use the parallel runner, we ran into the following error:

ℹ Testing xxx
Starting 2 test processes
✔ | F W  S  OK | Context

⠋ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠙ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠹ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠸ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠼ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠴ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠦ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠧ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠇ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠏ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠋ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠙ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...                          
⠹ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] Starting up...

I start my tests with devtools::test().

The Starting up... is the only log output that I see. No tests will be run and the pipeline task will timeout after however many minutes are configured as max time for a task.

It's difficult for me to setup a completely reproducible example as CI/CD pipelines tend to be quite custom, so my question / issue is: Is there a way to enable advanced debugging / logging for testthat? It would help if I could get a sense of what is happening when the new processes start.

I suspect that the process that should start in the background to run the tests fails on startup and so we are in this endless loop of trying to start a new process to run the tests but this process exits immediately.