Breakpoints not working with `devtools::test` + `testthat`

[katrinabrock]

NOTE: Before submitting an issue, please make sure to install the latest version of both the vscode extension and the R package. This can usually be achieved by running the command r.debugger.updateRPackage in vscode's command palette (F1).

Describe the bug devtools::test() doesn't trigger breakpoints in testing file or package.

MRE

1. Open this repo as an r project with vsc r debugger enabled: https://github.com/katrinabrock/minimal-r-package
2. Open tests/testthat/test-output.R).
3. Add some breakpoints.
4. Run the Debug testthat test config. (This runs devtools::test(filter='output'))

Your Launch config If applicable, the launch config that causes the bug. E.g.:

https://github.com/katrinabrock/minimal-r-package/blob/main/.vscode/launch.json#L15-L23

Expected behavior Dropped into debug console at breakpoints.

Actual behavior Tests run without stopping at breakpoints.

Desktop (please complete the following information):

• OS: Desktop is Mac OS, using VS Code Remote..remote (where R is running) is Ubuntu
  • R Version: 4.3.2
  • VSCode version: 1.92.2
• vscDebugger Version: 0.5.3
• vscode-r-debugger Version: v0.5.4

Additional context

• Breakpoints in the code that calls devtools::test (in this case example2.R): working ✅
• Breakpoints in the test files themselves: not working ❌
• Breakpoints in package functions called within the tests: not working: ❌
• Direct browser calls: working everywhere ✅ I believe this is failing because testthat runs pkgload::load_all(): https://github.com/r-lib/testthat/blob/fe38519d72247a8907228a4ecaf926483aa5d4ff/R/test-files.R#L235 and that call is not successfully getting overwritten with vscDebugger::.vsc.load_all

[katrinabrock]

thanks. I didn't click the link. :-)

[ManuelHentschel]

Thanks for raising this. Your assumption about pkdload::load_all() sounds reasonable, but I'm not sure if there is an easy way to fix this. I'll have a look, though.

[ManuelHentschel]

Update: The breakpoints that you indicated in the picture above (i.e. in tests/testthat/...) would be quite hard to implement, because they are neither in the package, nor in the normal R code. Instead they are loaded and executed by testthat in a rather complicated fashion, which would be difficult to modify reliably.

One way around this is to define the test-code in a separate function, which is part of the main script, and then call that function from your test file:

# [In main file (example2.R)]
f <- function() {
    cat('Alice!\n')    # <- Breakpoint here
}

# [In tests/testthat/test-output.R]
test_that('Output Test #3', {
  f()
  expect_true(TRUE)
})

In order to set breakpoints in package code, adding the following sinippet at the beginning of your example2.R file and removing "debuggedPackages": [ "temp" ] from the launch config seems to yield the desired behavior. In more complex scenarios, e.g. involving multiple packages, this might lead to undesired behavior, though.

env <- loadNamespace('pkgload')
unlockBinding('load_all', env)
env$load_all <- function(...) cat('Ignoring pkgload::load_all!\n')
lockBinding('load_all', env)

[katrinabrock]

@ManuelHentschel

Thanks for the response!

I am hesitant to move any of the testing logic out of the test file and into example2.R because ultimately example2.R is a short-lived launcher script. However, I could move the logic to a helper file, and source the helper file in main/launcher script (example2.R).

The second option of putting the breakpoints in the package did work. I've updated my example package to demonstrate this (https://github.com/katrinabrock/minimal-r-package/commit/1dd6f51bd749127fd33b9650e8f8cbb1224c3823). With a breakpoint like this:

I still think there may be a cleaner way to achieve this...like some magical combination of Config/testthat/load-all: list(...) in the DESCRIPTION file, but I haven't found it yet. :-/

[ManuelHentschel]

I am hesitant to move any of the testing logic out of the test file and into example2.R because ultimately example2.R is a short-lived launcher script. However, I could move the logic to a helper file, and source the helper file in main/launcher script (example2.R).

Breakpoints in helper files seem to work if you prevent source_test_helpers from reloading the file and thus overwriting the breakpoints:

env <- loadNamespace('testthat')
unlockBinding('source_test_helpers', env)
env$source_test_helpers <- function(...) cat('Ignoring testthat::source_test_helpers!\n')
lockBinding('source_test_helpers', env)

However, if I understood correctly, the code from test-XXX.R scripts is loaded here. Modifying that would get quite hacky, so I don't see an easy way to have breakpoints in these files for now.

[katrinabrock]

Getting breakpoints working the package and helpers is already a big help. Thanks :-)