Cannot use `future`/`future_lapply` and `box` together: `Error in env$.packageName : name '.packageName' not found in 'env' Calls: <Anonymous> ... cleanup.Globals -> isPackageNamespace -> $ -> $.box$mod Execution halted`

[dereckmezquita]

Error description

I'm a big fan of this module system. There are just a few tools which are absolutely necessary which if I cannot use with box then it is a deal breaker: S4 and future.

Minimal example

prependPrint.R

#' @export
prependedPrint <- function(...) {
    message(paste("Hello there", ..., sep = ", "))
}

index.R just some code to show an example:

box::use(./utils/R/prependedPrint[ prependedPrint ])

future::plan(strategy = "multisession", workers = 3)

future.apply::future_lapply(iris, \(iris_col) {
    prependedPrint(iris_col)

    return(iris_col[1])
}

Error in env$.packageName : name '.packageName' not found in 'env'
Calls: <Anonymous> ... cleanup.Globals -> isPackageNamespace -> $ -> $.box$mod
Execution halted

R version

R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin21.3.0 (64-bit)
Running under: macOS Monterey 12.2.1

Matrix products: default
BLAS:   /opt/homebrew/Cellar/openblas/0.3.20/lib/libopenblasp-r0.3.20.dylib
LAPACK: /opt/homebrew/Cellar/r/4.2.0/lib/R/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.2.0    tools_4.2.0       data.table_1.14.2

‘box’ version

1.1.2

[klmr]

This appears to be an issue in the ‘globals’ package: the error occurs in the function globals::isPackageNamespace. I’m not familiar with that package (or the futureverse) so I’m not entirely sure what the purpose of this function is (or why it exists at all, since isNamespace provides very similar functionality). But the function contains the following code:

    packageName <- env$.packageName

This will fail if env is a ‘box’ module reference, since ‘box’ intentionally performs strict name lookup: subsetting a module with a non-existent name raises an error rather than silently returning a bogus value (NULL). I’ll re-raise the bug report in the ‘globals’ package, let’s see if we can collaboratively find a solution.

[klmr]

Closing this issue as the upstream bug in the ‘globals’ package has been fixed and this should therefore now work with the next version of ‘globals’.

[HenrikBengtsson]

@dereckdemezquita , thanks for reporting. I've fixed this in the develop branch of globals;

remotes::install_github("HenrikBengtsson/globals", ref="develop")

Could you please retry and verify it now works for you? Then, I'll submit globals to CRAN as soon as possible after that.

[dereckmezquita]

Wonderful I am a big fan of both you and your work! So thank you very much :)

@HenrikBengtsson will re-run my code with the new version of globals and report back.

@klmr I want to ask, what was your process for diagnosing and debugging this issue? I tried to open box source code but I got lost as I'm not familiar with your project structure, makefile etc.

[klmr]

Project familiarity helps a lot, as does recognising the exact error message. After that, it’s a matter of looking at the stack trace — I reran the reproducible example (thanks for that!) inside an interactive session and afterwards looked at the stack trace generated by traceback(). This allowed me to find out how my code was being called and work backwards from there.

Another great tool is debug() — putting a break point into a function before an error is raised. I used this to understand the code flow that led to the error, since I wasn’t familiar with either the ‘future.apply’ or the ‘global’ package. Lastly, I grabbed the source code of both packages from the respective GitHub pages, and checked the function documentation to understand what’s happening here.

[dereckmezquita]

I often use traceback() along with browser() - however I wasn't able to build box after inserting browser() in the code - I am not familiar with makefiles.

I will checkout debug() I am not familiar with it.

Thank you for that!