Foreach is unable to access custom atlas object in make_brainGraphList()

[soffiafdz]

Hello, @cwatson. I'm trying to build several graphs of BOLD data using different atlases. The first two run with no issues (as they're both implemented in your latest version), but when I get to the third make_brainGraphList stops and throws: Error in { : task 1 failed - "object 'schaefer100x7' not found".

I found another similar issue [#4], but I still can't resolve my problem. I followed the User Guide in building the data.table atlas object (it has the required columns and then I ran it hrough as_atlas). It is accessible in my environment and has the same name as the atlas string object that is used as an argument for make_brainGraphList. When I run get(atlas) before running make_brainGraphList, the data.table comes without any issue. I tried saving it as an .rda and then loading it again. Nothing seems to work.

When I ran the function after using debug(), I could see that it crashes just as it reaches the foreach (%dopar%) loop; but it is so deep in the make_brainGraph method, that I don't know how to fix it myself.

Any help would be greatly appreciated.

[cwatson]

Can you try putting the following code in your script before the call to make_brainGraphList: .GlobalEnv$schaefer100x7 <- schaefer100x7

[soffiafdz]

Yeah, no, it's still not working on my end

>source(here("scripts/chaefer.R"))
> rds_dir     <- here("data/processed/rds")
> covars      <- read_rds(here(rds_dir, "covars.rds"))
> thresholds  <- read_rds(here(rds_dir, "thresholds.rds"))
> mats        <- read_rds(here(rds_dir, "mats.rds"))
> A.norm.mean <- mats[[1]][[1]][[3]]$A.norm.mean
> atlas       <- "schaefer100x7"
> head(get(atlas))
                 name x.mni y.mni z.mni lobe hemi index network
1: 7Networks_LH_Vis_1   -26   -34   -18   OCC    L     1     Vis
2: 7Networks_LH_Vis_2   -26   -76   -14   OCC    L     2     Vis
[...]
> find(atlas)
[1] ".GlobalEnv"
> .GlobalEnv$schaefer100x7 <- schaefer100x7
> g <- make_brainGraphList(A.norm.mean[[21]], atlas, modality="fmri", 
threshold = thresholds[[1]][21], weighted = T, gnames = gnames)
[1] "Start time: 2021-03-01 21:36:32"
  |                                                                      |   0%Error in { : task 1 failed - "object 'schaefer100x7' not found"

[cwatson]

What system are you running on? Can you return the output of sessionInfo()? This may be a Windows-related issue.

[soffiafdz]

I'm running on a cluster, which has CentOS. Because of the files sizes, I haven't tried using my personal laptop, but that runs Linux, too (Void), so I don't think it would make much of a difference.

> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS:   /cm/shared/apps/r/4.0.2-studio/lib64/R/lib/libRblas.so
LAPACK: /cm/shared/apps/r/4.0.2-studio/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_4.0.2

[cwatson]

The only other quick fix I can think of is to edit the function and change the foreach line to read g <- foreach(i=seq_len(kNumGraphs), .export=atlas) %dopar% { although this may be more difficult if you are on a cluster.

I am also not sure if it will matter, but you can try with .progress=FALSE.

[soffiafdz]

I'll try adding the .export argument to the foreach function. I did have a hard time accessing the inner functions and methods, but I'll do my best.

And yeah, I left the progress bar in the script I ran interactively, but in the Rscript I send as a job I have disabled the progress bar for a cleaner log and it doesn't make a difference.

[cwatson]

Closing due to inactivity and because I'm not sure I'd be able to fix this properly. Please feel free to reopen the issue if it is important and I will look into it.