search

czarrar / connectir

Fast and memory efficient approach for connectome-wide association studies in R and C++. Uses custom implementation of multivariate distance matrix regression to associate brain and behavior.
15 stars 9 forks source link

MDMR issue with bigmemory_CCleanindices #9

Open maxwellelliott opened 5 years ago

maxwellelliott commented 5 years ago

Below is my call to connectir_mdmr.R and the output. I have tried to track down the error and it seems to be an issue with CCleanIndices in bigextensions/R/utils.R . I think that bigmemory has updated something since this fix 2 years ago (https://github.com/czarrar/bigextensions/commit/14a11f8e38c531270cb5a258405db3bff020c592). I tried to fix it myself but can't figure it out.

/dcc/Scripts/Tools/CWAS/connectir_mdmr.R -i /dcc/Projects/Max/DBIS/pFactor/CWAS/subdist.n746_060719 --formula 'sex + p' --model /dcc/Projects/Max/DBIS/pFactor/lists/model_746_wFunc_p38_060719.csv --factors2perm 'p' --memlimit 5 -c 1 --save-perms -p 100 --ignoreprocerror --skip-subdist-check --debug mdmr.n746_060719

Loading niftir version < 1.0.

Setting 1 parallel forks Setting 1 threads for matrix algebra operations Checking options Setting up inputs ...data dimensions ...formula ...factors to permute ...factors2perm ...model ...creating output directory '/dcc/Projects/Max/DBIS/pFactor/CWAS/subdist.n746_060719/mdmr.n746_060719' Reading in subject distances ...checking input Determining MDMR memory demands ...1.1 MB used for permutation indices ...0.0 MB used for p-values ...minimum of 17.0 MB used for distance matrices ...minimum of 0.0 MB used for permuted pseudo-F stats ...minimum of 8.5 MB used for permuted model matrices ...minimum of 8.5 MB used for permuted error matrices ...minimum of 0.0 MB used for temporary matrices ...memory limit is 5.00 GB and a minimum of 0.043 GB is needed ...setting super-block size to 376 (out of 376 voxels) ...setting block size to 101 (out of 101 permutations) ...3.98 GB of RAM will be used Generating hat matrices and like (with mdmr_model) ...creating right-hand design matrix ...calculating QR decomposition p 2 [1] 1 ...checking/adjusting for rank deficiencies ...creating the hat matrices (H2s and IHs) ...calculating degrees of freedom ...combining elements to return Preparing data ...preparing permutation matrices Gathering permuted observation indices ...for factor p (#1) |======================================================================| 100% ...preparing file-backed pesudo-F matrices Computing MDMR across 1 large blocks ...with 1 smaller blocks within each larger one large block 1 ...grabbing subset of Gower matrices ...preparing partial pesudo-F matrices ...looping through 1 permutation block(s) | | 0%...preparing subset of permutation indices Generating permuted hat matrices (H2s) ...for factor p (#1) Generating permuted hat matrices (IHs) ...for factor p (#1) ...calculating Pseudo-F stats ...removing permutation indices Error in .Call("bigmemory_CCleanIndices", as.double(rows), as.double(nr), : "bigmemory_CCleanIndices" not available for .Call() for package "bigmemory"

An error was detected: with piece 1: [1] 1

Called by: NULL

Saving options...

Removing everything from memory ...sucesss

czarrar commented 5 years ago

I do not actively maintain the package anymore. Did you try installing an older version of bigmemory: 4.4.6?

Also did you use voxelwise or parcels? If you used parcels, I could recommend a native R version of MDMR.

maxwellelliott commented 5 years ago

Got it. Thanks for the advice.

I reinstalled an older version of bigmemory and the command worked again!

dseok commented 5 years ago

Which version of bigmemory did you end up installing? I tried 4.4.6 and 4.4.5 and I'm still getting the same error you are.

maxwellelliott commented 5 years ago

Here's the setup that I have right now that works most of the time for me:

bigmemory_4.5.19 bigmemory.sri_0.1.3

On Sat, Oct 5, 2019 at 6:13 PM dseok notifications@github.com wrote:

Which version of bigmemory did you end up installing? I tried 4.4.6 and 4.4.5 and I'm still getting the same error you are.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/czarrar/connectir/issues/9?email_source=notifications&email_token=ACK4CCT5YEGF5Y6GJ3FQWPLQNEGQ3A5CNFSM4HV4JAL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAN45UA#issuecomment-538693328, or mute the thread https://github.com/notifications/unsubscribe-auth/ACK4CCXFA542CAG73Z7OW7TQNEGQ3ANCNFSM4HV4JALQ .