Error: Failed to retrieve the result of MulticoreFuture (future_lapply-21) in FindIntegrationAnchors

[vertesy]

 tic(); anchors <- FindIntegrationAnchors(object.list = ls.Seurat, dims = 1:p$'n.CC', reduction = "rpca"); toc(); say();

Computing 2000 integration features
Scaling features for provided objects
Error: Failed to retrieve the result of MulticoreFuture (future_lapply-21) from the forked worker (on localhost; PID 85677). Post-mortem diagnostic: No process exists with this PID, i.e. the forked localhost worker is no longer alive.
In addition: There were 19 warnings (use warnings() to see them)

[vertesy]

Missing step

features <- SelectIntegrationFeatures(object.list = ls.Seurat)
ls.Seurat <- lapply(X = ls.Seurat, FUN = function(x) {
  x <- ScaleData(x, features = features, verbose = FALSE)
  x <- RunPCA(x, features = features, verbose = FALSE)
})

[vertesy]

Integrating data
Merging dataset 29 32 18 34 33 35 36 37 38 into 19 21 20 25 27 26 28 30 22 23 24
Extracting anchors for merged samples
Finding integration vectors
Finding integration vector weights
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Integrating data
Merging dataset 31 into 2 4
Extracting anchors for merged samples
Finding integration vectors
Finding integration vector weights
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Integrating data
Merging dataset 15 8 7 6 9 10 17 16 12 13 14 1 5 3 11 into 19 21 20 25 27 26 28 30 22 23 24 29 32 18 34 33 35 36 37 38
Extracting anchors for merged samples
Finding integration vectors
Error in validityMethod(as(object, superClass)) :
  long vectors not supported yet: ../../src/include/Rinlinedfuns.h:535
In addition: Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,  :
  there is no package called ‘MarkdownReports’

[vertesy]

Using rPCA

**************************************************|
Integrating data
Merging dataset 31 into 2 4
Extracting anchors for merged samples
Finding integration vectors
Finding integration vector weights
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Integrating data
Merging dataset 15 8 7 6 9 10 17 16 12 13 14 1 5 3 11 into 19 21 20 25 27 26 28 30 22 23 24 29 32 18 34 33 35 36 37 38
Extracting anchors for merged samples
Finding integration vectors
Error in validityMethod(as(object, superClass)) :
  long vectors not supported yet: ../../src/include/Rinlinedfuns.h:535
>   isave.RDS(combined.obj, suffix = "rPCA",inOutDir = T)

[vertesy]

Hello @ -sbdaxia, did you find any solution?

There is also no logical reference datasets in my case.

1. I have 38 10X datasets, normalize them old school (not SCT)
2. I calculate the anchors (rPCA, CCA both work)
3. IntegrateData fails every time after running multicore for several hours

Tried and did not help:

1. Calculate Anchor set with reverse pCA
2. Reduce the nr. of genes to ~2500 in features.to.integrate (in IntegrateData).
3. Reduce the dimensions from 50 to 30 in features.to.integrate (in IntegrateData).

Definitions from help page

--	IntegrateData
features	Vector of features to use when computing the PCA to determine the weights. Only set if you want a different set from those used in the anchor finding process
features.to.integrate	Vector of features to integrate. By default, will use the features used in anchor finding.

[shahrozeabbas]

Hello, @vertesy how many cores and how much memory are you using?

In my experience, if you lower the number of cores or increase memory, the issue is solved. I believe one of the forked nodes might be running out of memory by the time your job gets to calling IntegrateData.

[vertesy]

Hi @shahrozeabbas , I tried many different combinations, and I cant fully recall how I solved it, but it may have been with the way you proposed! Thank you, Abel