using multiple cores

[MuBulut]

Hi

I have question concerning the ncpus option: Whenever I start my R-script (on a multi core) machine, it seems, that there is never more than 1 cpu utilized, The load is always around 1 with one thread at 100%.

I tried to specify ncpus=10 and also commenting out the option (which would mean all cores found should be used, right?), but could not see any change in behaviour.

Could you please comment on how to make sure (and check), that more than one cpu is used by rMVP?

[hyacz]

I usually check that the scripts are properly parallelized via a monitoring tool (e.g. top). can you show me the OS and R version, and the script?

[MuBulut]

Top is exactly the tool I used and realized, the load is never more than 1 and only 1 thread seems to be running. OS: Ubuntu Linux 20.04 R: 4.1.2

the script:

cleaned rMVP

INSTALLATION

library("rMVP") library("bigmemory")

setwd("/data/adonut/testdata")

options(bitmapType="cairo")

Files must called:

Phenotype

DATA PREPARATION

MVP.Data(fileVCF = "SNP.vcf", sep.vcf="\t", sep.phe="\t", filePhe = "all_traits_phenotype_log_rMVP.txt", out="mvp")

KINSHIP

Calculate from genofile

MVP.Data.Kin(TRUE, mvp_prefix = 'mvp', out = 'mvp', priority = "speed", sep = "\t")

Principal components

Calculate from genofile

MVP.Data.PC(TRUE, mvp_prefix = 'mvp', pcs.keep = 5, sep = "\t", priority = "speed" ) #play around with settings

genotype <- attach.big.matrix('mvp.geno.desc') phenotype <- read.table("mvp.phe", header = TRUE) #at which step should I prepare this??? kinship <- attach.big.matrix('mvp.kin.desc') map <- read.table("mvp.geno.map" , head = TRUE)

don't use covariates if you use GLM, MLM or FarmCPU

Covariates_PC <- bigmemory::as.matrix(attach.big.matrix('mvp.pc.desc'))

if you want to add additional covariates (like breed, replicate, environment etc.)

Covariates <- model.matrix(~as.factor(breed)+as.factor(sex)+as.numeric(weight), data=yourdata)

when PC should be added to covariates

Covariates <- cbind(Covariates, Covariates_PC)

STARTING GWAS

For multiple phenotypes

for(i in 2:ncol(phenotype)){ imMVP <- MVP( phe = phenotype[, c(1, i)], #phenotype data geno = genotype, #genotype data K = kinship, map = map, #mapping data --> i dont have it since i use vcf-files

CV.GLM=Covariates,

CV.MLM=Covariates,

CV.FarmCPU=Covariates,

nPC.GLM=5,

nPC.MLM=3, nPC.FarmCPU=3, priority="speed", ncpus=50, vc.method="HE", #"BRENT", "EMMA", and "GEMMA" maxLoop=10, #maximum iterations allowed in FarmCPU method.bin="FaST-LMM", #"FaST-LMM","EMMA", "static"

permutation.threshold=TRUE,#threshold of permutation will be used in manhattan plot

#95% quantile value of this vector is recommended to be used as significant threshold

permutation.rep=100,

threshold=c(0.5, 0.05), #0.05/marker size, a cutoff line on manhattan plot

how can I change it to effective size of independent marker???

method=c("MLM", "GLM", "FarmCPU") #are there more? --> no )

gc() }

[hyacz]

There is a line like this in MVP's log output

Number of threads used: <threads>

Can you check the number here?

I guess it may be that the OpenMP environment is abnormal or the use of threads is restricted by the system level