Open GregorDall opened 10 months ago
Dear Eduardo,
We are trying to use A.mat with 75000k lines and 10k markers, getting the following error:
Error: Mat::init(): requested size is too large; suggest to enable ARMA_64BIT_WORD
Still seems to work with 50k lines, but 75k doesn't work any more. Must be related to the number of individuals, because I can replicate it with only 5 markers too. rrBLUP::A.mat does not seem to have this problem, but I thought sommer::A.mat might be more efficient?I am not sure if that is something related to sommer or Rcpp...
Best Gregor
Hello, I also encountered the same problem as you. Is there any solution?
Dear Eduardo, We are trying to use A.mat with 75000k lines and 10k markers, getting the following error:
Error: Mat::init(): requested size is too large; suggest to enable ARMA_64BIT_WORD
Still seems to work with 50k lines, but 75k doesn't work any more. Must be related to the number of individuals, because I can replicate it with only 5 markers too. rrBLUP::A.mat does not seem to have this problem, but I thought sommer::A.mat might be more efficient? I am not sure if that is something related to sommer or Rcpp... Best GregorHello, I also encountered the same problem as you. Is there any solution?
Hi, my solution was to use A.mat from the rrBLUP package.
Best Gregor
Dear Eduardo, We are trying to use A.mat with 75000k lines and 10k markers, getting the following error:
Error: Mat::init(): requested size is too large; suggest to enable ARMA_64BIT_WORD
Still seems to work with 50k lines, but 75k doesn't work any more. Must be related to the number of individuals, because I can replicate it with only 5 markers too. rrBLUP::A.mat does not seem to have this problem, but I thought sommer::A.mat might be more efficient? I am not sure if that is something related to sommer or Rcpp... Best GregorHello, I also encountered the same problem as you. Is there any solution?
Hi, my solution was to use A.mat from the rrBLUP package.
Best Gregor
Okay thanks for your reply.
Sorry that I never followed up with this issue. I tried some solutions but I haven't succeeded. I am not sure why this happens. It seems to be more related to RcppArmadillo than sommer itself but I will keep trying things and see if I can make it work.
Cheers,
Sorry that I never followed up with this issue. I tried some solutions but I haven't succeeded. I am not sure why this happens. It seems to be more related to RcppArmadillo than sommer itself but I will keep trying things and see if I can make it work.
Cheers,
Hi, the problem seems to be in MNR.cpp, I tried the following changes to avoid such error messages.
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4; indent-tabs-mode: nil; -*-
// we only include RcppArmadillo.h which pulls Rcpp.h in for us
#define ARMA_64BIT_WORD
#define ARMA_DONT_PRINT_ERRORS
#include "RcppArmadillo.h"
#include "stdlib.h"
#include <progress.hpp>
// [[Rcpp::export]]
arma::mat amat(const arma::mat & Xo, const bool & vanraden, double minMAF) {
// remove min.MAF
arma::rowvec pfreq = mean(Xo+1,0)/2; // frequency of p
arma::mat pqfreq = arma::join_cols(pfreq,1-pfreq); // frequencies of p and q
arma::rowvec MAF = min(pqfreq,0); // minor allele freqs
arma::uvec indexMAF = find(MAF > minMAF); // index for good markers > minMAF
arma::mat Xo2 = Xo.cols(indexMAF); // new X only with polymorphic markers
// remove monomorphic markers
arma::rowvec xVar = var(Xo2,0); // column variance
arma::uvec index = find(xVar > 0); // index for good markers
arma::mat X = Xo2.cols(index); // new X only with polymorphic markers
// initialize A
int p = X.n_cols;// number of markers
int n = X.n_rows;
arma::mat A(n,n);
if(vanraden == true){ // regular vanRaden
arma::rowvec ms012 = mean( X+1, 0 ); // means of columns
arma::rowvec freq = ms012/2;
double v = 2 * mean(freq % (1 - freq));
arma::mat one(n, 1, arma::fill::ones);
arma::mat freqmat = one * freq;
arma::mat W = (X + 1) - (2 * freqmat);
//
arma::mat K = W * W.t();
A = K/v/p;
}else{ // Endelman (currently we have a bug here)
// IN R: M <- scale(X, center = TRUE, scale = FALSE)
arma::rowvec ms = mean( X, 0 ); // means of columns
arma::mat M = X.each_row() - ms;
// IN R: tcrossprod(M)
arma::mat K = M * M.t();
// IN R: K/mean(diag(K)) mean(K.diag())
double v = mean(diagvec(K));
A = K/v;
}
return A;
}
I already tried that and it solves it for isolated functions but it does not work for compiling the entire package. But thanks for trying on your side. This issue seems to arise from functions that use the sparse matrix functions from the Armadillo library. I will keep trying other things.
Dear Eduardo,
We are trying to use A.mat with 75000k lines and 10k markers, getting the following error:
Error: Mat::init(): requested size is too large; suggest to enable ARMA_64BIT_WORD
Still seems to work with 50k lines, but 75k doesn't work any more. Must be related to the number of individuals, because I can replicate it with only 5 markers too. rrBLUP::A.mat does not seem to have this problem, but I thought sommer::A.mat might be more efficient?I am not sure if that is something related to sommer or Rcpp...
Best Gregor