Closed pettyalex closed 2 years ago
It looks like possibly SAIGE could use https://cran.r-project.org/web/packages/RhpcBLASctl/index.html to set the number of threads for this job to 1, and then reset it back to default after?
Thanks! BTW, the following command can also work. export MKL_NUM_THREADS=1;export MKL_DYNAMIC=false; export OMP_NUM_THREADS=1; export OMP_DYNAMIC=false;
I've seen several issues over the years of users finding that Step 2 by default will consume every core available on the machine and take much longer to complete because of thread contention. My own group ran into this, and it wasn't immediately apparent that there was a workaround available, we let SAIGE run for days using 72 threads per job for this.
Do we know where the calls into BLAS code are being done? Is there a way that we can add an nThreads option to Step 2, or even have it only use 1 thread and avoid this problem?
https://github.com/weizhouUMICH/SAIGE/issues/313 https://github.com/weizhouUMICH/SAIGE/issues/341 https://github.com/weizhouUMICH/SAIGE/issues/102