make test failed and number of cores not limitable?

[jvollme]

After installing all dependencies, downloading the GATB-pipeline and running make test , i got the following output:

Checking Python
scipy OK
numpy OK
mathstats OK
pysam OK
There is nothing to make. All programs are provided as binaries.
cd test ; rm -Rf assembly* ; ../gatb --12 small_test_reads.fa.gz #--no-error-correction
No debugging option
(2017-12-13 15:47:51) GATB-pipeline starting
(2017-12-13 15:47:51) Command line: ../gatb --12 small_test_reads.fa.gz 

(2017-12-13 15:47:51) Running bloocoo on 32 cores
(2017-12-13 15:47:51) Execution of 'bloocoo/Bloocoo'. Command line: 
     /home/john/gatb-minia-pipeline/bloocoo/Bloocoo -file assembly.list_reads -out assembly.corrected_with_bloocoo.fa -abundance-min 2 -kmer-size 31 -nb-cores 32 -slow -high-precision
(2017-12-13 15:47:51) Exception:[Errno 2] No such file or directory
(2017-12-13 15:47:51) Execution of 'bloocoo/Bloocoo' failed. Command line: 
     /home/john/gatb-minia-pipeline/bloocoo/Bloocoo -file assembly.list_reads -out assembly.corrected_with_bloocoo.fa -abundance-min 2 -kmer-size 31 -nb-cores 32 -slow -high-precision
make: *** [test] Error 1

The less important question here is, what this output might mean and how to fix it.

What scares me a lot more here though, is that the "bloocoo" step is simply using all available cores on my machine. I am ultimately planing to use this on my universities grid clusters, where using all available cores is definitely not an option. Running ./gatb without arguments did not indicate any option for limiting the number of cores.

Does this mean the pipeline is really set to always use all available cores?

[sletort]

Hi John, I don't understand why your test failed, but your are right, gatb try to use all the core available. nb_cores_system = int(check_output(["nproc"]))

I think it's no big deal to add a parameter. I'll check with the project manager.

[sletort]

Pull request is done, but it seems that Bloocoo has been compiled properly. This project lacks tests.

@rchikhi

gatb-minia-pipeline/bloocoo/Bloocoo -h
minia-pipeline/bloocoo/Bloocoo: /home/rayan/.linuxbrew/lib/ld.so: bad ELF interpreter: Aucun fichier ou dossier de ce type

[jvollme]

Thanks, an argument for setting the cores would really help. If you are setting a default value for this, i generally think it is a good idea to set it low (not as the maximum number of available cores). I am always anxious to allow tools on shared computation servers, that have default values grabbing all the cores (or memory). The problem is that sometime some user is going to forget setting the approriate parameter and then all other users are screwed as well. If you set the default at a low value, the worst that can happen is that this users job runs slowly (reminding him to set the appropriate argument next time).

[rchikhi]

Apologies for the rough edges, this project is not battle-tested hence you'll likely to run into issues of this type.

Thanks Sebastien for handling this issue and Charles for the commit. John, it looks like in the latest master version you can specify the number of cores. Regarding the error you're getting with Bloocoo, I suggest you bypass it for now using the --no-error-correction flag.

[rchikhi]

Bloocoo has been disabled by default in current version