Closed vals closed 12 years ago
Valentine; The demultiplexing test does take a long time to run, although not four hours here. Are you using a different reference genome than the one in tests/data/genomes? The lines you're seeing above are from GATK processing, and some GATK walkers will be as slow as the size of the reference genome not the number of input fastqs. The test genomes are slimmed down to a couple of chromosomes to help with this.
Practically, you shouldn't need to run the whole test suite every time unless you are specifically working on multiplexing. The variant calling workflow tests most of the major functionality:
nosetests -v -s test_automated_analysis.py:AutomatedAnalysisTest.test_1_variantcall
and is what I run when adjusting anything with the processing pipeline.
Brad; Thank you for the testing guidelines, I think we will look over how and what we test.
Regarding the reference genomes, is there a setting somewhere to specify them? I might have accidentally started using other ones, but I'm not finding where.
Valentine; The genomes are specified by the location files in the testing directory:
I asked because you mentioned 'chr5' in your error message. Those genomes are restricted subsets with just chr22 and chrM to make things run a bit faster.
Brad; Aha! While I was testing the distributed functionality I had pointed the tests to our production universe_wsgi.ini which had the information about our RabbitMQ server in it (to avoid duplicating information). And looking around, it seems that the programs are looking for a tool-data directory in the same directory as the universe_wsgi.ini. Thus I'm guessing it used all the reference genomes, and it took a long time. Thank you for your help!
Hi Brad, we have started to experience really long times for running the test suite, have the run time for tests increased for you as well? Or are we doing something odd? Running all the tests on a node with 8 cores takes about 4 hours.
Something new I have noted in the stdout from
nosetests -s -v test_automated_analysis.py
is thousands of lines of the formwhich seem to perform some work for several hours.