Added test suite for UPPMAX

[brainstorm]

Hello @arvestad, @tzcoolman,

Here you have a minimal testsuite that would be useful to debug the issues we've been seeing with DRASS (or my misuse of it).

Please, test your changes against it and share your results! Enze, please, merge this pull request too so that Lars can update his copy and test his changes.

To run the tests, one just has to run:

$ make tests

That should build the bloom filter against the ecoli K12 reference genome present on uppmax and then query against the bloom filter. I might have messed up with those arguments, but it still segfaults for me:

$ make tests ./tests/fastq_dummy.py 50 tests/data/ecoli_dummy.fastq ./bloom_build -k 21 -l tests/reference_genomes.list -p tests/data/ecoli.bloom K_mer size: The argument of -k is 21 File list : The argument of -l is tests/reference_genomes.list Prefix : The argument of -p is tests/data/ecoli.bloom source->/bubo/nobackup/uppnex/reference/biodata/genomes/Ecoli/eschColi_K12/seq/eschColi_K12.fa

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx capacity->4705967 Vector size->67975111 ideal hashes->10 error_rate->0.000500 bloom_init(67975111,10) => (67975111,10) =>0.000500 real size->8496888 finish 1 processing... obj_file->/bubo/nobackup/uppnex/reference/biodata/genomes/Ecoli/eschColi_K12/seq/eschColi_K12.fa bloom name->tests/data/ecoli.bloomeschColi_K12.bloom big file process OK xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx all finished... total=10 sec Same K_mer->143840 ,New K_mer->4562095 ./simple_check -m 1 -q tests/data/ecoli_dummy.fastq -l tests/bloom_filters.list Mode : The argument of -m is 1 Query : The argument of -q is tests/data/ecoli_dummy.fastq Bloom list : The argument of -l is tests/bloom_filters.list distributing... offset->777 TYPE->2 make: *\ [tests] Segmentation fault (core dumped)

[tzcoolman]

@brainstorm Did you recompile all the stuff Roman? I have tested it and segmentation fault no longer occurred.

[tzcoolman]

@brainstorm And also, did you try your previous test data?? I had run it and the result was okay. One e.coli read is captured from total 25 reads.

[tzcoolman]

@brainstorm Anyway I ll come to the lab tomorrow so that we can work this out.

[tzcoolman]

@brainstorm Hej Roman... The reason that it got seg fault is that the reads are two few. Only 25. When I doubled the reads number. It worked fine.

[brainstorm]

Can you pull and run "make tests" on your account?

Also, please feel free to add or change the commands used in the Makefile (those under the "tests:" target) in order to have common reproducible steps we can work with.

Otherwise, can you state the exact commands you're using?

[brainstorm]

Enze, the tests (make tests) Makefile target generates a 50-line fastq file by default:

./tests/fastq_dummy.py 50 tests/data/ecoli_dummy.fastq

Therefore the problem does not lay in the number of reads being too low... even if it was there, we need a bloom filter that works regardless of the number of reads, pretty much like @henrikstranneheim FACS (I guess) did.

[brainstorm]

Hello @tzcoolman, @arvestad,

I think I've hit the problem, and typically with C there are memory handling leaks/issues that make simple_check fail on machines with different kernel/memory/security configurations:

The argument of -l is tests/bloom_filters.list distributing... offset->777 TYPE->2 ==3847== Invalid read of size 1 ==3847== at 0x4A091C1: strstr (mc_replace_strmem.c:1037) ==3847== by 0x40197B: get_parainfo (simple_check_1_ge.c:297) ==3847== by 0x402F7D: main (simple_check_1_ge.c:91) ==3847== Address 0x30a is not stack'd, malloc'd or (recently) free'd

Enze, that only happens on biologin, I used "valgrind" to debug memory leaks: http://valgrind.org

Please, merge in my upcoming pull request and "make tests". You should see something like:

(...) ==6278== ERROR SUMMARY: 40605 errors from 31 contexts (suppressed: 6 from 6)

Memory leaks, even when "silent" do require further attention.