search

knights-lab / shi7

Auto-parameterizing QC Pipeline for Illumina sequencing runs
GNU Affero General Public License v3.0
18 stars 7 forks source link

'shi7en_trimmer' is silently failing. #8

Closed bhillmann closed 7 years ago

bhillmann commented 8 years ago

Output to the log file is not being properly recorded for the shi7en_trimmer causing a silent failure. This causes the pipeline to write blank output files.

cmstaley commented 8 years ago

After following the directions to get shi7en set up, I tried to run using the following commands: 1) $ shi7en -i /panfs/roc/data_release/0/umgc/sadowsky/miseq/161028_M00262_0137_000000000-APB42_Analysis/ -o /panfs/roc/scratch/staleyc/byron/ssg/ --no_combine_fasta 2) $ shi7en -i /panfs/roc/data_release/0/umgc/sadowsky/miseq/161028_M00262_0137_000000000-APB42_Analysis/ -o /panfs/roc/scratch/staleyc/byron/ssg/ --no_combine_fasta --adaptor Nextera 3) $ shi7en -i /panfs/roc/data_release/0/umgc/sadowsky/miseq/161028_M00262_0137_000000000-APB42_Analysis/ -o /panfs/roc/scratch/staleyc/byron/ssg/ --no_combine_fasta --adaptor Nextera --debug

In all instances, the software seemed to run fine, but output fasta files with no data (0kb). The most recent log file is attached. ninja_shi7.txt

bhillmann commented 8 years ago

There should be a temp folder in the output directory that you specified. 

cd /panfs/roc/scratch/staleyc/byron/ssg/temp

Can you look in the folder.

cd /panfs/roc/scratch/staleyc/byron/ssg/temp/flash

And verify that the FASTQs still have data?

head <sample_name>.fastq
cmstaley commented 8 years ago

The fastq files in /flash/ still have data, but the ones in /trimmer/ do not.

You should have permission to look at all the files in the scratch directory.

bhillmann commented 8 years ago

Thank you Chris, can you wait until tomorrow?

@GabeAl @kaiweiang Tomorrow morning at the shi7en meeting we should be able to debug this with the exact in and out files. Looks like it is related to the piping of files from flash to shi7en.

cmstaley commented 8 years ago

No problem. Thanks for the help.

GabeAl commented 8 years ago

It turns out (we think) your shizen trimmer wasn't added to your PATH environmental variable. Even if it's directly in your script folder and you call it, UNIX (Linux and Mac) can't run the program without the "./" in front of the name unless it's in your PATH.

We're adding instructions to the readme to let folks know how to actually accomplish this (basically you add a line to your .bashrc pointing to the folder containing shi7en trimmer).

cmstaley commented 7 years ago

I downloaded the updated trimmer and set the path in the bashrc file and I now have fasta files that have data. Thanks!

bhillmann commented 7 years ago

Great, I'm glad that it worked.

Do you know what the average insert size between paired end reads where? Stitching can be tough for shotgun reads.