search

idinov / informatics

Automatically exported from code.google.com/p/informatics
0 stars 0 forks source link

BasicQC1-error stream #4

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi Ivo,

I ran  also the BasicQC1. Here the problem is still the script from my 
understanding.  Please find attached the module and the script to double check 
if there is some info missing(I changed the paths and nothing else more).

thanks,

Fed

Original issue reported on code.google.com by federica...@gmail.com on 20 Jul 2011 at 6:49

Attachments:

GoogleCodeExporter commented 9 years ago 
Sorry the *.sh script did not get attached. Perhaps change the file extension 
to TXT or wrap it in a ZIP file and repost.

What was the reason to suspect the script was the problem? It seems like 
SAMtools is generating a floating point runtime exception, no?

Original comment by iwaterp...@gmail.com on 20 Jul 2011 at 11:42

GoogleCodeExporter commented 9 years ago 
I attach here the zipped script. I was asking bc a line of the script was 
recalled..so I interpreted that was a script problem. This "floating point 
runtime exception" means that is a problem of samtools? I will try also with 
the new samtools build.

fed

Original comment by federica...@gmail.com on 20 Jul 2011 at 11:51

Attachments:

GoogleCodeExporter commented 9 years ago 
I ran this module also with an upgraded version of samtools, but I have back 
the same error.

Original comment by federica...@gmail.com on 21 Jul 2011 at 9:58

GoogleCodeExporter commented 9 years ago 
Hi,

I haven't had any feedback from andrew on this issue, let me know if on your 
server you are experiencing the same issues. I re-run on fgene2 also, but I got 
back the same floating error(see attached).

Even using command line:

/applications/samtools-0.1.7_x86_64-linux/samtools calmd -b 
/projects/pipelineCache/pipeline/2011August19_10h51m17s724ms/FixMateInformation_
1.Output-1 
/projects1/Reference_genomes/hg18_ensembl/gatk-canonical/gatk-hg18_ensembl.fa 
/projects/pipelineCache/pipeline/2011August19_10h51m17s724ms/SamToolsCamldMDtag_
1.OutputNo-DuplicatesBAMfile-1.bam
Floating exception

Fed

Original comment by federica...@gmail.com on 19 Aug 2011 at 8:13

Attachments:

GoogleCodeExporter commented 9 years ago 
Sorry for not responding. We are experiencing the same floating point error on 
our servers. Still looking into solutions to this.

Sam

Original comment by shobe...@gmail.com on 22 Aug 2011 at 7:12

GoogleCodeExporter commented 9 years ago 
Thanks,

Fed

Original comment by federica...@gmail.com on 26 Aug 2011 at 8:00

GoogleCodeExporter commented 9 years ago 
The solution to this problem is that you need full permission to the reference 
file. I cannot change the permissions to this file so instead, I created a 
symbolic link to the it and used the link as my data source input. 

This is the command for symbolic linkage: "ln -s 
/projects1/Reference_genomes/hg18_ensembl/gatk-canonical/gatk-hg18_ensembl.fa 
/projects1/test_pipeline/NEW_ORGANIZATION/Basic_QC/gatk-hg18_ensembl.fa"

I then put 
/projects1/test_pipeline/NEW_ORGANIZATION/Basic_QC/gatk-hg18_ensembl.fa in the 
data source. 

I modified the workflow significantly to make it simpler and attached it here. 
I could even add a module that does the linking automatically, or you can just 
make sure you have full permissions to the reference file. Whatever you want.  

Sam

Original comment by shobe...@gmail.com on 14 Sep 2011 at 9:46

Attachments:

GoogleCodeExporter commented 9 years ago 
More accurately, you need write permission to the directory that the reference 
file is in. You don't need full permission to the reference file. This is why 
symbolically linking the reference file to a folder you can write to fixes the 
problem.

Original comment by shobe...@gmail.com on 14 Sep 2011 at 10:42

GoogleCodeExporter commented 9 years ago 
Oh Sam,

thank you so much! I'm gonna test it right away!

fed

Original comment by federica...@gmail.com on 15 Sep 2011 at 5:11

GoogleCodeExporter commented 9 years ago 
Hi,

what does it mean when a run is backlogged? is it due to any errors? It's 
happening to me for both the QC modules..

fed

Original comment by federica...@gmail.com on 15 Sep 2011 at 5:26

Attachments:

GoogleCodeExporter commented 9 years ago 
It just means the server is very busy and cannot accept another job. Although, 
you may still be able to run other jobs with smaller memory requirements.

Original comment by shobe...@gmail.com on 15 Sep 2011 at 6:12

GoogleCodeExporter commented 9 years ago 
ok, tks a lot I'll shift to any other fgenes

Original comment by federica...@gmail.com on 15 Sep 2011 at 6:16

GoogleCodeExporter commented 9 years ago 
Hi,

the run was perfect until the indexing. The output file until then are perfect:

fgene3 [/usr/pl_cache/pipeline/2011September16_14h21m09s250ms] ll
total 19648
-rw-rw-rw- 1 pipeline pipeline 6738598 Sep 16 14:21 
SamToolsRemoveDuplicates_1.OutputNo-DuplicatesBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6592761 Sep 16 14:21 
SamToolsSort_1.OutputSortedBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6738662 Sep 16 14:21 
SamToolsView_1.OutputBAMfile-1.bam
drwxrwxrwx 2 pipeline pipeline    4096 Sep 16 16:08 streams

But looking to the indexing process even if it is green, I have the error that 
it cannot open the BAM (see pic). If I open up che error logs of the output 
folders i have the error saying that it cannot copy the .bai file BUT the .bai 
file (as you can see) has not been produced. Note that I have the permissions 
on the output folders I have set.

Fed

Original comment by federica...@gmail.com on 17 Sep 2011 at 12:25

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi,

the run was perfect until the indexing. The output file until then are perfect:

fgene3 [/usr/pl_cache/pipeline/2011September16_14h21m09s250ms] ll
total 19648
-rw-rw-rw- 1 pipeline pipeline 6738598 Sep 16 14:21 
SamToolsRemoveDuplicates_1.OutputNo-DuplicatesBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6592761 Sep 16 14:21 
SamToolsSort_1.OutputSortedBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6738662 Sep 16 14:21 
SamToolsView_1.OutputBAMfile-1.bam
drwxrwxrwx 2 pipeline pipeline    4096 Sep 16 16:08 streams

But looking to the indexing process even if it is green, I have the error that 
it cannot open the BAM (see pic). If I open up che error logs of the output 
folders i have the error saying that it cannot copy the .bai file BUT the .bai 
file (as you can see) has not been produced. Note that I have the permissions 
on the output folders I have set.

Fed

Original comment by federica...@gmail.com on 17 Sep 2011 at 12:26

Attachments:

GoogleCodeExporter commented 9 years ago 
It looks like the Calmd module outputs the BAM file to the standard output. 
Thus, it isn't passing any file to the Index module. I wrote a wrapper script 
to redirect stdout to the output parameter and put it here: 
/projects1/samtools_calmd.sh. You can copy it wherever you want and use the 
path to that script as the executable in the calmd module. 

Sam

Original comment by shobe...@gmail.com on 17 Sep 2011 at 3:02

GoogleCodeExporter commented 9 years ago 
Use the same script for the calmd module in Basic QC2.

Original comment by shobe...@gmail.com on 17 Sep 2011 at 3:02

GoogleCodeExporter commented 9 years ago 
ok, tks running now.

Fed

Original comment by federica...@gmail.com on 17 Sep 2011 at 3:15

GoogleCodeExporter commented 9 years ago 
Oh yes, BasicQC1 went ok!

Original comment by federica...@gmail.com on 17 Sep 2011 at 4:25