search

idinov / informatics

Automatically exported from code.google.com/p/informatics
0 stars 0 forks source link

Basic QC2-error stream #3

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 

Hi Ivo,

so, I ran overnight the Basic_QC2 module and it stopped (see pics. The Clean 
sam steps had problems, the error says thet the INPUT otpiopn has not been 
recognized. If I run the same command line on the server by itself it 
works..and also the strange thing is that it starts to produce the output 
(OUTPUT=/usr/pl_cache/pipeline/2011July19_18h01m22s328ms/CleanSam_1.Output-1.sam
') that is a perfect partial sam file.

Find attached the module.

Original issue reported on code.google.com by federica...@gmail.com on 20 Jul 2011 at 6:32

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi Federica:

This appears to be some (versioning?) problem in SAMtools? I saw this forum 
posting which indicated an upgrade may fix the problem:
http://seqanswers.com/forums/archive/index.php/t-7056.html 
(seqanswers.com/forums/archive/index.php/t-7056.html).

Does the same issue arise with other SAM files? Any other clues? I'd also 
copy-paste the exact command-line executable from the Module Logs into an 
fgene3 shell and see if the standard output/error streams provide any 
additional info.

 - Ivo

Original comment by iwaterp...@gmail.com on 20 Jul 2011 at 11:29

GoogleCodeExporter commented 9 years ago 
Hi Ivo,

(1) I used the upgraded version of samtools but I have the same error.

(2) I am testing right away on another .sam file I have.

(3) The same command in the command line:

fgene3 [/projects1/test_pipeline/NEW_ORGANIZATION/scripts] 
/applications/SAMTOOLS/samtools-0.1.16/samtools calmd -b 
/usr/pl_cache/pipeline/2011July20_16h54m02s872ms/SamToolsSort_1.OutputSortedBAMf
ile-1.bam 
/projects1/Reference_genomes/hg18_ensembl/gatk-canonical/gatk-hg18_ensembl.fa > 
/usr/pl_cache/pipeline/2011July20_16h54m02s872ms/SamToolsCamldMDtag_1.OutputNo-D
uplicatesBAMfile-1.bam
Floating exception

Andy, have u ever experienced those problems?

fed

Original comment by federica...@gmail.com on 21 Jul 2011 at 12:15

GoogleCodeExporter commented 9 years ago 
Hi all-

I have experienced this same floating exception with samtools on the fgene
servers now, too, even with standalone test runs from the command line.

I'm not sure if the problem originates from something in our test BAM file
or from something more complicated, like a slight change in a shared/linked
library (if it uses any shared libraries--I can check). We will want to
debug this issue in more detail to look at a few potential causes of the
samtools error.

-andy

Original comment by apcexcha...@gmail.com on 21 Jul 2011 at 9:53

GoogleCodeExporter commented 9 years ago 
I ran also basicQc2 with the upgraded version of samtools, same error. I 
noticed that the number before floating ids different for QC1 and QC2. Maybe it 
doesn't mean anything, just to keep track.

fed

Original comment by federica...@gmail.com on 21 Jul 2011 at 10:03

Attachments:

GoogleCodeExporter commented 9 years ago 
I think this has the same problem as BasicQC1. Try this workflow.

Original comment by shobe...@gmail.com on 14 Sep 2011 at 10:45

Attachments:

GoogleCodeExporter commented 9 years ago 
I am doing that,

Fed

Original comment by federica...@gmail.com on 15 Sep 2011 at 5:21

GoogleCodeExporter commented 9 years ago 
Hi,

unfortunately the QC2 modules was not succesfull as the QC1.

I had those two errors:

-ValidateSAM: Exception in thread "main" java.lang.NoClassDefFoundError: 
/applications/picard_1/38/picard-tools-1/38/ValidateSamFile/jar
Caused by: java.lang.ClassNotFoundException: 
.applications.picard_1.38.picard-tools-1.38.ValidateSamFile.jar
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: 
/applications/picard_1.38/picard-tools-1.38/ValidateSamFile.jar.  Program will 
exit.

-and this one on the indexing:

[bam_index_core] the alignment is not sorted 
(GA-BETA1544-1_0001:1:1:1019:15730#0): 136885159 > 136885061 in 2-th chr

It is a clear error due to the fact that the bam file seems not being sorted. 
BTW it is weird because for sure (even looking at the pipe) the alignmen filet 
has been previously sorted with the sort module right?

I am also attaching the module.

fed

Original comment by federica...@gmail.com on 19 Sep 2011 at 6:44

Attachments:

GoogleCodeExporter commented 9 years ago 
The first problem is nothing to worry about. To fix it, right click the 
"Validate SAM File" module and click edit module (or however you do this on a 
mac); in the Parameters tab select the parameter called "Jar"; then in the box 
under "Command line prefix" type "-jar" (without the quotes). The module will 
still fail but it will be working.

Original comment by shobe...@gmail.com on 19 Sep 2011 at 10:28

GoogleCodeExporter commented 9 years ago 
I'm not sure what to do about the second problem. I'm running it on my own to 
see if I can figure out something. It is possible the sort module is not 
functioning properly.

Original comment by shobe...@gmail.com on 19 Sep 2011 at 10:51

GoogleCodeExporter commented 9 years ago 
Ok, I have already changed the jar flag..let me know what comes up on your side.

Tks,

Federica

Original comment by federica...@gmail.com on 20 Sep 2011 at 12:02

GoogleCodeExporter commented 9 years ago 
The problem here seems to be that one of the reads causes an error in the calmd 
module and becomes unsorted and can't be indexed. It does not appear to be a 
problem with the sorting module. I'm running it now with the latest version of 
samtools to see if that fixes anything but since I know very little about what 
these tools actually do, I don't know what else to try.

Original comment by shobe...@gmail.com on 22 Sep 2011 at 9:27

GoogleCodeExporter commented 9 years ago 
I am trying also with another input as I wanna see if the problem is related 
simply to the file I was using. Will keep you posted.

Fed

Original comment by federica...@gmail.com on 22 Sep 2011 at 10:17

GoogleCodeExporter commented 9 years ago 
I am testing this pipe always with teh simulation data. It is running since one 
day...and I ahd this error...that is new to me:

Exception in thread "main" java.lang.NoClassDefFoundError: 
/applications/picard_1/38/picard-tools-1/38/ValidateSamFile/jar
Caused by: java.lang.ClassNotFoundException: 
.applications.picard_1.38.picard-tools-1.38.ValidateSamFile.jar
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: 
/applications/picard_1.38/picard-tools-1.38/ValidateSamFile.jar.  Program will 
exit.

Before we were having errors related to the input..now seems super weird that 
it cannot find the program..and I haven't changed any parameters...

Fed

Original comment by federica...@gmail.com on 29 Sep 2011 at 8:38

Attachments:

GoogleCodeExporter commented 9 years ago 
This error occurs because the -jar flag is missing in the Jar parameter. You 
can edit this parameter and add the -jar flag and this will get the program to 
run, however it will still fail. This is because the program always returns an 
exit code 1 (failure) and can't be fixed without running it from a script that 
returns an exit code 0.

Original comment by shobe...@gmail.com on 29 Sep 2011 at 8:49

GoogleCodeExporter commented 9 years ago 
Oh sorry, my fault here.. I have re-started the module...Hope it will not be 
stuck for days.

Tks so much,

Fed

Original comment by federica...@gmail.com on 29 Sep 2011 at 9:02

GoogleCodeExporter commented 9 years ago 
No worries, were you able to make any progress with the calmd error causing 
problems for the index module?

Original comment by shobe...@gmail.com on 29 Sep 2011 at 9:07

GoogleCodeExporter commented 9 years ago 
...still running super slow..I'll let you know..hoperfully soon :)

Fed

Original comment by federica...@gmail.com on 29 Sep 2011 at 9:11

GoogleCodeExporter commented 9 years ago 
Hi,

so now the calmid is going BUT there is always the probelem that the sortins 
seems not be working properly: together with the expected validate sam error 
there is the index error:

[bam_index_core] the alignment is not sorted 
(10_100002809_100003337_0_1_0_0_1:0:0_3:0:0_32ebc): 100002939 > 100002808 in 
10-th chr

Fed

Original comment by federica...@gmail.com on 30 Sep 2011 at 5:33

Attachments:

GoogleCodeExporter commented 9 years ago 
Yes, this is the same problem we've been having. The calmd module is the 
problem even though it completes. Is it possible to run the indexing module 
before the calmd or will this not produce the results you need?

Original comment by shobe...@gmail.com on 30 Sep 2011 at 7:53

GoogleCodeExporter commented 9 years ago 
Mmm, not really  bc the meaning is that after we have done the calmid we have 
the final clean bam and of this one we make the index to use it furthermore...

andy what do you think? have you experienced this problem ever? In command line 
actually no, right?

Fed

Original comment by federica...@gmail.com on 30 Sep 2011 at 8:00