Open GoogleCodeExporter opened 9 years ago
Hi Federica,
I can't reproduce any of these problems on our server, so if you could post the
error/output streams it would be super helpful. Also I have some questions:
(1) Which flag are you talking about? I take it the GATK_covariate_analysis
module is failing for you? What is in the error/output streams?
(4) If you don't use a data sink, you can usually find the output files in the
"Cache Path" directory, which is listed under the Info tab of the Execution
Information window. Note that this directory gets removed when you reset the
workflow.
(5) If there was a permissions issue, it would likely show up when the temp
directory is created, which seems to be at the Realigning module, not in the
Picard tools. Could you please post the Picard errors?
Thanks,
Alex
Original comment by alexge...@gmail.com
on 5 Aug 2011 at 6:45
Hi,
I had to re-run this workflow again as I lost connection to the server in the
middle of the long run. I will send you all the info as soon as it finishes.
fed
Original comment by federica...@gmail.com
on 9 Aug 2011 at 7:33
Hi,
I re-run the entire workflow, I am replying hopefully to all your question.I am
finally learning the info that you need :)
(0) samtoolsindex error: Exception in thread "main"
java.lang.NoClassDefFoundError: index
Caused by: java.lang.ClassNotFoundException: index
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: index. Program will exit.
This is strange bc the same step works afterwards (see the green samtools
index).
(1) GATK Covariates analisys error stream:
Unrecognized option: -recalFile
Could not create the Java virtual machine.
In the command line there is a "flag" called -recal that is working in the
previous step (covariate calculation) but here seems to give an error. See the
point (1) of this issue. This error comes up everytime we have to run the
covariate analysis. I have the same error if I run the command line. I checke
don the website and the flag is correct.
(3) I don't know how is possible but now tha callability worked (is green). I
found the files in the cache folder.
(5) PICARD
- Validate Sam File: [Mon Aug 08 15:51:21 PDT 2011]
net.sf.picard.sam.ValidateSamFile
INPUT=/usr/pl_cache/pipeline/2011August08_14h23m25s677ms/FixMateInformation_1.Ou
tput-1.bam
OUTPUT=/usr/pl_cache/pipeline/2011August08_14h23m25s677ms/ValidateSamFile_1.Outp
ut-1.validation MODE=VERBOSE MAX_OUTPUT=100 IGNORE_WARNINGS=false
VALIDATE_INDEX=true IS_BISULFITE_SEQUENCED=false MAX_OPEN_TEMP_FILES=8000
TMP_DIR=/tmp/pipeline VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT
COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false
CREATE_MD5_FILE=false
[Mon Aug 08 15:51:22 PDT 2011] net.sf.picard.sam.ValidateSamFile done.
Runtime.totalMemory()=1058865152
-all the other PICARD steps give an error like this one:
Exception in thread "main" java.lang.NoClassDefFoundError:
TMP_DIR=/projects1/test_pipeline/NEW_ORGANIZATION/Advanced_QC/temp/
Caused by: java.lang.ClassNotFoundException:
TMP_DIR=.projects1.test_pipeline.NEW_ORGANIZATION.Advanced_QC.temp.
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class:
TMP_DIR=/projects1/test_pipeline/NEW_ORGANIZATION/Advanced_QC/temp/. Program
will exit.
I have FULL PERMISSIONS on all the "tree" of folders.
I attach the .pipe (I haven't change anything from the last run) and the
picture of the errors.
Federica
Original comment by federica...@gmail.com
on 9 Aug 2011 at 10:10
Attachments:
Most of these java exceptions should be fixed with the lastest version of
pipeline (released today). Let me know if the problem persists.
For (5), from what I've seen of Validate Sam, when the VALIDATION_STRINGENCY
flag is set to "strict", the validation is too strict and it often fails. Try
setting this flag to "lenient". If it still fails, maybe the file is actually
invalid, in which case we'll debug further.
Alex
Original comment by alexge...@gmail.com
on 10 Aug 2011 at 9:10
..sorry for the naive question but do you think that if I download the updated
version of the client now I will loose somehow any of the active sessions that
are running?
federica
Original comment by federica...@gmail.com
on 10 Aug 2011 at 10:24
Ok done, WIll let you know how it goes.
Fed
Original comment by federica...@gmail.com
on 11 Aug 2011 at 10:29
I have exactly the same errors I had before after the overnight run with the
new verion of the pipeline...and I have also set the validtaion parameter as
silent.
Fed
Original comment by federica...@gmail.com
on 12 Aug 2011 at 5:37
Hi,
one of the suggestions to solve the java problem I encountered in teh run of
the advanced QC was to dwnload the newest version. Ivo set up a jar file of the
newest pipeline in:
/projects1/idinov/projects/Pipeline_genomics_informatics_2011/PipelineWorkflows_
July_2011/others. I was not able to unzip the file..seems unable to create the
jar file....
fgene3
[/projects1/idinov/projects/Pipeline_genomics_informatics_2011/PipelineWorkflows
_July_2011/others] unzip Pipeline.jar.zip
Archive: Pipeline.jar.zip
Fed
error: cannot create Pipeline.jar
Fed
Original comment by federica...@gmail.com
on 15 Aug 2011 at 7:54
Hi Federica,
Sorry for the late response.
A lot of these modules are pointing to the wrong executables. They are supposed
to be running from bash scripts that don't appear to be on fgene1 yet, so I
copied them over and pointed all the modules to their correct script.
I noticed the original workflow you posted is running from fgene3, which I
don't have an account for yet. I'm testing this latest workflow on fgene1, and
hoping the new scripts are in a directory that is shared between the two
servers. To run this workflow on fgene3, click "Tools" from the pipeline menu
bar, click on "Server Changer", select fgene3 from the dropdown menu and click
"change". this should switch all the modules from fgene1 to fgene3.
Feel free to give this workflow a try. I'm running it as well.
Alex
Original comment by alexge...@gmail.com
on 17 Aug 2011 at 6:35
Attachments:
Thanks alex! I will try it immediately on fgene2 as is the only one fgene I can
have access to by now. Will keep you posted. WHen you are saying that some
scripts were not on the machine, to which one are you referring to? Bc seemed
to me that the paths were set up correctly...just to understand and to avoid
errors in the future.
fed
Original comment by federica...@gmail.com
on 17 Aug 2011 at 6:41
Thanks alex! I will try it immediately on fgene2 as is the only one fgene I can
have access to by now. Will keep you posted. WHen you are saying that some
scripts were not on the machine, to which one are you referring to? Bc seemed
to me that the paths were set up correctly...just to understand and to avoid
errors in the future.
fed
Original comment by federica...@gmail.com
on 17 Aug 2011 at 6:59
for example, the picard tools that failed need to be run from bash scripts that
set R environment variables before the java executable is called. So i just
added the bash scripts that we have on our server (MeanQualityByCycle.sh,
CollectAlignmentSummaryMetrics.sh, etc.) to the picard tools directory on
fgene1: /applications/picard_1.38/picard-tools-1.38. I also did this for a
couple GATK modules.
Original comment by alexge...@gmail.com
on 17 Aug 2011 at 7:05
[deleted comment]
Thank you so much! I will try to run it on fgene2 as I am completely unable to
connect to either fgene1 or fgene3.
Fed
Original comment by federica...@gmail.com
on 17 Aug 2011 at 7:34
Sorry, I forgot to update the permissions on these scripts. If the last set of
picard tools or GATK covariate analysis modules failed, just restart them.
Original comment by alexge...@gmail.com
on 17 Aug 2011 at 9:37
Perfect, I am running it again. I'll be back to you.
fed
Original comment by federica...@gmail.com
on 17 Aug 2011 at 11:43
Hi, perfect! The run has been completely successfull! Thank you so much, great
job!I am able also to download all the output plots and stats on my local mac.
(1) Only one thing/curiosity: I checked the cache folders for my inputs (i
didn't reset the modules) and they are not existing seems:
fgene3 [/projects1/test_pipeline/NEW_ORGANIZATION/scripts] cd
usr/pl_cache/pipeline/2011August18_14h14m50s213ms/
usr/pl_cache/pipeline/2011August18_14h14m50s213ms/: No such file or directory.
SO I was wondering, if a data sync is not specified there isn't any copy of the
output file on the server where the module is running?
(2) I checked almost all the output file and they are ok...exception for the
covariate analysis. I have (see screenshot) a folder with only .dat files and
not plots..I checked the command line:
/applications/gatk/GATK-2010-01-14/dist/AnalyzeCovariates.sh -recalFile
/usr/pl_cache/pipeline/2011August18_14h14m50s213ms/GATKCovariatescalculation_2.G
ATKCovariateCatalogListOutputFile-1.csv -Rscript
/applications/R-2.11.1/bin/Rscript -resources /applications/GATK_source/Sting/R
--output_dir
/usr/pl_cache/pipeline/2011August18_14h14m50s213ms/GATKCovariatesanalysis_2.Outp
utDirectory-1
And it is identical to the one I used in my experience:
java -jar AnalyzeCovariates.jar -recalFile
/applications/GATK_source/usc-build/dist/JLK-227.recal_2.csv -outputDir
/applications/GATK_source/usc-build/dist/test -resources
/applications/GATK_source/Sting/R -Rscript /applications/R-2.11.1/bin/Rscript
...are you having any pdf picture if you run it on cranium?
Thanks,
Fed
Original comment by federica...@gmail.com
on 18 Aug 2011 at 11:26
..actually I have a report a minor error (I thought was due to the VALIDATION
STRINGENCY PARAMETER, but it's not): the validation SAm is giving an error. I
changed the validation stringency to silent but still I got this error stream:
[Thu Aug 18 17:21:16 PDT 2011] net.sf.picard.sam.ValidateSamFile
INPUT=/usr/pl_cache/pipeline/2011August18_16h23m50s362ms/FixMateInformation_1.Ou
tput-1.bam
OUTPUT=/usr/pl_cache/pipeline/2011August18_16h23m50s362ms/ValidateSamFile_1.Outp
ut-1.validation MODE=VERBOSE MAX_OUTPUT=100 IGNORE_WARNINGS=false
VALIDATE_INDEX=true IS_BISULFITE_SEQUENCED=false MAX_OPEN_TEMP_FILES=8000
TMP_DIR=/tmp/pipeline VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT
COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false
CREATE_MD5_FILE=false
[Thu Aug 18 17:21:18 PDT 2011] net.sf.picard.sam.ValidateSamFile done.
Runtime.totalMemory()=1012006912
..that actually is not an error message at all...
The validate sam tool is really strict, maybe the pipeline is interpreting as
an error simply the fact that the software is reporting some issues about the
file (with the validate sam is super common to have minor issue in the output
log).
Fed
Original comment by federica...@gmail.com
on 19 Aug 2011 at 12:48
assuming the pl_cache directory is in /usr/pl_cache on fgene3, you are missing
a forward slash at the beginning of the path.
I need to make some adjustments to the AnalyzeCovariates module, I'll let you
know, but it should fix the pdf problem.
The validate sam module doesn't give very helpful errors, does it? I'll
investigate it more, but there are validation errors in the previous module,
Fix Mate information that might give some clues. Could this be a problem with
running truncated bam files?
Original comment by alexge...@gmail.com
on 19 Aug 2011 at 3:08
Mmm, if I go in the path suggetsed to find the stream error I don't find the
folder...is not a matter of slash I think.
See,
fgene3 [/usr/pl_cache/pipeline] ll
total 44
drwxrwxrwx 3 pipeline pipeline 4096 Aug 10 12:05 2011August09_12h42m56s909ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 9 15:11 2011August09_15h11m34s245ms
drwxrwxrwx 4 pipeline pipeline 4096 Aug 11 14:38 2011August11_12h36m23s565ms
drwxrwxrwx 4 pipeline pipeline 4096 Aug 11 14:38 2011August11_12h36m28s681ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 11 13:52 2011August11_12h43m29s106ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 11 15:09 2011August11_12h44m43s142ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 11 17:50 2011August11_15h28m54s512ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 12 13:05 2011August12_11h53m17s230ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 12 13:05 2011August12_11h53m21s138ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 12 12:46 2011August12_11h56m18s342ms
drwxrwxrwx 3 pipeline pipeline 4096 Aug 12 16:15 2011August12_16h15m47s911ms
...did I understand correctly what you are saying me? Maybe I am wrong :)
Ok for the pdf, thank you.
And I agree, the problem of the validate sam is that is useful to discover HUGE
format error, but is also giving a lot of messages...and my guess is that
probably the pipeline step fails because somehow (sorry for the naive
explanation) the simple fact that the validate sam jar gives a message is
interpreted as an error...the input I am using in reality are not truncated..I
eman all the genome is represented here, we only picked few reads from each
chromosome but I don't think it may be seen by PICARD as trunk...also bc, you
see, in the error message thare are not errors at all! Weird stuff.. I will
also investigate on forums...
Thanks!
Fed
Original comment by federica...@gmail.com
on 19 Aug 2011 at 5:21
Attachments:
Ok, covariates analysis generates the pdf's now. It was setting environment
variables using cranium paths instead of fgene. No need to change anything,
it's running from the same script location, just restart it.
I could be wrong, but I think pipeline modules report errors when the last
system call returns a non-zero status. So something after printing that message
to stderr is returning a non-zero status. I'm looking into it now.
I can't imagine why the cache directory doesn't exist if you haven't restarted
the workflow. But I do notice the cache directory in that screenshot is
different from the one you reported earlier:
fgene3 [/projects1/test_pipeline/NEW_ORGANIZATION/scripts] cd
usr/pl_cache/pipeline/2011August18_14h14m50s213ms/
That directory doesn't exist either?
Original comment by alexge...@gmail.com
on 19 Aug 2011 at 6:41
Thanks, I am running it right away. Probably I reported bunch of different
folder bc I tried different runs, but this one doesn't exist either...I agree
that is weird!
Original comment by federica...@gmail.com
on 19 Aug 2011 at 8:05
I finished the today run, and also here the directory doesn't exist, and I have
the same validatesam error:
/projects2/ServerLibrary
fgene3 [/projects2/ServerLibrary] cd
/usr/pl_cache/pipeline/2011August19_13h03m23s890ms/
/usr/pl_cache/pipeline/2011August19_13h03m23s890ms/: No such file or directory.
fed
Original comment by federica...@gmail.com
on 19 Aug 2011 at 11:47
So I just realized that the validation exceptions are written to the output
*.validation file. If you're still having trouble accessing the pipeline cache
directory you won't be able to see it, but mine reads:
ERROR: Record 70991, Read name GA-BETA1544-1_0001:1:1:2583:20310#0, MAPQ should
be 0 for unmapped read.
ERROR: Record 70991, Read name GA-BETA1544-1_0001:1:1:2583:20310#0, CIGAR
should have zero elements for unmapped read.
ERROR: Read name GA-BETA1544-1_0001:1:1:3489:8719#0, Mate not found for paired
read
Does this indicate anything to you about what might cause the failed validation?
Also, if you haven't already, you should notify your systems administrator
about your inability to access those directories.
Alex
Original comment by alexge...@gmail.com
on 23 Aug 2011 at 5:39
MMM, yes they are notes about format and also about reads where the mate is
missing...but if we try disabling the verbose parameter? does something change?
Btw I don't think the problem is permission...for me those folder don't exist
at all.....do you think is possible I have the message no such file or
directory bc I have no permissions?
Fed
Original comment by federica...@gmail.com
on 26 Aug 2011 at 8:17
What do you mean the mate is missing? Are we supposed to be running this with
paired forward and reverse reads?
I've tried different levels of verbosity, but it only seems to change the
amount of error logs, and the module still fails.
I agree, I don't think you're having a permissions issue. Those usually say
"Permission denied." But I can't really investigate until I have access to
fgene3, which might happen in the next few days. I'll let you know.
Alex
Original comment by alexge...@gmail.com
on 26 Aug 2011 at 4:36
Hi,
I agree that is weird this mate missing problem..but sometimes happens, maybe
due to a base calling failure. Did you have the access to fgene3? Is there any
news?
Federica
Original comment by federica...@gmail.com
on 6 Sep 2011 at 7:49
I still don't have access to fgene3, I'm not sure if that's going to happen.
However, I just noticed in that screenshot that your server address is
fgene2.bic.uci.edu, not fgene3. Maybe try ssh-ing into fgene2 and see if you
can access these /usr/pl_cache directories.
should we try altering this workflow to accept paired reads? I might need some
help with that, but maybe we should do it if that's the problem.
Alex
Original comment by alexge...@gmail.com
on 6 Sep 2011 at 8:26
Hi alex,
this workflow is already able to accept paired end reads, all the softwares
involved are able to manage PE reads...no I didn't have access either on fgene2
to those folders..
federica
Original comment by federica...@gmail.com
on 6 Sep 2011 at 8:31
Hi,
I was running this workflow to take the timing. I was using the same 150k reads
I used when I validated them (and everything worked well).
Now, I am running the same pipe (see attached) with same input file on tehs ame
server and I have this error:
Exception in thread "main" java.lang.NoClassDefFoundError:
'-Djava/io/tmpdir=/projects1/test_pipeline/NEW_ORGANIZATION/Advanced_QC/temp2/'
Caused by: java.lang.ClassNotFoundException:
'-Djava.io.tmpdir=.projects1.test_pipeline.NEW_ORGANIZATION.Advanced_QC.temp2.'
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
Could not find the main class:
'-Djava.io.tmpdir=/projects1/test_pipeline/NEW_ORGANIZATION/Advanced_QC/temp2/'.
Program will exit.
the folder exists and has full permissions.
Original comment by federica...@gmail.com
on 18 Oct 2011 at 11:14
Attachments:
Can you copy and paste the command being run? Right click the failed module ->
Execution info -> Command
Alex
Original comment by alexge...@gmail.com
on 19 Oct 2011 at 6:13
/usr/java/jre1.6.0_24/bin/java
'-Djava.io.tmpdir=/projects1/test_pipeline/NEW_ORGANIZATION/Advanced_QC/temp2/'
-Xmx2g -jar /applications/GATK_source/usc-build/dist/GenomeAnalysisTK.jar -I
/projects2/USC/canonical-test-data/hg18_ensembl-bwa/mini-bam/s_1.sorted.bam -R
/projects1/Reference_genomes/hg18_ensembl/gatk-canonical/gatk-hg18_ensembl.fa
-T IndelRealigner -targetIntervals
/usr/pl_cache/pipeline/2011October19_11h16m25s086ms/IntervalCreation_1.OutputInt
ervalsFile-1.intervals -o
/usr/pl_cache/pipeline/2011October19_11h16m25s086ms/Realigning_1.RealignedBAMout
putfile-1.bam -D /applications/GATK/resources/dbsnp_129_b36.rod
Original comment by federica...@gmail.com
on 19 Oct 2011 at 9:11
I can't reproduce this when I run it from the command line. I can't access
fgene1 from pipeline right now, but once I can I'll investigate it there.
Original comment by alexge...@gmail.com
on 19 Oct 2011 at 9:29
I can't run workflows on fgene2 for some reason, but this error doesn't occur
on fgene1. Is it reasonable to just use the results from fgene1 and set these
fgene2 issues aside for now?
Original comment by alexge...@gmail.com
on 24 Oct 2011 at 10:53
have you tried already on fgene3? I had the same issue on fgene3....if you have
the timing of the run on fgene1 is wonderful so we can fill them in the running
time table..even if this weird behavior across different servers is scaring
me....
Original comment by federica...@gmail.com
on 27 Oct 2011 at 7:54
sure, i'll let you know the timing once i can reconnect (fgene1 down at the
moment)
Original comment by alexge...@gmail.com
on 27 Oct 2011 at 8:14
Original issue reported on code.google.com by
federica...@gmail.com
on 21 Jul 2011 at 9:43Attachments: