search

ssadedin / ximmer

Ximmer is a system for CNV calling on exome and targeted genomic sequencing
http://ssadedin.github.io/ximmer/
GNU Lesser General Public License v2.1
19 stars 10 forks source link

Upgrade to hg38 #27

Closed bopohdr closed 5 years ago

bopohdr commented 5 years ago

What would be necessary to update from files of Ximmer to run it for the hg38 version ? I assume dgv and Refgene files. Is there something else necessary ?

ssadedin commented 5 years ago

That's something we have been doing actually, at least for the CNV calling pipeline part (simulation part, I don't think should be build dependent.).

The work is happening on the 3.0-alpha-1 branch, but that has many (many) other changes too.

Like you say, the main things are:

We are successfully running the analysis pipeline on GRCh38 now off the 3.0-alpha-1 branch.

bopohdr commented 5 years ago

Thank you ! I will try it out. Though, I have some questions:

1) As I understand, CODEX in alfa branch already configured to recalculate GC content and to do other necessary stuff for the provided ref.genome ? I was able to find only in the codex.groovy :

CODEX_HG38_RELOAD_CODE = """

        library(BSgenome.Hsapiens.UCSC.hg38)

        getgc = function (chr, ref) {
          if (chr == "X" | chr == "x" | chr == "chrX" | chr == "chrx") {
            chrtemp <- 23
          } else if (chr == "Y" | chr == "y" | chr == "chrY" | chr == "chry") {
            chrtemp <- 24
          } else {
            chrtemp <- as.numeric(mapSeqlevels(as.character(chr), "NCBI")[1])
          }
          if (length(chrtemp) == 0) message("Chromosome cannot be found in NCBI Homo sapiens database!")
          chrm <- unmasked(Hsapiens[[chrtemp]])
          seqs <- Views(chrm, ref)
          af <- alphabetFrequency(seqs, baseOnly = TRUE, as.prob = TRUE)
          gc <- round((af[, "G"] + af[, "C"]) * 100, 2)
          gc
        }

Not sure how this genome library(BSgenome.Hsapiens.UCSC.hg38) gets downloaded (is it configured to be the genome that I provide ?)

2) And I cannot really find - how Ximmer finds provided gnomad and Decipher databases. Could you please shortly explain how to do it ?

I found only in the SummarizeCNVs.groovy:

    private static initFrequencyAnnotator(OptionAccessor opts, Regions target, SummarizeCNVs summarizer) {
        if(opts.dgv || opts.ddd || opts.gmd) {

            Map databases = [:]
            if(opts.ddd) {
                log.info "Loading Decipher annotations from $opts.ddd ..."
                databases["DDD"] = new DecipherCNVs(opts.ddd).parse()
            }

            if(opts.dgv) {
                log.info "Loading DGV annotations from $opts.dgv ..."
                databases["DGV"] = new DGV(opts.dgv).parse()
            }

            if(opts.gmd) {
                log.info "Adding gnomAD annotation source from $opts.gmd"
                databases["GMD"] = new GnomADCNVDatabase(new VCFIndex(opts.gmd))
            }

            summarizer.cnvAnnotator = new TargetedCNVAnnotator(databases, target)
        }

But cannot see where the path for opts.gmd is shown.

bopohdr commented 5 years ago

To be able to start ximmer, I had to return in bpipe.config file pysam installation instructions, that was removed from 3.0-alpha version only.

After running receive an about missing memory variable in the bpipe.config error:

======================================= Stage calc_qc_stats ======================================== ERROR: stage calc_qc_stats failed: Command in stage calc_qc_stats: set -o pipefail; unset GROOVY_HOME; java -Xmx${memory}g -cp /usr/local/ximmer/eval/pipeline/tools/groovy/2.4.6/embeddable/groovy-all-2.4.6.jar:/usr/local/ximmer/eval/pipeline/tools/groovy-ngs-utils/1.0.9/groovy-ngs-utils.jar gngs.tools.MultiCov -cvj common/rawqc/ximmer.coeffv.js -stats -cv -corr . -2pass -targetmeans common/rawqc/ximmer.merge.sample_interval_summary -covo common/rawqc/ximmer.cov.js -co common/rawqc/ximmer.correlations.tsv -co common/rawqc/ximmer.correlations.js -bed /data/Trusight_hg19_XYonly.bed /data/BAM/XX/1-ZL1000_ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/12-D-120ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/16-ZL556_ARVD_S8_L001_001_rmdup.bam /data/BAM/XX/19-ZL1499_ARVD_S11_L001_001_rmdup.bam /data/BAM/XX/20-ZL1500_ARVD_S13_L001_001_rmdup.bam /data/BAM/XX/21-ZL1501_ARVD_S14_L001_001_rmdup.bam /data/BAM/XX/26-ZL555_ARVD_S19_L001_001_rmdup.bam /data/BAM/XX/28-ZL554ARVD_S21_L001_001_rmdup.bam /data/BAM/XX/3-ZL553_ARVD-S23_L001_001_rmdup.bam /data/BAM/XX/34-ZL873_ARVD_S28_L001_001_rmdup.bam | gzip -c > common/rawqc/ximmer_per_base.coverage.tsv.gz; contains reference to $memory variable, but no memory is specified for this command in the configuration. Please add an entry for memory to the configuration of this command in your bpipe.config file ========================================= Pipeline Failed ========================================== Command in stage calc_qc_stats: set -o pipefail; unset GROOVY_HOME; java -Xmx${memory}g -cp /usr/local/ximmer/eval/pipeline/tools/groovy/2.4.6/embeddable/groovy-all-2.4.6.jar:/usr/local/ximmer/eval/pipeline/tools/groovy-ngs-utils/1.0.9/groovy-ngs-utils.jar gngs.tools.MultiCov -cvj common/rawqc/ximmer.coeffv.js -stats -cv -corr . -2pass -targetmeans common/rawqc/ximmer.merge.sample_interval_summary -covo common/rawqc/ximmer.cov.js -co common/rawqc/ximmer.correlations.tsv -co common/rawqc/ximmer.correlations.js -bed /data/Trusight_hg19_XYonly.bed /data/BAM/XX/1-ZL1000_ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/12-D-120ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/16-ZL556_ARVD_S8_L001_001_rmdup.bam /data/BAM/XX/19-ZL1499_ARVD_S11_L001_001_rmdup.bam /data/BAM/XX/20-ZL1500_ARVD_S13_L001_001_rmdup.bam /data/BAM/XX/21-ZL1501_ARVD_S14_L001_001_rmdup.bam /data/BAM/XX/26-ZL555_ARVD_S19_L001_001_rmdup.bam /data/BAM/XX/28-ZL554ARVD_S21_L001_001_rmdup.bam /data/BAM/XX/3-ZL553_ARVD-S23_L001_001_rmdup.bam /data/BAM/XX/34-ZL873_ARVD_S28_L001_001_rmdup.bam | gzip -c > common/rawqc/ximmer_per_base.coverage.tsv.gz; contains reference to $memory variable, but no memory is specified for this command in the configuration. Please add an entry for memory to the configuration of this command in your bpipe.config file Use 'bpipe errors' to see output from failed commands.

bopohdr commented 5 years ago

To be able to start ximmer, I had to return in bpipe.config file pysam installation instructions, that was removed from 3.0-alpha version only.

After running receive an about missing memory variable in the bpipe.config error:

======================================= Stage calc_qc_stats ======================================== ERROR: stage calc_qc_stats failed: Command in stage calc_qc_stats: set -o pipefail; unset GROOVY_HOME; java -Xmx${memory}g -cp /usr/local/ximmer/eval/pipeline/tools/groovy/2.4.6/embeddable/groovy-all-2.4.6.jar:/usr/local/ximmer/eval/pipeline/tools/groovy-ngs-utils/1.0.9/groovy-ngs-utils.jar gngs.tools.MultiCov -cvj common/rawqc/ximmer.coeffv.js -stats -cv -corr . -2pass -targetmeans common/rawqc/ximmer.merge.sample_interval_summary -covo common/rawqc/ximmer.cov.js -co common/rawqc/ximmer.correlations.tsv -co common/rawqc/ximmer.correlations.js -bed /data/Trusight_hg19_XYonly.bed /data/BAM/XX/1-ZL1000_ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/12-D-120ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/16-ZL556_ARVD_S8_L001_001_rmdup.bam /data/BAM/XX/19-ZL1499_ARVD_S11_L001_001_rmdup.bam /data/BAM/XX/20-ZL1500_ARVD_S13_L001_001_rmdup.bam /data/BAM/XX/21-ZL1501_ARVD_S14_L001_001_rmdup.bam /data/BAM/XX/26-ZL555_ARVD_S19_L001_001_rmdup.bam /data/BAM/XX/28-ZL554ARVD_S21_L001_001_rmdup.bam /data/BAM/XX/3-ZL553_ARVD-S23_L001_001_rmdup.bam /data/BAM/XX/34-ZL873_ARVD_S28_L001_001_rmdup.bam | gzip -c > common/rawqc/ximmer_per_base.coverage.tsv.gz; contains reference to $memory variable, but no memory is specified for this command in the configuration. Please add an entry for memory to the configuration of this command in your bpipe.config file ========================================= Pipeline Failed ========================================== Command in stage calc_qc_stats: set -o pipefail; unset GROOVY_HOME; java -Xmx${memory}g -cp /usr/local/ximmer/eval/pipeline/tools/groovy/2.4.6/embeddable/groovy-all-2.4.6.jar:/usr/local/ximmer/eval/pipeline/tools/groovy-ngs-utils/1.0.9/groovy-ngs-utils.jar gngs.tools.MultiCov -cvj common/rawqc/ximmer.coeffv.js -stats -cv -corr . -2pass -targetmeans common/rawqc/ximmer.merge.sample_interval_summary -covo common/rawqc/ximmer.cov.js -co common/rawqc/ximmer.correlations.tsv -co common/rawqc/ximmer.correlations.js -bed /data/Trusight_hg19_XYonly.bed /data/BAM/XX/1-ZL1000_ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/12-D-120ARVD_S1_L001_001_rmdup.bam /data/BAM/XX/16-ZL556_ARVD_S8_L001_001_rmdup.bam /data/BAM/XX/19-ZL1499_ARVD_S11_L001_001_rmdup.bam /data/BAM/XX/20-ZL1500_ARVD_S13_L001_001_rmdup.bam /data/BAM/XX/21-ZL1501_ARVD_S14_L001_001_rmdup.bam /data/BAM/XX/26-ZL555_ARVD_S19_L001_001_rmdup.bam /data/BAM/XX/28-ZL554ARVD_S21_L001_001_rmdup.bam /data/BAM/XX/3-ZL553_ARVD-S23_L001_001_rmdup.bam /data/BAM/XX/34-ZL873_ARVD_S28_L001_001_rmdup.bam | gzip -c > common/rawqc/ximmer_per_base.coverage.tsv.gz; contains reference to $memory variable, but no memory is specified for this command in the configuration. Please add an entry for memory to the configuration of this command in your bpipe.config file Use 'bpipe errors' to see output from failed commands.

I have corrected manually $memory and got another error for GC calculation

CoverageCombinerActor   INFO    |12:00:30 Processed 0            @ 0.00/s Combining chrX:22000630887 (X:630887) with 1/6 reported, count buffer size=1 (Samples = [8-ZL-2545_S3_L001_001:18]) 
An exception occurred in the Actor thread Actor Thread 8
java.lang.NullPointerException
    at gngs.coverage.CoveragePrinter.writePosition(CoveragePrinter.groovy:67)
    at gngs.coverage.CoverageSummarizer.process(CoverageSummarizer.groovy:250)
    at gngs.coverage.CoverageSummarizer.process(CoverageSummarizer.groovy)
    at gngs.RegulatingActor.processSingleMessage(RegulatingActor.groovy:153)
    at gngs.RegulatingActor.access$1(RegulatingActor.groovy)
    at gngs.RegulatingActor$1.doRun(RegulatingActor.groovy:132)
    at groovyx.gpars.MessagingRunnable.call(MessagingRunnable.java:66)
    at groovyx.gpars.actor.DefaultActor.onMessage(DefaultActor.java:94)
    at groovyx.gpars.actor.DefaultActorClosure.call(DefaultActorClosure.java:38)
    at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMessages(AbstractLoopingActor.java:293)
    at groovyx.gpars.actor.AbstractLoopingActor.access$400(AbstractLoopingActor.java:30)
    at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:93)
    at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
An exception occurred in the Actor thread Actor Thread 2
java.lang.IllegalStateException: The actor cannot accept messages at this point.
    at groovyx.gpars.actor.Actor.createActorMessage(Actor.java:288)
    at groovyx.gpars.actor.AbstractLoopingActor.send(AbstractLoopingActor.java:241)
    at gngs.RegulatingActor.sendLimited(RegulatingActor.groovy:237)
    at gngs.RegulatingActor.sendLimited(RegulatingActor.groovy)
    at gngs.RegulatingActor.send(RegulatingActor.groovy:197)
    at gngs.RegulatingActor.sendDownstream(RegulatingActor.groovy:187)
    at gngs.coverage.CoverageCombinerActor.process(CoverageCombinerActor.groovy:126)
    at gngs.coverage.CoverageCombinerActor.process(CoverageCombinerActor.groovy)
    at gngs.RegulatingActor.processSingleMessage(RegulatingActor.groovy:153)
    at gngs.RegulatingActor.access$1(RegulatingActor.groovy)
    at gngs.RegulatingActor$1.doRun(RegulatingActor.groovy:132)
    at groovyx.gpars.MessagingRunnable.call(MessagingRunnable.java:66)
    at groovyx.gpars.actor.DefaultActor.onMessage(D
efaultActor.java:94)
    at groovyx.gpars.actor.DefaultActorClosure.call(DefaultActorClosure.java:38)
    at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMessages(AbstractLoopingActor.java:293)
    at groovyx.gpars.actor.AbstractLoopingActor.access$400(AbstractLoopingActor.java:30)
    at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:93)
    at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
An exception occurred in the Actor thread Actor Thread 5
An exception occurred in the Actor thread Actor Thread 7
An exception occurred in the Actor thread Actor Thread 3
An exception occurred in the Actor thread Actor Thread 1
...long output in between...
Exception in thread "main" java.lang.IllegalStateException: java.lang.IllegalStateException: The actor cannot accept messages at this point.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at jsr166y.ForkJoinTask.getThrowableException(ForkJoinTask.java:504)
    at jsr166y.ForkJoinTask.reportResult(ForkJoinTask.java:567)
    at jsr166y.ForkJoinTask.join(ForkJoinTask.java:611)
    at jsr166y.ForkJoinPool.invoke(ForkJoinPool.java:1492)
    at groovyx.gpars.extra166y.ParallelArrayWithMapping.apply(ParallelArrayWithMapping.java:48)
    at groovyx.gpars.extra166y.ParallelArray.apply(ParallelArray.java:329)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1212)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at groovyx.gpars.pa.GParsPoolUtilHelper.eachParallelPA(GParsPoolUtilHelper.groovy:192)
    at groovyx.gpars.GParsPoolUtil.eachParallel(GParsPoolUtil.java:358)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.runtime.metaclass.ReflectionMetaMethod.invoke(ReflectionMetaMethod.java:54)
    at org.codehaus.groovy.runtime.metaclass.NewInstanceMetaMethod.invoke(NewInstanceMetaMethod.java:56)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1212)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at gngs.tools.MultiCov$_run_closure5.doCall(MultiCov.groovy:275)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at groovyx.gpars.GParsPool$_withExistingPool_closure1.doCall(GParsPool.groovy:174)
    at groovyx.gpars.GParsPool$_withExistingPool_closure1.doCall(GParsPool.groovy)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at groovy.lang.Closure.call(Closure.java:426)
    at groovy.lang.Closure.call(Closure.java:420)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:110)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.access$400(GroovyCategorySupport.java:68)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:250)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.use(DefaultGroovyMethods.java:407)
    at org.codehaus.groovy.runtime.dgm$755.invoke(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrapNoCoerce.invoke(StaticMetaMethodSite.java:151)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:214)
    at groovyx.gpars.GParsPool.withExistingPool(GParsPool.groovy:173)
    at groovyx.gpars.GParsPool$withExistingPool$1.callStatic(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:214)
    at groovyx.gpars.GParsPool.withPool(GParsPool.groovy:145)
    at groovyx.gpars.GParsPool$withPool$0.callStatic(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:222)
    at groovyx.gpars.GParsPool.withPool(GParsPool.groovy:121)
    at groovyx.gpars.GParsPool$withPool.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
    at gngs.tools.MultiCov.run(MultiCov.groovy:264)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
    at gngs.tools.MultiCov$_run_closure3.doCall(MultiCov.groovy:157)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at groovy.lang.Closure.call(Closure.java:426)
    at groovy.lang.Closure.call(Closure.java:442)
    at org.codehaus.groovy.runtime.IOGroovyMethods.withWriter(IOGroovyMethods.java:1155)
    at org.codehaus.groovy.runtime.IOGroovyMethods.withWriter(IOGroovyMethods.java:1264)
    at org.codehaus.groovy.runtime.dgm$880.invoke(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at gngs.tools.MultiCov.run(MultiCov.groovy:135)
    at gngs.tools.MultiCov$run.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
    at gngs.ToolBase.cli(ToolBase.groovy:148)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1466)
    at org.codehaus.groovy.runtime.callsite.StaticMetaClassSite.callStatic(StaticMetaClassSite.java:65)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:230)
    at gngs.tools.MultiCov.main(MultiCov.groovy:75)
Caused by: java.lang.IllegalStateException: The actor cannot accept messages at this point.
at gngs.RegulatingActor.sendLimited(RegulatingActor.groovy:237)
    at gngs.RegulatingActor.sendLimited(RegulatingActor.groovy)
    at gngs.RegulatingActor.send(RegulatingActor.groovy:197)
    at gngs.coverage.CoverageCalculatorActor$_processBAM_closure4.doCall(CoverageCalculatorActor.groovy:304)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at groovy.lang.Closure.call(Closure.java:426)
    at groovy.lang.Closure.call(Closure.java:442)
    at gngs.SAM$_withIterator_closure1.doCall(SAM.groovy:263)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.in
voke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at groovy.lang.Closure.call(Closure.java:426)
    at groovy.lang.Closure.call(Closure.java:442)
    at gngs.SAM.withReader(SAM.groovy:251)
    at gngs.SAM.withIterator(SAM.groovy:260)
    at gngs.coverage.CoverageCalculatorActor.processBAM(CoverageCalculatorActor.groovy:300)
    at gngs.coverage.CoverageCombinerActor.processBAM(CoverageCombinerActor.groovy:158)
    at gngs.coverage.CoverageCombinerActor.processBAM(CoverageCombinerActor.groovy)
    at gngs.coverage.CoverageCombinerActor$processBAM.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:141)
    at gngs.tools.MultiCov$_run_closure5$_closure35.doCall(MultiCov.groovy:282)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:57)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at groovyx.gpars.pa.GParsPoolUtilHelper$_eachParallelPA_closure8.doCall(GParsPoolUtilHelper.groovy:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)
    at groovy.lang.Closure.call(Closure.java:426)
    at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
    at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
    at com.sun.proxy.$Proxy8.op(Unknown Source)
    at groovyx.gpars.extra166y.AbstractParallelAnyArray$OUPap.leafApply(AbstractParallelAnyArray.java:640)
    at groovyx.gpars.extra166y.PAS$FJOApply.atLeaf(PAS.java:147)
    at groovyx.gpars.extra166y.PAS$FJBase.internalCompute(PAS.java:118)
    at groovyx.gpars.extra166y.PAS$FJBase.compute(PAS.java:106)
    at jsr166y.RecursiveAction.exec(RecursiveAction.java:148)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:305)
    at jsr166y.ForkJoinWorkerThread.execTask(ForkJoinWorkerThread.java:575)
    at jsr166y.ForkJoinPool.scan(ForkJoinPool.java:755)
    at jsr166y.ForkJoinPool.work(ForkJoinPool.java:617)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:369)
Cleaned up file common/rawqc/ximmer_per_base.coverage.tsv.gz to .bpipe/trash/ximmer_per_base.coverage.tsv.gz
ERROR: stage calc_qc_stats failed: Command in stage calc_qc_stats failed with exit status = 1 : 
set -o pipefail;  unset GROOVY_HOME;  java -Xmx28g -cp /usr/local/ximmer/eval/pipeline/tools/groovy/2.4.6/embeddable/groovy-all-2.4.6.jar:/usr/local/ximmer/eval/pipeline/tools/groovy-ngs-utils/1.0.9/groovy-ngs-utils.jar gngs.tools.MultiCov -cvj common/rawqc/ximmer.coeffv.js -stats  -cv   -corr . -2pass -targetmeans common/rawqc/ximmer.merge.sample_interval_summary -covo common/rawqc/ximmer.cov.js -co common/rawqc/ximmer.correlations.tsv -co common/rawqc/ximmer.correlations.js -bed /data/TruSight_one_hg38_XY_only.bed /data/BAM/XX/10-ZL821_S2_L001_001.rmdup.realigned.bam /data/BAM/XX/11-ZL2575_S3_L001_001.rmdup.realigned.bam /data/BAM/XX/8-ZL-2545_S3_L001_001.markdup.realigned.bam /data/BAM/XX/ZL1308_S18_L001_001.rmdup.realigned.bam /data/BAM/XX/ZL2585_S25_L001_001.rmdup.realigned.bam /data/BAM/XX/ZL2789_S22_L001_001.rmdup.realigned.bam | gzip -c > common/rawqc/ximmer_per_base.coverage.tsv.gz; 
========================================= Pipeline Failed ==========================================
Command in stage calc_qc_stats failed with exit status = 1 : 
set -o pipefail;  unset GROOVY_HOME;  java -Xmx28g -cp /usr/local/ximmer/eval/pipeline/tools/groovy/2.4.6/embeddable/groovy-all-2.4.6.jar:/usr/local/ximmer/eval/pipeline/tools/groovy-ngs-utils/1.0.9/groovy-ngs-utils.jar gngs.tools.MultiCov -cvj common/rawqc/ximmer.coeffv.js -stats  -cv   -corr . -2pass -targetmeans common/rawqc/ximmer.merge.sample_interval_summary -covo common/rawqc/ximmer.cov.js -co common/rawqc/ximmer.correlations.tsv -co common/rawqc/ximmer.correlations.js -bed /data/TruSight_one_hg38_XY_only.bed /data/BAM/XX/10-ZL821_S2_L001_001.rmdup.realigned.bam /data/BAM/XX/11-ZL2575_S3_L001_001.rmdup.realigned.bam /data/BAM/XX/8-ZL-2545_S3_L001_001.markdup.realigned.bam /data/BAM/XX/ZL1308_S18_L001_001.rmdup.realigned.bam /data/BAM/XX/ZL2585_S25_L001_001.rmdup.realigned.bam /data/BAM/XX/ZL2789_S22_L001_001.rmdup.realigned.bam | gzip -c > common/rawqc/ximmer_per_base.coverage.tsv.gz;
Use 'bpipe errors' to see output from failed commands.
bopohdr commented 5 years ago

Unfortunately, I was not able to run Alpha build, but I was able to run Ximmer with all CNV callers excluding CODEX (because it is not suitable for hg38 without GC recalculation) on Hg38 genome. Based on simulations, cn.mops produce enormous amount of false positives with very low sensitivity even after tuning, so I am not using it.

To improve annotation of the CNV calls, I changed in dockerfile source for DGV and refGene from /hg19/ to /hg38/ Additionally, in the cnv_report.js file, changed UCSC hypersites from hg19 to hg38 - so that .html report would redirect to the correct locus in UCSC.

thedam commented 1 year ago

hey, so ximmer finally can be run on hg38? I the docker files I see only references to hg19...