search

broadinstitute / gatk-dataflow

Development dataflow
BSD 3-Clause "New" or "Revised" License
4 stars 1 forks source link

Fix java.lang.VerifyError: Bad type on operand stack #36

Open akiezun opened 9 years ago

akiezun commented 9 years ago

From @jean-philippe-martin on August 7, 2015 17:44

When ReadsDataflowSource.getReadPCollection calls Google Dataflow's ReadBAMTransform.getReadsFromBAMFilesSharded we get a java.lang.VerifyError. The full error looks like this:

Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    com/google/cloud/genomics/dataflow/readers/bam/ReadBAMTransform.getReadsFromBAMFilesSharded(Lcom/google/cloud/dataflow/sdk/Pipeline;Lcom/google/cloud/genomics/utils/GenomicsFactory$OfflineAuth;Ljava/lang/Iterable;Lcom/google/cloud/genomics/dataflow/readers/bam/ReaderOptions;Ljava/util/List;)Lcom/google/cloud/dataflow/sdk/values/PCollection; @25: invokevirtual
  Reason:
    Type 'com/google/cloud/dataflow/sdk/transforms/Create' (current frame, stack[2]) is not assignable to 'com/google/cloud/dataflow/sdk/transforms/PTransform'
  Current Frame:
    bci: @25
    flags: { }
    locals: { 'com/google/cloud/dataflow/sdk/Pipeline', 'com/google/cloud/genomics/utils/GenomicsFactory$OfflineAuth', 'java/lang/Iterable', 'com/google/cloud/genomics/dataflow/readers/bam/ReaderOptions', 'java/util/List', 'com/google/cloud/genomics/dataflow/readers/bam/ReadBAMTransform' }
    stack: { 'com/google/cloud/dataflow/sdk/values/TupleTag', 'com/google/cloud/dataflow/sdk/Pipeline', 'com/google/cloud/dataflow/sdk/transforms/Create' }
  Bytecode:
    0x0000000: bb00 0159 2db7 0002 3a05 1905 2bb6 0003
    0x0000010: b200 042a 1904 b800 05b6 0006 c000 07b8
    0x0000020: 0008 b600 09b8 000a b200 0b2a 2cb8 0005
    0x0000030: b600 06c0 0007 120c b800 0db6 0009 b600
    0x0000040: 0e3a 0619 0519 06b6 000f b0            

    at org.broadinstitute.hellbender.engine.dataflow.datasources.ReadsDataflowSource.getReadPCollection(ReadsDataflowSource.java:130)
    at org.broadinstitute.hellbender.dev.tools.walkers.bqsr.BadTypeRepro.ingestReadsAndGrabHeader(BadTypeRepro.java:100)
    at org.broadinstitute.hellbender.dev.tools.walkers.bqsr.BadTypeRepro.setupPipeline(BadTypeRepro.java:74)

This is the same error I saw when upgrading to google-cloud-dataflow-java-sdk-all:0.4.150710 (#754), so perhaps this is caused by a version mismatch somewhere.

I wrote a small bug-reproducing class, BadTypeRepro, in its eponymous branch.

Copied from original issue: broadinstitute/hellbender#791

akiezun commented 9 years ago

From @droazen on August 7, 2015 18:59

Thanks for looking into this, JP -- would you be able to continue investigating this error?

akiezun commented 9 years ago

From @jean-philippe-martin on August 7, 2015 19:34

Sure, I can look into it.

akiezun commented 9 years ago

From @droazen on August 7, 2015 19:36

Part of https://github.com/broadinstitute/hellbender/issues/751

akiezun commented 9 years ago

From @jean-philippe-martin on August 7, 2015 21:57

It appears that increasing the version of dataflow-sdk that genomics-dataflow uses fixes the problem. I'm going to cut a new version of genomics-dataflow soon.

akiezun commented 9 years ago

From @droazen on August 10, 2015 19:18

So is it fair to say that this is a dataflow bug rather than a regression in hellbender itself?

akiezun commented 9 years ago

From @jean-philippe-martin on August 10, 2015 21:36

It appears this is a Java bug. Or perhaps this is working as intended and were we more informed we'd have known that using two different versions of dataflow-sdk together wouldn't work.

Arguably, we should not have updated Hellbender without noticing that this broke the BAM reader. Doing so transitioned us from a working BAM reader to a broken one - a regression. This is why I think we're all eager to have the cloud regression tests enforced.