search

openvax / varlens

commandline manipulation of genomic variants and NGS reads
Apache License 2.0
19 stars 2 forks source link

Multiple tests fail on a vanilla installation #1

Closed armish closed 9 years ago

armish commented 9 years ago

Checking out the latest the code from the repository, after I did a pip install ., some of the nose tests fail as follows:

$ nosetests .
EE......EEE..EE..
======================================================================
ERROR: test.test_allele_support.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/hammerlab/varlens/test/test_allele_support.py", line 35, in test_basic
    "--locus", "chr22:46929964",
  File "/home/travis/build/hammerlab/varlens/test/__init__.py", line 36, in run_and_parse_csv
    function(*args)
  File "/home/travis/build/hammerlab/varlens/varlens/commands/allele_support.py", line 73, in run
    grouped = dict(source.pileups([locus]).group_by_allele(locus))
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 117, in pileups
    self.index_if_needed()
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 74, in index_if_needed
    if self.handle.is_bam and not self.handle._hasIndex():
AttributeError: 'pysam.csamfile.Samfile' object has no attribute '_hasIndex'
-------------------- >> begin captured logging << --------------------
root: INFO: Loaded 2 genomic loci.
root: INFO: Reading from: bam_5.bam
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: Failure: AttributeError ('pysam.csamfile.Samfile' object has no attribute '_hasIndex')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python2.7/site-packages/nose/loader.py", line 251, in generate
    for test in g():
  File "/home/travis/build/hammerlab/varlens/test/test_allele_support.py", line 70, in test_simple
    "--variants", data_path("CELSR1/vcfs/vcf_1.vcf#genome=b37"),
  File "/home/travis/build/hammerlab/varlens/test/__init__.py", line 36, in run_and_parse_csv
    function(*args)
  File "/home/travis/build/hammerlab/varlens/varlens/commands/allele_support.py", line 73, in run
    grouped = dict(source.pileups([locus]).group_by_allele(locus))
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 117, in pileups
    self.index_if_needed()
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 74, in index_if_needed
    if self.handle.is_bam and not self.handle._hasIndex():
AttributeError: 'pysam.csamfile.Samfile' object has no attribute '_hasIndex'
-------------------- >> begin captured logging << --------------------
root: INFO: Loaded 5 genomic loci.
root: INFO: Reading from: bam_0.bam
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test.test_reads.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/hammerlab/varlens/test/test_reads.py", line 32, in test_basic
    "--reads", data_path("CELSR1/bams/bam_0.bam"),
  File "/home/travis/build/hammerlab/varlens/test/__init__.py", line 36, in run_and_parse_csv
    function(*args)
  File "/home/travis/build/hammerlab/varlens/varlens/commands/reads.py", line 106, in run
    for read in read_source.reads(loci):
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 87, in reads
    self.index_if_needed()
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 74, in index_if_needed
    if self.handle.is_bam and not self.handle._hasIndex():
AttributeError: 'pysam.csamfile.Samfile' object has no attribute '_hasIndex'
======================================================================
ERROR: test.test_reads.test_loci_filtering
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/hammerlab/varlens/test/test_reads.py", line 56, in test_loci_filtering
    "--reads", data_path("CELSR1/bams/bam_5.bam"),
  File "/home/travis/build/hammerlab/varlens/test/__init__.py", line 36, in run_and_parse_csv
    function(*args)
  File "/home/travis/build/hammerlab/varlens/varlens/commands/reads.py", line 106, in run
    for read in read_source.reads(loci):
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 87, in reads
    self.index_if_needed()
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 74, in index_if_needed
    if self.handle.is_bam and not self.handle._hasIndex():
AttributeError: 'pysam.csamfile.Samfile' object has no attribute '_hasIndex'
======================================================================
ERROR: test.test_reads.test_round_trip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/hammerlab/varlens/test/test_reads.py", line 97, in test_round_trip
    "--out", out,
  File "/home/travis/build/hammerlab/varlens/varlens/commands/reads.py", line 106, in run
    for read in read_source.reads(loci):
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 87, in reads
    self.index_if_needed()
  File "/home/travis/build/hammerlab/varlens/varlens/reads_util.py", line 74, in index_if_needed
    if self.handle.is_bam and not self.handle._hasIndex():
AttributeError: 'pysam.csamfile.Samfile' object has no attribute '_hasIndex'
======================================================================
ERROR: test.test_variants.test_fields
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/hammerlab/varlens/test/test_variants.py", line 93, in test_fields
    "gene_names[0]",
  File "/home/travis/build/hammerlab/varlens/test/__init__.py", line 36, in run_and_parse_csv
    function(*args)
  File "/home/travis/build/hammerlab/varlens/varlens/commands/variants.py", line 49, in run
    df = variants_util.variants_to_dataframe(variants, extra_columns)
  File "/home/travis/build/hammerlab/varlens/varlens/variants_util.py", line 195, in variants_to_dataframe
    callable_or_expression, variant_collection, variant))
  File "/home/travis/build/hammerlab/varlens/varlens/variants_util.py", line 168, in evaluate_variant_expression
    error_value=error_value)
  File "/home/travis/build/hammerlab/varlens/varlens/evaluation.py", line 125, in evaluate_expression
    raise_(ValueError, str(e) + "\n" + extra, traceback)
  File "/home/travis/build/hammerlab/varlens/varlens/evaluation.py", line 118, in evaluate_expression
    return eval(expression, standard_environment, bindings)
  File "<string>", line 1, in <module>
  File "/home/travis/build/hammerlab/varlens/varlens/evaluation.py", line 81, in __getitem__
    return getattr(wrapped, key)
  File "/home/travis/miniconda/lib/python2.7/site-packages/memoized_property.py", line 40, in fget_memoized
    setattr(self, attr_name, fget(self))
  File "/home/travis/miniconda/lib/python2.7/site-packages/varcode/variant.py", line 346, in gene_names
    self.contig, self.start, self.end)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 467, in gene_names_at_locus
    return self.db.distinct_column_values_at_locus(
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 243, in db
    gtf=self.gtf,
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 227, in gtf
    self._set_local_paths()
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 186, in _set_local_paths
    overwrite=overwrite)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 154, in _get_gtf_path
    overwrite=overwrite)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 147, in _get_cached_path
    overwrite=overwrite)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/download_cache.py", line 292, in local_path_or_install_error
    self._raise_missing_file_error({field_name: path_or_url})
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/download_cache.py", line 278, in _raise_missing_file_error
    raise ValueError(error_message)
ValueError: Missing genome data file from ftp://ftp.ensembl.org/pub/release-75/gtf/homo_sapiens/Homo_sapiens.GRCh37.75.gtf.gz. Run pyensembl install --release 75 --species homo_sapiens
Error while evaluating: 
    gene_names[0]
on:
<Environment with bindings: BasicFields alt alt_allele_index coding_genes coding_transcripts collection contig effect_on_transcript effects end ensembl fields filter from_dict from_json gene_ids gene_names genes id inclusive_end inclusive_start info interbase_end interbase_start metadata original_alt original_ref original_start qual ref reference_name short_description sources start to_json transcript_ids transcript_names transcripts variant>
-------------------- >> begin captured logging << --------------------
root: INFO: Loaded 3 variants.
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test.test_variants.test_round_trip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/hammerlab/varlens/test/test_variants.py", line 109, in test_round_trip
    "gene_names[0]",
  File "/home/travis/build/hammerlab/varlens/varlens/commands/variants.py", line 49, in run
    df = variants_util.variants_to_dataframe(variants, extra_columns)
  File "/home/travis/build/hammerlab/varlens/varlens/variants_util.py", line 195, in variants_to_dataframe
    callable_or_expression, variant_collection, variant))
  File "/home/travis/build/hammerlab/varlens/varlens/variants_util.py", line 168, in evaluate_variant_expression
    error_value=error_value)
  File "/home/travis/build/hammerlab/varlens/varlens/evaluation.py", line 125, in evaluate_expression
    raise_(ValueError, str(e) + "\n" + extra, traceback)
  File "/home/travis/build/hammerlab/varlens/varlens/evaluation.py", line 118, in evaluate_expression
    return eval(expression, standard_environment, bindings)
  File "<string>", line 1, in <module>
  File "/home/travis/build/hammerlab/varlens/varlens/evaluation.py", line 81, in __getitem__
    return getattr(wrapped, key)
  File "/home/travis/miniconda/lib/python2.7/site-packages/memoized_property.py", line 40, in fget_memoized
    setattr(self, attr_name, fget(self))
  File "/home/travis/miniconda/lib/python2.7/site-packages/varcode/variant.py", line 346, in gene_names
    self.contig, self.start, self.end)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 467, in gene_names_at_locus
    return self.db.distinct_column_values_at_locus(
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 243, in db
    gtf=self.gtf,
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 227, in gtf
    self._set_local_paths()
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 186, in _set_local_paths
    overwrite=overwrite)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 154, in _get_gtf_path
    overwrite=overwrite)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/genome.py", line 147, in _get_cached_path
    overwrite=overwrite)
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/download_cache.py", line 292, in local_path_or_install_error
    self._raise_missing_file_error({field_name: path_or_url})
  File "/home/travis/miniconda/lib/python2.7/site-packages/pyensembl/download_cache.py", line 278, in _raise_missing_file_error
    raise ValueError(error_message)
ValueError: Missing genome data file from ftp://ftp.ensembl.org/pub/release-75/gtf/homo_sapiens/Homo_sapiens.GRCh37.75.gtf.gz. Run pyensembl install --release 75 --species homo_sapiens
Error while evaluating: 
    gene_names[0]
on:
<Environment with bindings: BasicFields alt alt_allele_index coding_genes coding_transcripts collection contig effect_on_transcript effects end ensembl fields filter from_dict from_json gene_ids gene_names genes id inclusive_end inclusive_start info interbase_end interbase_start metadata original_alt original_ref original_start qual ref reference_name short_description sources start to_json transcript_ids transcript_names transcripts variant>
-------------------- >> begin captured logging << --------------------
root: INFO: Loaded 3 variants.
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 17 tests in 6.370s
FAILED (errors=7)

Verified this is not specific to my local environment via Travis: https://travis-ci.org/hammerlab/varlens/jobs/93789685

timodonnell commented 9 years ago

thanks @armish , great to have someone else working with this repo!

Looks like pysam 0.8.4 has broken this. Version 0.8.3 works. I'll look into a fix, but if you're blocked on it right now quick fix is to downgrade to pysam 0.8.3.

armish commented 9 years ago

Thanks much for the fix, Tim! That was quick — next time I will look into the problem by myself and then bother you if I cannot resolve it