search

kevlar-dev / kevlar

Reference-free variant discovery in large eukaryotic genomes
https://kevlar.readthedocs.io
MIT License
40 stars 9 forks source link

Failed tests #369

Closed serosko closed 4 years ago

serosko commented 4 years ago

Hi, I am tying to install kevlar by following the instructions at https://kevlar.readthedocs.io/en/latest/install.html, but the tests in the end keep on failing. This is what I do:

conda create --name kevlar-env python=3.6 conda activate kevlar-env conda install bwa networkx pysam pandas scipy intervaltree pip install git+https://github.com/dib-lab/khmer.git pip install biokevlar pip install pytest pytest --pyargs kevlar.tests

Becaus the output of the tests is so long, I attached it as a txt-file: testresults.txt

I also tried installing the dependencies via pip instead of conda, but I still get the same results. Do you maybe have some hints how to solve this?

Thanks in advance!

tseemann commented 4 years ago

I get similar:

pytest --pyargs kevlar.tests
======================================= test session starts =======================================
platform linux -- Python 3.7.5, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/brewsci/homebrew-bio/Formula
plugins: cov-2.8.1
collected 373 items

test_alac.py ...........................                                                    [  7%]
test_assemble.py ...............                                                            [ 11%]
test_augment.py .....                                                                       [ 12%]
test_call.py ...........................                                                    [ 19%]
test_cigar.py .........                                                                     [ 22%]
test_cli.py F...................                                                            [ 27%]
test_count.py ..........F................                                                   [ 34%]
test_dist.py ..........                                                                     [ 37%]
test_evaluate.py .                                                                          [ 37%]
test_filter.py ......                                                                       [ 39%]
test_gentrio.py F...............FF..........                                                [ 46%]
test_localize.py ......F.....................                                               [ 54%]
test_mutablestring.py .....                                                                 [ 55%]
test_mutate.py ..F....F.                                                                    [ 58%]
test_novel.py FF..............                                                              [ 62%]
test_partition.py .FFFFFF                                                                   [ 64%]
test_progress.py ..                                                                         [ 64%]
test_readgraph.py FF                                                                        [ 65%]
test_readpair.py .........                                                                  [ 67%]
test_reference.py ..F.F                                                                     [ 69%]
test_seqio.py ........F....F                                                                [ 72%]
<snip>
filename = 'five-snvs-fmtstr-mismatch.vcf', errormsg = 'format data mismatch'

    @pytest.mark.parametrize('filename,errormsg', [
        ('five-snvs-fmt-mismatch.vcf', 'sample number mismatch'),
        ('five-snvs-fmtstr-mismatch.vcf', 'format data mismatch'),
    ])
    def test_reader_format_mismatch(filename, errormsg):
        instream = kevlar.open(data_file(filename), 'r')
        reader = kevlar.vcf.VCFReader(instream)
        with pytest.raises(kevlar.vcf.VariantAnnotationError) as vae:
            calls = list(reader)
>       assert errormsg in str(vae)
E       AssertionError: assert 'format data mismatch' in '<ExceptionInfo VariantAnnotationError tblen=3>'
E        +  where '<ExceptionInfo VariantAnnotationError tblen=3>' = str(<ExceptionInfo VariantAnnotationError tblen=3>)

../../../../../../opt/python/lib/python3.7/site-packages/kevlar/tests/test_vcf.py:218: AssertionError
======================================== warnings summary =========================================
/home/linuxbrew/.linuxbrew/opt/python/lib/python3.7/site-packages/screed/screedRecord.py:12
  /home/linuxbrew/.linuxbrew/opt/python/lib/python3.7/site-packages/screed/screedRecord.py:12: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    from collections import MutableMapping

-- Docs: https://docs.pytest.org/en/latest/warnings.html
===================== 48 failed, 325 passed, 1 warnings in 109.27s (0:01:49) ======================
standage commented 4 years ago

I think this is related to changes with pytest, specifically how error messages are tested. This changed in pytest 5.0, and I haven't had a chance to update the test suite since then. In the mean time, please let me know if downgrading to pytest 4.x fixes the problem.

tseemann commented 4 years ago

Maybe the conda and other packages need to be updated to ensure pytest < 5.0 ?

standage commented 4 years ago

Well, kevlar itself isn't a conda package. And pytest isn't required to run kevlar, only to run the tests. So the "bugfix" here would be to update the documentation (and the devenv target in the Makefile) to reflect that pytest >=4.0,<5.0 is required. The real solution will be to fix the test, at which point only pytest >=5.0 will be supported.

serosko commented 4 years ago

Thank you for the reply. However, this only seems to fix some of the errors. Previously, I got 48 failed tests, while with pytest=4.6.4, I get 26 failed tests:

Hoping this is just a problem with pytest and not kevlar, I moved on and tried following the quick-start example at https://kevlar.readthedocs.io/en/latest/quick-start.html, and the mark I tutorial, but all links pointing to files hosted at AWS are dead. I opened a new issue for that problem here.

standage commented 4 years ago

I fixed a few weird bugs in #371, and the kevlar test suite should now pass using pytest version 4.x. Updates to the tests for pytest 5.x support are pending.

I'll investigate the missing data files and update you when they're accessible again.

standage commented 4 years ago

Ok, the tests were updated in #373 and should now work for pytest >= 5.0. Let me know if that fixes it for you.

serosko commented 4 years ago

This did the trick, thanks a lot.