python 3.6 install - pytest failures - 26 failed, 347 passed (pytest 5.3.5?)

[blajoie]

platform linux -- Python 3.6.10, pytest-5.3.5, py-1.8.1, pluggy-0.13.1

Ran into the below while attempting to construct a docker-image and then run the pytests.

   pip install snakemake && \
    pip install pysam && \
    pip install pysam && \
    pip install intervaltree && \
    git config --global http.sslVerify false && \
    pip install pysam>=0.11.2 networkx>=2.0 pandas scipy git+https://github.com/dib-lab/khmer.git && \
    pip install matplotlib && \
    pip install numpy && \
    pip install biokevlar && \
    pip install pytest

Looks like some sort of install/config issue? Any suggestions?

platform linux -- Python 3.6.10, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /scratch
collected 373 items                                                                                                                                                                 
test_alac.py ...........................
test_assemble.py ...............
test_augment.py .....
test_call.py ...........................
test_cigar.py .........
test_cli.py ....................
test_count.py ...........................
test_dist.py ..........
test_evaluate.py .
test_filter.py ......
test_gentrio.py ............................
test_localize.py ............................
test_mutablestring.py .....
test_mutate.py .........
test_novel.py ................
test_partition.py .FFFFFF
test_progress.py ..
test_readgraph.py FF
test_readpair.py .........
test_reference.py ....F
test_seqio.py ..............
test_simlike.py ......FFFFF.F.....................FF
test_sketch.py ..............FF.
test_split.py ..
test_timer.py .
test_unband.py FFF
test_varfilter.py FFFF
test_varmap.py ..........................
test_vcf.py ............

example failures

    def plog(*args, **kwargs):
        """Print logging output."""
        if logstream is not None:
>           print(*args, **kwargs, file=logstream)
E           ValueError: I/O operation on closed file.

/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/__init__.py:79: ValueError

self = <kevlar.readgraph.ReadGraph object at 0x7eff48e195f8>, recordname = 'bogus-genome-chr1_3255_3575_0:0:0_0:0:0_4d4/1'

    def get_record(self, recordname):
>       return self.node[recordname]['record']
E       AttributeError: 'ReadGraph' object has no attribute 'node'

/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/readgraph.py:41: AttributeError

Also ran into an error while running the example workflow

Thu Feb  6 03:50:19 2020]
Job 16: Partition reads by shared novel k-mers.

kevlar --tee --logfile Logs/partition.log partition --min-abund 5 --out NovelReads/partitioned.augfastq.gz NovelReads/filtered.augfastq.gz
[kevlar] running version 0.7
[kevlar::partition] Loading reads
[kevlar::partition] Reads loaded in 0.08 sec
[kevlar::partition] Building read graph in relaxed mode
[kevlar::partition] Graph built in 0.04 sec
[kevlar::partition] Partition readgraph
Traceback (most recent call last):
  File "/opt/conda/envs/kevlar-env/bin/kevlar", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/__main__.py", line 30, in main
    mainmethod(args)
  File "/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/partition.py", line 68, in main
    for partnum, part in partitioner:
  File "/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/partition.py", line 44, in partition
    for n, part in enumerate(part_iter, 1):
  File "/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/readgraph.py", line 144, in partitions
    readstream = [self.get_record(readid) for readid in cc]
  File "/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/readgraph.py", line 144, in <listcomp>
    readstream = [self.get_record(readid) for readid in cc]
  File "/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/readgraph.py", line 41, in get_record
    return self.node[recordname]['record']
AttributeError: 'ReadGraph' object has no attribute 'node'

[blajoie]

And ran into a error unless I explicitly set the below?

export PYTHONIOENCODING=UTF-8

error

Traceback (most recent call last):
  File "/opt/conda/envs/kevlar-env/bin/kevlar", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/envs/kevlar-env/lib/python3.6/site-packages/kevlar/__main__.py", line 24, in main
    kevlar.cli.parser().parse_args(['-h'])
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 1734, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 1766, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 1972, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 1912, in consume_optional
    take_action(action, args, option_string)
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 1840, in take_action
    action(self, namespace, argument_values, option_string)
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 1024, in __call__
    parser.print_help()
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 2366, in print_help
    self._print_message(self.format_help(), file)
  File "/opt/conda/envs/kevlar-env/lib/python3.6/argparse.py", line 2372, in _print_message
    file.write(message)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 48-116: ordinal not in range(128)

[standage]

The encoding issue is almost certainly related to these non-ASCII characters.

https://github.com/kevlar-dev/kevlar/blob/ebabd629e116cc08daf12ecdd1973c4a5d610538/kevlar/cli/__init__.py#L72-L79

But including this line at the top of the file has always fixed the issue for me.

https://github.com/kevlar-dev/kevlar/blob/ebabd629e116cc08daf12ecdd1973c4a5d610538/kevlar/cli/__init__.py#L2

Which version of kevlar are you using?

Have you tried setting a locale on the Docker image? I frequently run into these kind of issues when building Docker/Singularity images from a clean OS without common settings configured/packages installed.

[standage]

Which version of kevlar are you using?

Looking closer at your Docker build, you're installing kevlar directly from PyPI. Some changes have been made to kevlar since the latest release, including support for network ≥2.0. For example, the code below has already fixed one of the issues you described up above.

https://github.com/kevlar-dev/kevlar/blob/ebabd629e116cc08daf12ecdd1973c4a5d610538/kevlar/readgraph.py#L40-L41

I don't know when I'll have time to make a new stable release, but in the mean time I think you'll have to choose:

• install kevlar from PyPI and use networkx 1.x
• install kevlar directly from GitHub and use networkx ≥2.0

[blajoie]

Perfect thanks. Appreciate the speedy response. Will try the latest kevlar from github with networkx >= 2.0

[blajoie]

Thanks again - that did the trick. Also had to explicitly set the locale on my machine/docker-image.

export LC_ALL=C.UTF-8

git clone https://github.com/kevlar-dev/kevlar.git /git/kevlar && \
cd /git/kevlar && \
make devenv && \
pip install -e . &&\
pytest --pyargs kevlar.tests

All tests now pass. 388 passed in 105.88s (0:01:45)