installation issues and problem with waterer.py

[Pezhvuk]

Hi,

I have been attempting to install sumrep and partis from scratch and ran into quite a few dependencies issues. I could list these issues which maybe helpful if added to the installation guide.

Now that Partis is installed, I'm running into an issue when running the quick start command. The issue is with the waterer.py script and I receive the following key attribute error:

AttributeError: 'csamtools.AlignedRead' object has no attribute 'cigarstring'

And here is the whole output:

parameter dir '_output/_home_pejvak_partis_test_example' does not exist, so caching a new set of parameters before running action 'annotate' caching parameters vsearch: 250 / 250 v annotations (0 failed) in 0.1 sec keeping 20 / 239 v genes smith-waterman step seqs procs ig-sw time processing time 1 250 1 Traceback (most recent call last): File "./bin/partis", line 627, in <module> args.func(args) File "./bin/partis", line 189, in run_partitiondriver parter.cache_parameters() File "/home/pejvak/partis/python/partitiondriver.py", line 260, in cache_parameters self.find_new_alleles() File "/home/pejvak/partis/python/partitiondriver.py", line 189, in find_new_alleles self.run_waterer() File "/home/pejvak/partis/python/partitiondriver.py", line 170, in run_waterer waterer.run(cachefname if write_cachefile else None) File "/home/pejvak/partis/python/waterer.py", line 89, in run self.read_output(base_outfname, n_procs) File "/home/pejvak/partis/python/waterer.py", line 377, in read_output qinfo = self.read_query(sam.references, readlist) File "/home/pejvak/partis/python/waterer.py", line 521, in read_query if 'I' in read.cigarstring or 'D' in read.cigarstring: # shm indels! AttributeError: 'csamtools.AlignedRead' object has no attribute 'cigarstring' 1.5 Traceback (most recent call last): File "./bin/partis", line 627, in <module> args.func(args) File "./bin/partis", line 202, in run_partitiondriver check_call(newargv) File "/home/pejvak/anaconda3/envs/partis/lib/python2.7/subprocess.py", line 186, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['./bin/partis', 'cache-parameters', '--infname', '/home/pejvak/partis/test/example.fa']' returned non-zero exit status 1

I would appreciate any help with this. Cheers, Pej.

[psathyrella]

That looks like it's probably a version issue with pysam -- presumably they didn't used to have the cigar string as a member. pysam.__version__ on my machine gives 0.10.0, which works.

[BrandenOlson]

@Pezhvuk Would you mind posting an issue in sumrep about the dependency issues you had during installation?

[dunleavy005]

FYI @BrandenOlson , Kristian and I ran into a similar issue with pysam b/c default conda pysam doesn't have a recent enough version. Doing conda update -y -c bioconda pysam after the normal partis conda install fixed it for me.

[BrandenOlson]

@dunleavy005 I see, thank you for letting me know! I'll add a note to the README.

[Pezhvuk]

Hi @BrandenOlson ,

Sure I'll put the list together before the Monday meeting. I wish I had kept a proper note at the time, but I was rushing to get it to work so I could do some analysis before Monday. Though I remember some of them.

[Pezhvuk]

hi @psathyrella and @dunleavy005,

I tried conda update -y -c bioconda pysam, but I'm still getting the exact same error. Having said that, I used the update command outside the source activated environment. I Don't know if that's why the issue has persisted. I did, however, try the command in the source activated environment afterwards, but all of a sudden nothing by conda works within that environment anymore. This is very confusing and I have no idea why this is happening, because obviously I had no issue with conda within the source activated environment yesterday, otherwise I could not follow the installation guide. Here is what it shows, for instance, when I try your mafft installation procedure:

Failed to import the site module Traceback (most recent call last): File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 548, in <module> main() File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 282, in addusersitepackages user_site = getusersitepackages() File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 258, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 248, in getuserbase USER_BASE = get_config_var('userbase') File "/home/pejvak/anaconda3/lib/python3.6/sysconfig.py", line 601, in get_config_var return get_config_vars().get(name) File "/home/pejvak/anaconda3/lib/python3.6/sysconfig.py", line 550, in get_config_vars _init_posix(_CONFIG_VARS) File "/home/pejvak/anaconda3/lib/python3.6/sysconfig.py", line 421, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata_x86_64_conda_cos6_linux_gnu' Failed to import the site module Traceback (most recent call last): File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 548, in <module> main() File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 282, in addusersitepackages user_site = getusersitepackages() File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 258, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/home/pejvak/anaconda3/lib/python3.6/site.py", line 248, in getuserbase USER_BASE = get_config_var('userbase') File "/home/pejvak/anaconda3/lib/python3.6/sysconfig.py", line 601, in get_config_var return get_config_vars().get(name) File "/home/pejvak/anaconda3/lib/python3.6/sysconfig.py", line 550, in get_config_vars _init_posix(_CONFIG_VARS) File "/home/pejvak/anaconda3/lib/python3.6/sysconfig.py", line 421, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata_x86_64_conda_cos6_linux_gnu'

[psathyrella]

I don't think I follow the particulars of whether your conda install is broken or not, but since we seem to have established that updating pysam will fix the problem, you can check that by starting python from the command line and running

import pysam
print pysam.__version__

[psathyrella]

oh, and I meant to add -- using the docker installation instructions would require less familiarity with the underlying operating system, so might prove easier.

[Pezhvuk]

update: conda update -y -c bioconda pysam does work, but this has to be done within the source activated environment (SAE), which wasn't so obvious to me, as this was my first encounter with SAE. The reason I was receiving the error on my latest post, i.e. conda install/update being broken, was due to a troubleshooting I had to do to resolve an issue with the following part of the partis installation: R --vanilla --slave -e 'install.packages("TreeSim", repos="http://cran.rstudio.com/")'. I was getting a non-zero exit status for most of the dependencies of TreeSim, which complained that I was missing x86_64-conda_cos6-linux-gnu-cc directory and other errors that had to be resolved by installing the following modules through conda: gxx_linux-64 and gfortran_linux-64. These, however, would break the conda install/update within SAE for some reason, which prohibits further installations/updates through conda. Moreover, you have to install them both in the same line, as each one of them has that effect independently. Once I ran the example code, I ran into another issue, which was that yaml module does not exist. presumably much like pysam not being up to date in conda, yaml is not installed by default in conda, so, it should be added to the list of the dependencies in the installation command. Naturally then I had to redo everything all over again and install yaml before the gxx_linux-64 and gfortran_linux-64. I finally ran partis successfully on the example dataset. So, happy ending, but still need to make sense of the output and check if it is the right output. So I would appreciate some guidance on that.

[psathyrella]

OK, thanks for the feedback. As far as we can tell, those compilers and pyyaml are properly installed when things are done all at once and in order (for example this has been tested a lot) but I'm glad you got it working.

[michael-swift]

Hi all,

Thanks for creating this tool psathyrella! I get this same issue when running the latest docker image from here (https://hub.docker.com/r/matsen/partis/tags). I'm wondering if it would be possible to update the docker image that has been pushed to Dockerhub? Is this the correct docker image or is there a more up to data one? Thanks!

[psathyrella]

Ah, no! Sorry, we switched to quay.io since docker hub started being unusable on free tier. lmk if you run into more problems.

[michael-swift]

awesome found it thx!