ERROR:root:Unable to load cythonized quickconsensus module; this is likely because pysam has been updated since genomeview was originally install. To fix this, force reinstall genomeview: pip install --upgrade --force-reinstall genomeview

[moldach]

This is a follow up to #62 (which was on a shared academic HPC).

I've now tried to download/run svviz2 from Linux machine where I have root privileges.

sudo apt install python3-pip
sudo -H pip3 install rpy2==2.9.3
sudo snap install inkscape
mkdir svviz2; cd svviz2
wget http://tandem.bu.edu/trf/downloads/trf409.linux64
chmod +x trf409.linux64
echo "alias trf="/snap/trf409.linux64" >> ~/.bash_aliases
sudo apt-get install python3-venv
sudo python3 -m venv .venv
source .venv/bin/activate
sudo pip3 install cython
sudo pip3 install genomeview
sudo pip3 install -U git+git://github.com/nspies/svviz2.git

Then calling svviz2 I see the error:

(svviz2) mtg@pop-os:/snap$ svviz2
ssw library not found
ERROR:root:Unable to load cythonized quickconsensus module; this is likely because pysam has been updated since genomeview was originally install. To fix this, force reinstall genomeview: 
  pip install --upgrade --force-reinstall genomeview
Traceback (most recent call last):
  File "/usr/local/bin/svviz2", line 6, in <module>
    from svviz2.app.main import main
  File "/home/mtg/.local/lib/python3.7/site-packages/svviz2/app/main.py", line 7, in <module>
    from svviz2.visualize import visualize
  File "/home/mtg/.local/lib/python3.7/site-packages/svviz2/visualize/visualize.py", line 8, in <module>
    from genomeview import Document, ViewRow, GenomeView
  File "/home/mtg/.local/lib/python3.7/site-packages/genomeview/__init__.py", line 24, in <module>
    from genomeview.bamtrack import *
  File "/home/mtg/.local/lib/python3.7/site-packages/genomeview/bamtrack.py", line 6, in <module>
    from genomeview import MismatchCounts
ImportError: cannot import name 'MismatchCounts' from 'genomeview' (/home/mtg/.local/lib/python3.7/site-packages/genomeview/__init__.py)

running sudo pip3 install --upgrade --force-reinstall genomeview followed by svviz2

(svviz2) mtg@pop-os:/snap$ sudo pip3 install --upgrade --force-reinstall genomeview
Collecting genomeview
Collecting pysam (from genomeview)
Collecting numpy (from genomeview)
  Using cached https://files.pythonhosted.org/packages/b7/ce/d0b92f0283faa4da76ea82587ff9da70104e81f59ba14f76c87e4196254e/numpy-1.18.2-cp37-cp37m-manylinux1_x86_64.whl
Installing collected packages: pysam, numpy, genomeview
  Found existing installation: pysam 0.15.4
    Uninstalling pysam-0.15.4:
      Successfully uninstalled pysam-0.15.4
  Found existing installation: numpy 1.18.2
    Uninstalling numpy-1.18.2:
      Successfully uninstalled numpy-1.18.2
  Found existing installation: genomeview 1.0.1
    Uninstalling genomeview-1.0.1:
      Successfully uninstalled genomeview-1.0.1
Successfully installed genomeview-1.0.1 numpy-1.18.2 pysam-0.15.4
(svviz2) mtg@pop-os:/snap$ svviz2
ssw library not found
ERROR:root:Unable to load cythonized quickconsensus module; this is likely because pysam has been updated since genomeview was originally install. To fix this, force reinstall genomeview: 
  pip install --upgrade --force-reinstall genomeview
Traceback (most recent call last):
  File "/usr/local/bin/svviz2", line 6, in <module>
    from svviz2.app.main import main
  File "/home/mtg/.local/lib/python3.7/site-packages/svviz2/app/main.py", line 7, in <module>
    from svviz2.visualize import visualize
  File "/home/mtg/.local/lib/python3.7/site-packages/svviz2/visualize/visualize.py", line 8, in <module>
    from genomeview import Document, ViewRow, GenomeView
  File "/home/mtg/.local/lib/python3.7/site-packages/genomeview/__init__.py", line 24, in <module>
    from genomeview.bamtrack import *
  File "/home/mtg/.local/lib/python3.7/site-packages/genomeview/bamtrack.py", line 6, in <module>
    from genomeview import MismatchCounts
ImportError: cannot import name 'MismatchCounts' from 'genomeview' (/home/mtg/.local/lib/python3.7/site-packages/genomeview/__init__.py)

Here is my system information:

(svviz2) mtg@pop-os:/snap$ uname -a
Linux pop-os 5.3.0-7625-generic #27~1576774560~19.10~f432cd8-Ubuntu SMP Thu Dec 19 20:35:37 UTC  x86_64 x86_64 x86_64 GNU/Linux

[moldach]

Here is the latest message from one of the IT staff helping me trouble shoot this on ComputeCanada:

Hi Matthew,

Here is an explanation for the somewhat unintuitive behaviour of pip on Compute Canada clusters. Since we provide a variety of pre-built Python “wheels”, our version of pip is configured to favour binary packages (wheels) over source distributions. Usually, this causes no problem: either the package is provided by Compute Canada, in which case we highly recommend you use the pre-built version, or it is not provided, in which case PyPI is queried to download either a binary or a source distribution, depending on what a specific project offers.

However, in the case of genomeview, the project distributed wheels up to version 0.9.5, and then stopped, providing only source archives for the later versions. Therefore, pip thought that 0.9.5 was the latest available version since it favoured the latest 0.9.5 wheel and ignored source-only distributions for later versions. You can change this behaviour with pip install package --no-binary :all:. This will force pip not to use any wheel, and will build everything from source.

By using the --no-binary :all: option, I was able to install genomeview-1.0.1. Unfortunately, this does not solve the folloxwing error:

ERROR:root:Unable to load cythonized quickconsensus module; this is likely because pysam has been updated since genomeview was originally install. To fix this, force reinstall genomeview: 
  pip install --upgrade --force-reinstall genomeview ```

I tried uninstalling and reinstalling genomeview, but to no avail. The exact same error happens on non-Compute Canada machines, i.e. genomeview-1.0.1 installs fine but fails when issuing import genomeview, so this seems like a problem with the package itself. At this point, your best bet is to report this to the developers of GenomeView.

[thomasabeel]

GenomeView developer here. The python package 'genomeview-1.0.1' is maintained by nspies, not by us: https://pypi.org/project/genomeview/. GenomeView has not made public releases since April 2018., so I doubt it's a change on our end. GenomeView is a stand-alone piece of software made in Java, that appears to have been wrapped in python by nspies.

[moldach]

Hi @thomasabeelI I have not heard back from @nspies yet...

There is no e-mail listed on his github or personal site. Any idea how to contact him?

[slzarate]

@nspies I have also encountered this error while trying to Dockerize svviz2. You can see my installation script here: https://raw.githubusercontent.com/slzarate/parliament2/master/docker_images/svviz/Dockerfile

[nspies]

Hi all, I've long since left academia and don't have bandwidth to support svviz2 which is both old and only unofficially released.

Unfortunately, this appears to be a packaging issue, and one I'm not sure I can fix right away. The quick fix is to install genomeview from source after all other requirements are installed, which forces it to compile from scratch against the installed dependencies (pysam):

pip install -U git+https://github.com/nspies/genomeview.git

[moldach]

Hi @nspies thank you for getting back to us. I can confirm that after following your suggestions I have got svviz2 up and running. Thank you very much again, and all the best in your future endeavours

[slzarate]

Thanks, @nspies! Congrats and best of luck!

[moldach]

Hey @slzarate did you ever get your container working?

I'm trying it out but getting an error:

sudo docker run -v /home/mtg/svviz2:/mnt/mydata -it moldach686/svviz2 svviz ref /mnt/mydata/c_elegans.PRJNA13758.WS265.genomic.fa -b /mnt/mydata/VC109_Homo_trim_bwaMEM_sort.bam --type nslocation
Could not find fasta file 'ref' -- did you forget to specify a reference file?

[moldach]

Using an older version of svviz I get the opposite error - for the bam file.

(base) mtg@mtg-ThinkPad-P53:~/svviz2$ sudo docker run -v /home/mtg/svviz2/tmp:/mnt/mydata -it eppierj/svviz2 --ref /mnt/data/c_elegans.PRJNA13758.WS265.genomic.fa --variants /mnt/mydata/VARIANTS VC109_Homo_trim_bwaMEM_sort.bam
rpy2 could not be imported; dotplots will not be generated
Traceback (most recent call last):
  File "/usr/local/bin/svviz2", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/svviz2/app/main.py", line 59, in main
    datahub = get_datahub()
  File "/usr/local/lib/python3.5/dist-packages/svviz2/app/main.py", line 21, in get_datahub
    datahub.set_args(args)
  File "/usr/local/lib/python3.5/dist-packages/svviz2/app/datahub.py", line 276, in set_args
    sample = Sample(name, bam_path, self, extra_args)
  File "/usr/local/lib/python3.5/dist-packages/svviz2/app/sample.py", line 57, in __init__
    self._load(extra_args)
  File "/usr/local/lib/python3.5/dist-packages/svviz2/app/sample.py", line 61, in _load
    raise FileNotFoundError("Could not find bam file {}".format(self.bam_path))
FileNotFoundError: Could not find bam file VC109_Homo_trim_bwaMEM_sort.bam

[jammywwh]

Hi all, I've long since left academia and don't have bandwidth to support svviz2 which is both old and only unofficially released.

Unfortunately, this appears to be a packaging issue, and one I'm not sure I can fix right away. The quick fix is to install genomeview from source after all other requirements are installed, which forces it to compile from scratch against the installed dependencies (pysam):

pip install -U git+https://github.com/nspies/genomeview.git

Thx nspies! If anyone else try it and it does not works, you can add --force-reinstall: "pip install --force-reinstall -U git+https://github.com/nspies/genomeview.git".