search

veg / hivclustering

Infer molecular transmission networks from pairwise distance files (part of HIV-TRACE)
2 stars 5 forks source link

can this automatically install independence of HyPhy? #20

Open ghost opened 5 years ago

ghost commented 5 years ago

lixingguangtekiMacBook-puro:sbin lixingguang$ pip install hivclustering Requirement already satisfied: hivclustering in /anaconda3/lib/python3.6/site-packages (1.3.2) Collecting bioext>=0.19.0 (from hivclustering) Using cached https://files.pythonhosted.org/packages/c4/f0/779ed11ff3ce38d9a317a7988ec607202e8124d57e01b3f9b2d566860720/bioext-0.19.5.tar.gz Requirement already satisfied: hyphy-python>=0.1.9 in /anaconda3/lib/python3.6/site-packages (from hivclustering) (0.1.9) Requirement already satisfied: hppy>=0.9.9 in /anaconda3/lib/python3.6/site-packages (from hivclustering) (0.9.9) Requirement already satisfied: biopython>=1.58 in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.0->hivclustering) (1.72) Requirement already satisfied: numpy>=1.6 in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.0->hivclustering) (1.14.3) Requirement already satisfied: scipy>=0.15 in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.0->hivclustering) (1.1.0) Requirement already satisfied: pysam>=0.9 in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.0->hivclustering) (0.11.2.2) Collecting joblib (from bioext>=0.19.0->hivclustering) Downloading https://files.pythonhosted.org/packages/cd/c1/50a758e8247561e58cb87305b1e90b171b8c767b15b12a1734001f41d356/joblib-0.13.2-py2.py3-none-any.whl (278kB) 100% |████████████████████████████████| 286kB 248kB/s Requirement already satisfied: six in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.0->hivclustering) (1.11.0) Requirement already satisfied: fakemp>=0.9.1 in /anaconda3/lib/python3.6/site-packages (from hppy>=0.9.9->hivclustering) (0.9.1) Requirement already satisfied: Cython>=0.22.1 in /anaconda3/lib/python3.6/site-packages (from hppy>=0.9.9->hivclustering) (0.28.2) Building wheels for collected packages: bioext Running setup.py bdist_wheel for bioext ... done Stored in directory: /Users/lixingguang/Library/Caches/pip/wheels/d2/fc/06/11fbb7612a12286068f16bfce74bf0960a926291853abe23c0 Successfully built bioext hivtrace 0.1.6 requires HyPhy>=0.1.1, which is not installed. Installing collected packages: joblib, bioext Successfully installed bioext-0.19.5 joblib-0.13.2 You are using pip version 18.1, however version 19.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. lixingguangtekiMacBook-puro:sbin lixingguang$

ghost commented 5 years ago

lixingguangtekiMacBook-puro:sbin lixingguang$ pip install HyPhy Collecting HyPhy Could not find a version that satisfies the requirement HyPhy (from versions: ) No matching distribution found for HyPhy You are using pip version 18.1, however version 19.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. lixingguangtekiMacBook-puro:sbin lixingguang$

stevenweaver commented 5 years ago

Dear @lixingguang,

Please try

pip install hyphy-python

Best, Steven

ghost commented 5 years ago

Dear @stevenweaver, lixingguangtekiMacBook-puro:hivclustering-master lixingguang$ pip install hyphy-python Requirement already satisfied: hyphy-python in /anaconda3/lib/python3.6/site-packages (0.1.9) You are using pip version 18.1, however version 19.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. lixingguangtekiMacBook-puro:hivclustering-master lixingguang$ hivnetworkcsv -h Traceback (most recent call last): File "/anaconda3/bin/hivnetworkcsv", line 4, in import('pkg_resources').run_script('hivclustering==1.3.2', 'hivnetworkcsv') File "/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 658, in run_script self.require(requires)[0].run_script(script_name, ns) File "/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 1429, in run_script .format(locals()), pkg_resources.ResolutionError: Script 'scripts/hivnetworkcsv' not found in metadata at '/anaconda3/lib/python3.6/site-packages/hivclustering-1.3.2-py3.6.egg-info' lixingguangtekiMacBook-puro:hivclustering-master lixingguang$ hivnetworkcsv -h Traceback (most recent call last): File "/anaconda3/bin/hivnetworkcsv", line 4, in import('pkg_resources').run_script('hivclustering==1.3.2', 'hivnetworkcsv') File "/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 658, in run_script self.require(requires)[0].run_script(script_name, ns) File "/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 1429, in run_script .format(locals()), pkg_resources.ResolutionError: Script 'scripts/hivnetworkcsv' not found in metadata at '/anaconda3/lib/python3.6/site-packages/hivclustering-1.3.2-py3.6.egg-info' lixingguangtekiMacBook-puro:hivclustering-master lixingguang$

stevenweaver commented 5 years ago

Dear @lixingguang,

hivclustering is a little harder to test. If you install hivtrace, you should be able to see the commands for hivnetworkcsv there.

git clone https://github.com/veg/hivtrace.git
cd hivtrace
pip install -e .
pip install nose
nosetests

The tests should all pass, and that will tell you the entire pipeline is functional.

An actual command looks like so 

hivtrace -i test/rsrc/TEST.FASTA -a resolve -r HXB2_prrt -t 0.015 -m 500 -g 0.015 -s false --log align.hivtrace.log

If you look at the generated log, you can see the calls to all of the underlying tools

INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bealign -q -r HXB2_1497.fasta -m HIV_BETWEEN_F -K -R aligned.fasta /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-awowtkcb/aligned.fasta_output.bam
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bealign -q -r prrt -m HIV_BETWEEN_F -K -R test/rsrc/TEST.FASTA /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-jbj7qbxe/TEST.FASTA_output.bam
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bealign -q -r HXB2_prrt -m HIV_BETWEEN_F -K -R test/rsrc/TEST.FASTA /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-7im6uq8w/TEST.FASTA_output.bam
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bam2msa /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-7im6uq8w/TEST.FASTA_output.bam test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 2, "title": "Screening contaminants", "status": 2, "msg": ""}
DEBUG:root:tn93 -q -o /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-7im6uq8w/TEST.FASTA_contam.tn93output.csv -t 0.015 -a resolve -l 500 -g 1.0 -s HXB2_prrt -f csv test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bealign -q -r HXB2_prrt -m HIV_BETWEEN_F -K -R test/rsrc/TEST.FASTA /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-ij6uoagl/TEST.FASTA_output.bam
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bam2msa /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-ij6uoagl/TEST.FASTA_output.bam test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 2, "title": "Screening contaminants", "status": 2, "msg": ""}
DEBUG:root:tn93 -q -o /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-ij6uoagl/TEST.FASTA_contam.tn93output.csv -t 0.015 -a resolve -l 500 -g 1.0 -s HXB2_prrt -f csv test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bealign -q -r HXB2_prrt -m HIV_BETWEEN_F -K -R test/rsrc/TEST.FASTA /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-bq5738t6/TEST.FASTA_output.bam
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bam2msa /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-bq5738t6/TEST.FASTA_output.bam test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 2, "title": "Screening contaminants", "status": 2, "msg": ""}
DEBUG:root:tn93 -q -o /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-bq5738t6/TEST.FASTA_contam.tn93output.csv -t 0.015 -a resolve -l 500 -g 1.0 -s HXB2_prrt -f csv test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bealign -q -r HXB2_prrt -m HIV_BETWEEN_F -K -R test/rsrc/TEST.FASTA /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-6zyy4w9s/TEST.FASTA_output.bam
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bam2msa /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-6zyy4w9s/TEST.FASTA_output.bam test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 2, "title": "Screening contaminants", "status": 2, "msg": ""}
DEBUG:root:tn93 -q -o /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-6zyy4w9s/TEST.FASTA_contam.tn93output.csv -t 0.015 -a resolve -l 500 -g 1.0 -s HXB2_prrt -f csv test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bealign -q -r HXB2_prrt -m HIV_BETWEEN_F -R test/rsrc/TEST.FASTA /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-j3hrofjr/TEST.FASTA_output.bam
INFO:root:{"type": "status update", "index": 0, "title": "Aligning", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/bam2msa /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-j3hrofjr/TEST.FASTA_output.bam test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 1, "title": "BAM to FASTA conversion", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 3, "title": "Computing pairwise TN93 distances", "status": 2, "msg": ""}
DEBUG:root:tn93 -q -o /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-j3hrofjr/TEST.FASTA_user.tn93output.csv -t 0.015 -a resolve -l 500 -g 0.015 -f csv test/rsrc/TEST.FASTA_output.fasta
INFO:root:{"type": "status update", "index": 3, "title": "Computing pairwise TN93 distances", "status": 3, "msg": ""}
INFO:root:{"type": "status update", "index": 4, "title": "Inferring, filtering, and analyzing molecular transmission network", "status": 2, "msg": ""}
DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/hivnetworkcsv -i /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-j3hrofjr/TEST.FASTA_user.tn93output.csv -t 0.015 -f plain -j -o
INFO:root:{"type": "status update", "index": 4, "title": "Inferring, filtering, and analyzing molecular transmission network", "status": 2, "msg": "Fitting the degree distribution to various densities\n"}
INFO:root:{"type": "status update", "index": 4, "title": "Inferring, filtering, and analyzing molecular transmission network", "status": 3, "msg": "Fitting the degree distribution to various densities\n"}

Notice

DEBUG:root:/Users/sweaver/Programming/phylotree.js/env/bin/hivnetworkcsv -i /var/folders/zw/6rpd_b8j2lj5ycyc280d2wrm0000gn/T/hivtrace-j3hrofjr/TEST.FASTA_user.tn93output.csv -t 0.015 -f plain -j -o

Best, Steven

ghost commented 5 years ago

@stevenweaver , thanks, but hivtrace is also not easy to install: lixingguangtekiMacBook-puro:hivtrace lixingguang$ pip install -e . Obtaining file:///Users/lixingguang/Downloads/0-0t%26/hivclustering-master/HIVClustering-master/hivtrace Requirement already satisfied: biopython>=1.58 in /anaconda3/lib/python3.6/site-packages (from hivtrace==0.4.6) (1.72) Requirement already satisfied: bioext>=0.19.4 in /anaconda3/lib/python3.6/site-packages (from hivtrace==0.4.6) (0.19.5) Requirement already satisfied: hppy>=0.9.9 in /anaconda3/lib/python3.6/site-packages (from hivtrace==0.4.6) (0.9.9) Requirement already satisfied: tornado>=4.3 in /anaconda3/lib/python3.6/site-packages (from hivtrace==0.4.6) (5.0.2) Requirement already satisfied: hivclustering==1.3.2 in /anaconda3/lib/python3.6/site-packages (from hivtrace==0.4.6) (1.3.2) Requirement already satisfied: numpy in /anaconda3/lib/python3.6/site-packages (from biopython>=1.58->hivtrace==0.4.6) (1.14.3) Requirement already satisfied: pysam>=0.9 in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.4->hivtrace==0.4.6) (0.11.2.2) Requirement already satisfied: joblib in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.4->hivtrace==0.4.6) (0.13.2) Requirement already satisfied: scipy>=0.15 in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.4->hivtrace==0.4.6) (1.1.0) Requirement already satisfied: six in /anaconda3/lib/python3.6/site-packages (from bioext>=0.19.4->hivtrace==0.4.6) (1.11.0) Requirement already satisfied: hyphy-python>=0.1.8 in /anaconda3/lib/python3.6/site-packages (from hppy>=0.9.9->hivtrace==0.4.6) (0.1.9) Requirement already satisfied: fakemp>=0.9.1 in /anaconda3/lib/python3.6/site-packages (from hppy>=0.9.9->hivtrace==0.4.6) (0.9.1) Requirement already satisfied: Cython>=0.22.1 in /anaconda3/lib/python3.6/site-packages (from hppy>=0.9.9->hivtrace==0.4.6) (0.28.2) Installing collected packages: hivtrace Found existing installation: hivtrace 0.1.6 Uninstalling hivtrace-0.1.6: Successfully uninstalled hivtrace-0.1.6 Running setup.py develop for hivtrace Complete output from command /anaconda3/bin/python -c "import setuptools, tokenize;file='/Users/lixingguang/Downloads/0-0t&/hivclustering-master/HIVClustering-master/hivtrace/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" develop --no-deps: running develop running egg_info writing hivtrace.egg-info/PKG-INFO writing dependency_links to hivtrace.egg-info/dependency_links.txt writing entry points to hivtrace.egg-info/entry_points.txt writing requirements to hivtrace.egg-info/requires.txt writing top-level names to hivtrace.egg-info/top_level.txt reading manifest file 'hivtrace.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'hivtrace.egg-info/SOURCES.txt' running build_ext Creating /anaconda3/lib/python3.6/site-packages/hivtrace.egg-link (link to .) Adding hivtrace 0.4.6 to easy-install.pth file error: [Errno 13] Permission denied: '/anaconda3/lib/python3.6/site-packages/easy-install.pth'

----------------------------------------

Rolling back uninstall of hivtrace Command "/anaconda3/bin/python -c "import setuptools, tokenize;file='/Users/lixingguang/Downloads/0-0t&/hivclustering-master/HIVClustering-master/hivtrace/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" develop --no-deps" failed with error code 1 in /Users/lixingguang/Downloads/0-0t&/hivclustering-master/HIVClustering-master/hivtrace/ You are using pip version 18.1, however version 19.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. lixingguangtekiMacBook-puro:hivtrace lixingguang$

ghost commented 5 years ago

lixingguangtekiMacBook-puro:tn93 lixingguang$ pip3 install hivtrace Requirement already satisfied: hivtrace in /anaconda3/lib/python3.6/site-packages (0.1.6) Requirement already satisfied: biopython>=1.58 in /anaconda3/lib/python3.6/site-packages (from hivtrace) (1.72) Requirement already satisfied: BioExt>=0.17.2 in /anaconda3/lib/python3.6/site-packages (from hivtrace) (0.19.5) Collecting HyPhy>=0.1.1 (from hivtrace) Could not find a version that satisfies the requirement HyPhy>=0.1.1 (from hivtrace) (from versions: ) No matching distribution found for HyPhy>=0.1.1 (from hivtrace) You are using pip version 18.1, however version 19.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

ghost commented 5 years ago

I install hyphy with make install, and find that it installs HYPHYMP with default setting: lixingguangtekiMacBook-puro:hyphy lixingguang$ ls CMakeCache.txt Makefile install_manifest.txt CMakeFiles README.md messages.log CMakeLists.txt cmake res CMakeWin32 cmake_install.cmake run_tests.sh CONTRIBUTING.md cmake_uninstall.cmake scripts Examples conf src HYPHYGTEST contrib tests HYPHYMP errors.log LICENSE help

ghost commented 5 years ago

lixingguangtekiMacBook-puro:hyphy lixingguang$ HYPHYMP

 /HYPHY 2.3.14.20190425beta(MP) for Darwin on x86_64\

TYPES OF STANDARD ANALYSES 

(1) Selection Analyses
(2) Evolutionary Hypothesis Testing
(3) Relative evolutionary rate inference
(4) Coevolutionary analysis
(5) Basic Analyses
(6) Codon Selection Analyses
(7) Compartmentalization
(8) Data File Tools
(9) Miscellaneous
(10) Model Comparison
(11) Kernel Analysis Tools
(12) Molecular Clock
(13) Phylogeny Reconstruction
(14) Positive Selection
(15) Recombination
(16) Selection/Recombination
(17) Relative Rate
(18) Relative Ratio
(19) Substitution Rates

Please select type of analyses you want to list (or press ENTER to process custom batch file):

ghost commented 5 years ago

HYPHYMP can work, but, why hivtrace can not find that hyphy is already installed? lixingguangtekiMacBook-puro:hyphy lixingguang$ pip3 install hivtrace Requirement already satisfied: hivtrace in /anaconda3/lib/python3.6/site-packages (0.1.6) Requirement already satisfied: biopython>=1.58 in /anaconda3/lib/python3.6/site-packages (from hivtrace) (1.72) Requirement already satisfied: BioExt>=0.17.2 in /anaconda3/lib/python3.6/site-packages (from hivtrace) (0.19.5) Collecting HyPhy>=0.1.1 (from hivtrace) Could not find a version that satisfies the requirement HyPhy>=0.1.1 (from hivtrace) (from versions: ) No matching distribution found for HyPhy>=0.1.1 (from hivtrace) You are using pip version 18.1, however version 19.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.