Issue 343 fix ex ac ds - Githubissues

LeeTL1220 commented 8 years ago

This PR:

Removes the dependency on the custom pyvcf fork
Allows VCF 4.2 to be used as a datasource, though issues may still lurk.
Adjusts pysam dependency to make life easier for deployment.
Fixes issue where -w was not honored for VCF input. Unit test added to check.
Updates to utility scripts for generating COSMIC datasources. Documentation created, as well.
Pegging to python 2.7.9 where possible. This is a short term fix and needs to be addressed soon, since distribute breaks on later versions of python.
Eliminated distribute dependency, so comment above is probably no longer relevant. In other words, though untested, python versions 2.7.10 and 2.7.11 will probably work. See next comment...
Using raw file name for installed packages, instead of resource_stream, which was causing issues in some configurations (AttributeError: _io.BytesIO does not have attribute name). Needs verification: This probably fixes for python 2.7.10 and 2.7.11.

LeeTL1220 commented 8 years ago

Closes #346 Closes #345 Closes #343 Closes #288 Closes #338 Closes #232

LeeTL1220 commented 8 years ago

@lbergelson I filed an issue for broken travis. The unit tests all pass on both my laptop and on a Broad VM using create_oncotator_venv.sh. I also manually checked some mutations and the ExAc and new COSMIC annotations were there and correct.

LeeTL1220 commented 8 years ago

@lbergelson So when you have a chance... if you could review...

LeeTL1220 commented 8 years ago

Closes #348

coveralls commented 8 years ago

Changes Unknown when pulling fcfeff951e45e72806ea3fbf1b9e8c5c6edad3d2 on issue-343-Fix_ExAC_ds into \ on develop**.

LeeTL1220 commented 8 years ago

@lbergelson HA! And it works on docker as well. Try it when you can... (Monday I hope)

coveralls commented 8 years ago

Changes Unknown when pulling 34b3b80c4a46ba8e08c05282cb8c4a30e3061752 on issue-343-Fix_ExAC_ds into \ on develop**.

LeeTL1220 commented 8 years ago

Hopefully, Travis passes...

Using a small maflite file on the new candidate April default datasource corpus.

Deployment tests:

[x] Docker on my local VM
[x] clean install (no running of create_oncotator_venv.sh script. Just create a bare virtualenv, then call python setup.py install) on my local VM
[x] create_oncotator_venv then python setup.py install on local VM
[x] create_oncotator_venv then python setup.py install on Broad VM. This worked before, but just to confirm.

Additionally:

[x] unit tests on local VM (using create_oncotator_venv.sh)
[x] unit tests on Broad VM (using create_oncotator_venv.sh)

And travis seems to pass consistently as of this writing.

LeeTL1220 commented 8 years ago

@lbergelson Please do not merge until the checkboxes above are done. I discovered a nasty little error that did not manifest on travis, but did on my local machine.

LeeTL1220 commented 8 years ago

@lbergelson All set! Please review and tell me if I can merge. Whew... that was a slog.

LeeTL1220 commented 8 years ago

@lbergelson If you'd be willing to retry your test and see if you can create the venv, it'd be appreciated.

LeeTL1220 commented 8 years ago

@lbergelson Can you think of any downside to pegging the versions of software in the setup.py? If not, I still need to do that.

lbergelson commented 8 years ago

@LeeTL1220 I don't see any downside to specifying versions in setup.py other than that it means we need to specify versions in both the setup and the create_venv.

LeeTL1220 commented 8 years ago

@lbergelson create_oncotator_venv.sh looks like it just uses the latest version of dependencies. I'll have to update that. I'll peg versions to what was used in docker image.

LeeTL1220 commented 8 years ago

docker versions are same as my clean venv...

(oncotator_venv_clean)lichtens@OncobuntuMk3:~$ pip freeze
bcbio-gff==0.6.2
biopython==1.66
Cython==0.24
enum34==1.1.2
futures==3.0.5
more-itertools==2.2
natsort==4.0.4
ngslib==1.1.18
nose==1.3.7
numpy==1.11.0
Oncotator==1.9.0.1.dev0
pandas==0.18.0
parse==1.6.6
pysam==0.9.0
python-dateutil==2.5.2
python-memcached==1.57
pytz==2016.3
PyVCF==0.6.8
shove==0.6.6
six==1.10.0
SQLAlchemy==1.0.12
stuf==0.9.16

LeeTL1220 commented 8 years ago

@lbergelson Made updates to peg versions in setup.py and create_oncotator_venv.sh and make the version numbers the same. I was able to install successfully using both of these scripts on my local VM.

coveralls commented 8 years ago

Changes Unknown when pulling c3a627cb7821ef51481df68b0b93db683dcce5c5 on issue-343-Fix_ExAC_ds into \ on develop**.

lbergelson commented 8 years ago

@LeeTL1220 Installation works on my machine now. (running python 2.7.11)

lbergelson commented 8 years ago

@LeeTL1220 Review complete!

LeeTL1220 commented 8 years ago

Everything addressed. Running unit tests with new default datasource corpus on my local VM, since we cannot do that on Travis. I'll post back here when complete (<10 minutes)

LeeTL1220 commented 8 years ago

All unit tests complete successfully on my laptop VM, which includes new default datasource corpus.

coveralls commented 8 years ago

Changes Unknown when pulling a8fb014b2580368b5b25e80888da8abf625a4199 on issue-343-Fix_ExAC_ds into \ on develop**.

lbergelson commented 8 years ago

@LeeTL1220 :+1:

broadinstitute / oncotator

Issue 343 fix ex ac ds #344