search

SACGF / variantgrid

VariantGrid public repo
Other
23 stars 2 forks source link

VEP deployment - multiple versions on same disk #1057

Closed davmlaw closed 1 month ago

davmlaw commented 3 months ago

VG VEP install instructions assumed there would only be 1 VEP installed per system. This caused 2 issues:

Running wrong version on diff deployments

If you modify the VEP dir on 1 machine, it breaks another system that relied on it being a certain version (as eg code was not setup to handle that version of VEP). This caused prod issue https://github.com/SACGF/variantgrid_shariant/issues/158

Plugin breaking

We only passed "--dir" which assumes the plugins are installed in ${VEP_CACHE_DIR}/Plugins- if plugins change over time - this causes errors eg 

WARNING: Failed to instantiate plugin dbNSFP: ERROR: Could not find the required column aaalt for pep_match option in dbNSFP file
davmlaw commented 3 months ago

I updated the VEP wiki to explicitly have versions in the VEP paths

Also modifying pipeline to pass cache and plugins dir so we can have diff versions of plugins on the same machine

        "--dir_cache", settings.ANNOTATION_VEP_CACHE_DIR,
        "--dir_plugins", settings.ANNOTATION_VEP_PLUGINS_DIR,
davmlaw commented 3 months ago

Installed 110 in the right place on vg test, vg prod and Shariant shared drive.

Altered settings so it should work and use this next upgrade

Check whether it works via looking at pipeline runs

davmlaw commented 3 months ago

I've updated shariant prod setting to be VEP 108

davmlaw commented 3 months ago

tested - run annotations etc on vg test and works fine

EmmaTudini commented 2 months ago

@davmlaw How should we test this?

davmlaw commented 2 months ago

This is a deployment issue, so not really user testable

Shariant test has been deployed with this, and annotations work since then

I changed the description from "multiple versions on same machine" to "multiple versions on same disk" - to reflect what it really is, and why it was a issue for Shariant

TheMadBug commented 2 months ago

I've reviewed the layout for this - it's basically letting different versions of VEP data co-exist on the same disk (to stop Shariant Prod from breaking when Shariant Test next upgrades its version of VEP).

Which is to say, I believe it's effectively already been tested. If after the deploy if Shariant Prod's annotations don't start working, we can raise a new issue.