AnswerDotAI / bert24

Apache License 2.0
60 stars 3 forks source link

Migrate ColBERT to conda environment to leverage faiss-gpu #37

Open bclavie opened 4 months ago

bclavie commented 4 months ago

Follow-up to #29. faiss-gpu currently cannot be installed via pip, which will cause evals to take an unnecessarily long time. We should make sure faiss-gpu can be added as seamlessly as possible to the conda env.

NohTow commented 4 months ago

Could we add something like

{% if ColBERTL|default('false')|lower == 'true' %}
    - faiss-gpu
    - (other ColBERT dependencies)
{% endif %}

To the environment.yml And use ColBERT=true conda env create --file environment.yml to install the env?