TheDataRideAlongs / ProjectDomino

Scaling COVID public behavior change and anti-misinformation
Apache License 2.0
61 stars 13 forks source link

Install FAISS in Prefect Docker container #73

Closed 0xdecaf closed 4 years ago

0xdecaf commented 4 years ago

FAISS requires an OS dependency libomp-dev (on Ubuntu) or to be installed via conda install faiss-gpu cudatoolkit=10.0 -c pytorch # For CUDA10

See: https://github.com/facebookresearch/faiss/issues/821

lmeyerov commented 4 years ago

I ran

conda install --freeze-installed -c pytorch faiss-gpu

Seems to go ok; our base rapidsenv should have a lot as-is:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    blas-1.0                   |              mkl           6 KB
    certifi-2020.4.5.1         |           py37_0         155 KB
    faiss-gpu-1.6.3            |   py37h1a5d453_0        34.2 MB  pytorch
    intel-openmp-2020.0        |              166         756 KB
    mkl-2020.0                 |              166       128.9 MB
    mkl-service-2.3.0          |   py37he904b0f_0         218 KB
    mkl_fft-1.0.15             |   py37ha843d7b_0         154 KB
    mkl_random-1.1.0           |   py37hd6b4f25_0         321 KB
    numpy-1.18.1               |   py37h4f9e942_0           5 KB
    numpy-base-1.18.1          |   py37hde5b4d6_1         4.2 MB
    openssl-1.1.1g             |       h7b6447c_0         2.5 MB
    scikit-learn-0.22.1        |   py37hd81dba3_0         5.2 MB
    scipy-1.4.1                |   py37h0b6359f_0        14.5 MB
    ------------------------------------------------------------
                                           Total:       191.1 MB

The following NEW packages will be INSTALLED:

  blas               pkgs/main/linux-64::blas-1.0-mkl
  faiss-gpu          pytorch/linux-64::faiss-gpu-1.6.3-py37h1a5d453_0
  intel-openmp       pkgs/main/linux-64::intel-openmp-2020.0-166
  mkl                pkgs/main/linux-64::mkl-2020.0-166
  mkl-service        pkgs/main/linux-64::mkl-service-2.3.0-py37he904b0f_0
  mkl_fft            pkgs/main/linux-64::mkl_fft-1.0.15-py37ha843d7b_0
  mkl_random         pkgs/main/linux-64::mkl_random-1.1.0-py37hd6b4f25_0
  numpy-base         pkgs/main/linux-64::numpy-base-1.18.1-py37hde5b4d6_1

The following packages will be REMOVED:

  libblas-3.8.0-14_openblas
  libcblas-3.8.0-14_openblas
  liblapack-3.8.0-14_openblas

The following packages will be UPDATED:

  certifi                                 2019.11.28-py37_0 --> 2020.4.5.1-py37_0
  openssl                                 1.1.1d-h7b6447c_3 --> 1.1.1g-h7b6447c_0

The following packages will be SUPERSEDED by a higher-priority channel:

  numpy              conda-forge::numpy-1.18.1-py37h95a140~ --> pkgs/main::numpy-1.18.1-py37h4f9e942_0
  scikit-learn       conda-forge::scikit-learn-0.22.1-py37~ --> pkgs/main::scikit-learn-0.22.1-py37hd81dba3_0
  scipy              conda-forge::scipy-1.4.1-py37h921218d~ --> pkgs/main::scipy-1.4.1-py37h0b6359f_0

Proceed ([y]/n)? y
0xdecaf commented 4 years ago

topic_research/Dockerfile updated to include faiss-gpu and the requisite OS dependencies libomp-dev libopenblas-dev