Bioconductor / bioconductor_docker

Docker Containers for Bioconductor
https://bioconductor.org/help/docker/
Artistic License 2.0
73 stars 31 forks source link

volume setup for caching basilisk infrastructure? #33

Open vjcitn opened 3 years ago

vjcitn commented 3 years ago

The docker-compose item in README tells how to map a local folder to a volume for storing installed packages. As we expand the scope of Bioc-python interactions, it may be of interest to store, locally, the conda resources managed by basilisk. I have tried this out with -v $HOME/dockrcache_314:/root/.cache but ran into:

> example(basiliskRun)

bslskR> ## Don't show: 
bslskR> basilisk.utils::installConda()
PREFIX=/root/.cache/R/basilisk/1.5.0/0
Unpacking payload ...
Collecting package metadata (current_repodata.json): done                                                         
Solving environment: done

## Package Plan ##

  environment location: /root/.cache/R/basilisk/1.5.0/0

  added / updated specs:
    - _libgcc_mutex==0.1=main
    - ca-certificates==2020.1.1=0
    - certifi==2020.4.5.1=py37_0
    - cffi==1.14.0=py37he30daa8_1
    - chardet==3.0.4=py37_1003
    - conda-package-handling==1.6.1=py37h7b6447c_0
    - conda==4.8.3=py37_0
    - cryptography==2.9.2=py37h1ba5d50_0
    - idna==2.9=py_1
    - ld_impl_linux-64==2.33.1=h53a641e_7
    - libedit==3.1.20181209=hc058e9b_0
    - libffi==3.3=he6710b0_1
    - libgcc-ng==9.1.0=hdf63c60_0
    - libstdcxx-ng==9.1.0=hdf63c60_0
    - ncurses==6.2=he6710b0_1
    - openssl==1.1.1g=h7b6447c_0
    - pip==20.0.2=py37_3
    - pycosat==0.6.3=py37h7b6447c_0
    - pycparser==2.20=py_0
    - pyopenssl==19.1.0=py37_0
    - pysocks==1.7.1=py37_0
    - python==3.7.7=hcff3b4d_5
    - readline==8.0=h7b6447c_0
    - requests==2.23.0=py37_0
    - ruamel_yaml==0.15.87=py37h7b6447c_0
    - setuptools==46.4.0=py37_0
    - six==1.14.0=py37_0
    - sqlite==3.31.1=h62c20be_1
    - tk==8.6.8=hbc83047_0
    - tqdm==4.46.0=py_0
    - urllib3==1.25.8=py37_0
    - wheel==0.34.2=py37_0
    - xz==5.2.5=h7b6447c_0
    - yaml==0.1.7=had09818_2
    - zlib==1.2.11=h7b6447c_3

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  ca-certificates    pkgs/main/linux-64::ca-certificates-2020.1.1-0
  certifi            pkgs/main/linux-64::certifi-2020.4.5.1-py37_0
  cffi               pkgs/main/linux-64::cffi-1.14.0-py37he30daa8_1
  chardet            pkgs/main/linux-64::chardet-3.0.4-py37_1003
  conda              pkgs/main/linux-64::conda-4.8.3-py37_0
  conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.6.1-py37h7b6447c_0
  cryptography       pkgs/main/linux-64::cryptography-2.9.2-py37h1ba5d50_0
  idna               pkgs/main/noarch::idna-2.9-py_1
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
  libedit            pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0
  libffi             pkgs/main/linux-64::libffi-3.3-he6710b0_1
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
  ncurses            pkgs/main/linux-64::ncurses-6.2-he6710b0_1
  openssl            pkgs/main/linux-64::openssl-1.1.1g-h7b6447c_0
  pip                pkgs/main/linux-64::pip-20.0.2-py37_3
  pycosat            pkgs/main/linux-64::pycosat-0.6.3-py37h7b6447c_0
  pycparser          pkgs/main/noarch::pycparser-2.20-py_0
  pyopenssl          pkgs/main/linux-64::pyopenssl-19.1.0-py37_0
  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py37_0
  python             pkgs/main/linux-64::python-3.7.7-hcff3b4d_5
  readline           pkgs/main/linux-64::readline-8.0-h7b6447c_0
  requests           pkgs/main/linux-64::requests-2.23.0-py37_0
  ruamel_yaml        pkgs/main/linux-64::ruamel_yaml-0.15.87-py37h7b6447c_0
  setuptools         pkgs/main/linux-64::setuptools-46.4.0-py37_0
  six                pkgs/main/linux-64::six-1.14.0-py37_0
  sqlite             pkgs/main/linux-64::sqlite-3.31.1-h62c20be_1
  tk                 pkgs/main/linux-64::tk-8.6.8-hbc83047_0
  tqdm               pkgs/main/noarch::tqdm-4.46.0-py_0
  urllib3            pkgs/main/linux-64::urllib3-1.25.8-py37_0
  wheel              pkgs/main/linux-64::wheel-0.34.2-py37_0
  xz                 pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
  yaml               pkgs/main/linux-64::yaml-0.1.7-had09818_2
  zlib               pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3

Preparing transaction: done
Executing transaction: done
ERROR conda.core.link:_execute(698): An error occurred while installing package 'defaults::ncurses-6.2-he6710b0_1'.

CondaError: Cannot link a source that does not exist. /root/.cache/R/basilisk/1.5.0/0/pkgs/ncurses-6.2-he6710b0_1/share/terminfo/E/Eterm-color
Running `conda clean --packages` may resolve your problem.
()

Error in basilisk.utils::installConda() : 
  conda installation failed with status code '1'

If I remove that -v setting, the example runs fine, but the conda environment is built from scratch. That's what I am hoping to avoid. @LTLA might have a comment?

LTLA commented 3 years ago

The best bet is to set the BASILISK_USE_SYSTEM_DIR environment variable to build conda and its environments at package install time. This is typically used for including the conda stuff inside the container.

Not sure what happens it you want to mount it, though. The second-best bet is to set BASILISK_EXTERNAL_DIR to the mount directory so it looks there instead.