nansencenter / nersc-vagrant

Nansen-Cloud provisioning with vagrant
GNU General Public License v3.0
0 stars 0 forks source link

Error with current setup of testconda on windows #4

Closed aleksandervines closed 8 years ago

aleksandervines commented 8 years ago

As the shared folder is mounted from a windows file system, it does not support symlinking. You can therefore not install miniconda there.

Why do we want to install miniconda in the shared folder on the vm?

aleksandervines commented 8 years ago

Testing with conda_dir: /home/vagrant/miniconda works.

akorosov commented 8 years ago

It takes long time after each destroy to download all conda intallers. To 'cache' the miniconda.sh and other packages I install it in shared folder. After I do 'destroy' shared folder and it's contents is kept and next time 'up' is much faster. We can of course sacrifice speed to make it compatible with running from windows. At least that has to be done for miniconda dir.

akorosov commented 8 years ago

Hmm, I found how to 'cache' packages: http://stackoverflow.com/questions/22878104/is-it-possible-to-conda-install-with-dependencies-completely-offline/22917229#22917229

And hot to free up space in the base box: http://stackoverflow.com/questions/27567560/anaconda-vs-miniconda-space

akorosov commented 8 years ago

That method doesn't work. Suddenly many dependency errors appeared. Let's try to install minicond in home directory but move miniconda/pkgs to the shared directory and symlink.

akorosov commented 8 years ago

That role can be in the testconda only. The base box will keep pkgs inside but will clean them before packaging.

akorosov commented 8 years ago

That 'caching' requires too much testing. Speeding up conda slows down me. miniconda will be installed in shared dir for 'testconda' machine only. 'base' and other boxes will place it to /home/vagrant/miniconda

akorosov commented 8 years ago

Windows users can change 'conda_dir' variable in provisioning/group_vars/testconda

akorosov commented 8 years ago

Ok, I got it! Caching is done in several steps rather than in one:

  1. install conda
  2. copy package files *.bz2 from cache (e.g. shared/conda_pkgs) into miniconda/pkgs
  3. conda installs all the packages
  4. copy *.bz2 and urls.txt from miniconda/pkgs into a cache (e.g. shared/conda_pkgs)