search

conda-incubator / condacolab

Install Conda and friends on Google Colab, easily
MIT License
341 stars 49 forks source link

add docs and examples for custom constructor installers #10

Closed jaimergp closed 3 years ago

jaimergp commented 3 years ago

Comes from discussion at #8

@alexmalins, can you try these instructions? Thanks!

alexmalins commented 3 years ago

@jaimergp This is really well done - thanks! I'm testing it now.

Running constructor locally on my Windows machine resulted in a constructor/ncurses related file not found error. I'm sure it's fixable by following up the error message, or secondly by switching to building via WSL. However instead I built the installer directly on Colab as a fast solution for now as this is what I did before.

In fact I wonder if offering users instructions to build installers directly on Colab rather than locally is a better option. It would 1. avoid my issue and potentially others relating to local environments (so more robust), 2. users may not have conda installed already locally (e.g. locked-down workplace computers), and 3. colab has fast internet and CPUs so the installer build is likely to be faster. The downside is it requires uploading the yaml/sh file to colab and downloading the completed installer, so two more file transfer steps compared to the current example.

Other suggestions:

Once the upload of the installers are finished to my host I'll test the functionality.

jaimergp commented 3 years ago

Oh, this is awesome feedback! I love the Colab as a builder idea. Your other suggestions are spot-on; those were oversights on my end.

alexmalins commented 3 years ago

Ok, it worked fine except for the pip install option. I tagged in python -m pip install pip-install-test in one example, but import pip_install_example was failing. This might be because it's not a dependency of any conda package from the same build, or because I messed something up (apologies I cannot check now - it's late in Japan). Will check tomorrow.

One more suggestion: in the example construct.yaml lines 2 & 3, maybe append some # comments to let users know they are free to set the name and version number to something appropriate for their build.

alexmalins commented 3 years ago

Sorry I meant also to share the basic workflow for using constructor on colab:

!pip install -q condacolab
import condacolab
condacolab.install()
from google.colab import files
uploaded = files.upload()    # upload construct.yaml
!conda install -c conda-forge constructor
!constructor .
files.download('condacolab-0.1-Linux-x86_64.sh')

However files.download() was very slow for me. Again I didn't have time to debug what was going on. I just copied the installer to Google Drive then downloaded it from there. Maybe it works well for you? Another option could be to right click download on the files list in the left sidebar.

jaimergp commented 3 years ago

Hi @alexmalins, I tried and it works for me! Did you uncomment the post_install line in construct.yaml?

jaimergp commented 3 years ago

This is working for me. I am going to merge to main and hopefully you can test the tutorial linked in the FAQ? Thanks!

jaimergp commented 3 years ago

See here: https://github.com/jaimergp/condacolab#how-can-i-cache-my-installation-i-dont-want-to-wait-every-time-i-start-colab

alexmalins commented 3 years ago

Hi @jaimergp, yeap I did. I just tried it again with a different conda package without success. Here are the constructor files I used: https://alexmalins.com/condacolab/constructor.yaml https://alexmalins.com/condacolab/install-pip-dependencies.sh Was building the constructor installer on colab and hosting here: https://alexmalins.com/condacolab/condacolab-radioactivedecay-pip-0.1-Linux-x86_64.sh

jaimergp commented 3 years ago

Thanks for the report! I have enabled better logging in a new branch and now I can see why yours is failing:

__installer__.sh: /usr/local/pkgs/post_install.sh: /bin/sh^M: bad interpreter: No such file or directory
ERROR: executing post_install.sh failed

Your shebang line contains a weird newline character! This might be a weird Win/Unix interaction, I'm afraid. Can you follow the newly published Colab tutorial so all the edition happens in Unix world?

alexmalins commented 3 years ago

Ah right, this looks like Windows line end character issue. I've encountered that exact error in the context of transferring between Windows and Unix-based supercomputers many times before. Sorry!

alexmalins commented 3 years ago

Switching the line end chars CR+LF -> LF fixed it. Thanks again for all these changes!

jaimergp commented 3 years ago

Yay! I should cut a new release soon then.