mloncode / workshop

Machine Learning for Software Engineering: modelling the source code workshop.
4 stars 1 forks source link

Make hands-on Colab friendly #2

Open m09 opened 4 years ago

m09 commented 4 years ago

After more deliberation, the target to achieve is: no external dependency, or as few as possible.

Every dependency should be installed in the docker image. For Colab, a special cell at the start of the notebook should contain the installation part. To make sure it runs only on Colab, it's possible to use:


try:
    import google.colab
    # do stuff
except:
    pass
m09 commented 4 years ago

I isolated the dependencies in separate scripts so that they can be installed in one line in Colab. Proposed setup cell is now:

try:
  import google.colab
  from os import environ
  is_git_dir = !git rev-parse --is-inside-work-tree
  if not is_git_dir:
    !git init
    !git remote add origin https://github.com/mloncode/workshop.git
  !git fetch origin master
  !git reset --hard origin/master
  # !./scripts/install-bigartm
  # !./scripts/install-tree-sitter
  # !./scripts/install-nltk-data
  # !pip install -r conf/requirements-setup.txt
  # !pip install -r conf/requirements.txt
  # environ["SOME_ENV_VAR"] = "some_value"
except:
  # Not on Colab, environment is handled separately
  pass

Last step will probably be to split the requirements even further on a per-notebook basis to install only the minimum env for each notebook.