Different flavours of bundles

[betatim]

@rgbkrk pointed out in #7 that conda also has R environments and that the full pydata stack is pretty big in terms of space.

What different flavours would we want to make available?

• minimal python
• minimal R (?)
• full pydata stack

More, less, others??

[rgbkrk]

I think we should probably a small helper that can take a conda yaml (or maybe conda handles this directly). Then for each "stack" that we provide, it's a separate yaml that we build from.

As for stacks, I think we can do the three you mention and likely keeping even "full pydata" as pretty trim - the pandas + scikit-learn combo is pretty good (since it includes numpy and scipy).

P.S. pip totally works in this setup -- I just used it to install requests and it put it in the proper local site-packages. Amusingly, conda doesn't because it's a symlink to my conda setup:

kylek@flixbook /private/tmp/minikernel/bin$ ls -la
total 4512
drwxr-xr-x  41 kylek  wheel     1394 Jan 12 13:41 .
drwxr-xr-x   8 kylek  wheel      272 Jan 12 13:41 ..
-rwxrwxr-x   1 kylek  wheel    13576 Sep 27 07:49 .openssl-libcrypto-fix
-rw-rw-r--   1 kylek  wheel      225 Sep 27 07:49 .openssl-post-link.sh
lrwxr-xr-x   1 kylek  wheel        8 Jan 12 13:41 2to3 -> 2to3-3.6
-rwxrwxr-x   1 kylek  wheel      103 Jan 12 13:41 2to3-3.6
lrwxr-xr-x   1 kylek  wheel       85 Jan 12 13:41 activate -> /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/bin/activate
-rwxrwxr-x   1 kylek  wheel     5110 Jan 12 13:41 c_rehash
lrwxr-xr-x   1 kylek  wheel       82 Jan 12 13:41 conda -> /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/bin/conda
lrwxr-xr-x   1 kylek  wheel       87 Jan 12 13:41 deactivate -> /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/bin/deactivate
-rwxrwxr-x   1 kylek  wheel      168 Jan 12 13:41 easy_install
lrwxr-xr-x   1 kylek  wheel       12 Jan 12 13:41 easy_install-3.6 -> easy_install
lrwxr-xr-x   1 kylek  wheel        7 Jan 12 13:41 idle3 -> idle3.6
-rwxrwxr-x   1 kylek  wheel      101 Jan 12 13:41 idle3.6
-rwxrwxr-x   1 kylek  wheel      129 Jan 12 13:41 ipython
-rwxrwxr-x   1 kylek  wheel      129 Jan 12 13:41 ipython3
-rwxrwxr-x   1 kylek  wheel      146 Jan 12 13:41 jupyter
-rwxrwxr-x   1 kylek  wheel      172 Jan 12 13:41 jupyter-kernelspec
-rwxrwxr-x   1 kylek  wheel      146 Jan 12 13:41 jupyter-migrate
-rwxrwxr-x   1 kylek  wheel      156 Jan 12 13:41 jupyter-troubleshoot
-rwxrwxr-x   1 kylek  wheel   560888 Sep 27 07:49 openssl
-rwxrwxr-x   1 kylek  wheel      112 Jan 12 13:41 pip
lrwxr-xr-x   1 kylek  wheel        8 Jan 12 13:41 pydoc -> pydoc3.6
lrwxr-xr-x   1 kylek  wheel        8 Jan 12 13:41 pydoc3 -> pydoc3.6
-rwxrwxr-x   1 kylek  wheel       86 Jan 12 13:41 pydoc3.6
-rwxrwxr-x   1 kylek  wheel      138 Jan 12 13:41 pygmentize
lrwxr-xr-x   1 kylek  wheel        9 Jan 12 13:41 python -> python3.6
lrwxr-xr-x   1 kylek  wheel        9 Jan 12 13:41 python3 -> python3.6
lrwxr-xr-x   1 kylek  wheel       17 Jan 12 13:41 python3-config -> python3.6m-config
-rwxrwxr-x   1 kylek  wheel     9500 Dec 23 11:21 python3.6
lrwxr-xr-x   1 kylek  wheel       17 Jan 12 13:41 python3.6-config -> python3.6m-config
-rwxrwxr-x   1 kylek  wheel     9500 Dec 23 11:21 python3.6m
-rwxrwxr-x   1 kylek  wheel     2043 Jan 12 13:41 python3.6m-config
lrwxr-xr-x   1 kylek  wheel       10 Jan 12 13:41 pyvenv -> pyvenv-3.6
-rwxrwxr-x   1 kylek  wheel      443 Jan 12 13:41 pyvenv-3.6
-rwxr-xr-x   1 kylek  wheel  1432944 May 25  2016 sqlite3
-rwxr-xr-x   1 kylek  wheel    12712 Mar 16  2015 tclsh8.5
lrwxr-xr-x   1 kylek  wheel        2 Jan 12 13:41 unxz -> xz
-rwxrwxr-x   1 kylek  wheel      126 Jan 12 13:41 wheel
-rwxr-xr-x   1 kylek  wheel    25160 Mar 16  2015 wish8.5
-rwxrwxr-x   1 kylek  wheel    96556 Dec 28 12:26 xz

Seems --copy is only slightly adhered to. /cc @quasiben

[rgbkrk]

For context Ben, we're creating an isolated conda environment using:

conda create -p /tmp/snakestagram --copy -y -q python=3

In the hopes it's portable across machines as a bundled kernel for the nteract desktop app.

/cc @aterrel

[jdetle]

I think python + numpy would be a useful middle ground too

[quasiben]

It makes sense that conda/activate/deactivate are symlinked. Conda expects you are staying on the machine on which the env was created. However, with --copy as you know, the libraries are not sym/hard-link and are the new files. This means, you can relocate the contents but you won't have full use of the executables/scripts listed above and you will have to munge a bit for other installed scripts.

All other installed executables will have a header like: #!/tmp/snakestagram/bin/python. When you relocated that folder that probably won't be valid anymore. You probably know this but i'll state it for completeness, you can still execute that particular script by issuing the full path to python and the path to that script:

/other/path/snakestagram/bin/python /other/path/snakestagram/bin/2to3

[rgbkrk]

@quasiben - thanks for chiming in! I've been using /path/to/snakestagram/bin/python -m IPython for accessing main libs, this way is golden too. If I install conda in the conda environment can I conda install more packages?

[quasiben]

In theory yes, but I've seen things go awry. I'll try and reproduce.