Install cling jupyter kernel for interactive C++

[nthiery]

cling is a C++ interpreter based on clang/llvm. Would it be possible to install it on cocalc, together with its kernel? I'll be using it for teaching C++ this coming semester. We will be using Jupyter locally on the university's machines, but having a backup solution on cocalc would be helpful!

@SylvainCorlay implemented a Jupyter kernel implemented in C++ which is more robust than the one provided by cling. And will soon enable widgets and plots!

Installation instructions with conda:

conda install cling xeus-cling notebook -c QuantStack -c conda-forge

One should be able to install the kernel in another Jupyter deployment, but I haven't tried that.

[haraldschilly]

Hello, I took a note for cling in anaconda, which will be installed in a future upgrade of cocalc.

I don't know how to install the jupyter kernels, so, I leave this for later ...

[nthiery]

Hello, I took a note for cling in anaconda, which will be installed in a future upgrade of cocalc.

Thanks Harald!

I don't know how to install the jupyter kernels, so, I leave this for later ...

I have included some tentative instructions on the first comment to install the kernels.

For reference: Sylvain Corlay & co are working on an improved kernel, with upcoming supports for widgets, etc:

    https://github.com/QuantStack/xeus-cling

Apparently, the updated conda install scripts take care of installing the kernels in Jupyter.

[nthiery]

I have updated the installation instructions above.

[slel]

About making Jupyter kernels from conda also work on a different Jupyter install, maybe the discussion at the following StackOverflow answer helps? https://stackoverflow.com/questions/39296020/how-to-install-sagemath-kernel-in-jupyter/41595015#41595015

[slel]

There is demand for this feature. See for example this post from last May on the Software Carpentry "discuss" mailing list: http://lists.software-carpentry.org/pipermail/discuss/2017-May/005256.html and the following discussion. Harald, have you had a chance to try the instructions Nicolas provided in his edit to the first comment in this issue? https://github.com/sagemathinc/cocalc/issues/2324#issue-252654327

[haraldschilly]

Hmm, I don't remember, I've assigned this ticket to me and will update here when I have some progress.

[nthiery]

Hi, @slel forwarded me your exchange. I updated the installation instructions above. In a fresh miniconda install, nothing else is required: the kernels get installed automatically in the Jupyter of that miniconda's environment.

Now I don't know how cocalc's Jupyter setup works; maybe something additional is required to install it in Jupyter, if it lives outside of the conda environnement. Presumably something like running

      <cocalc>/bin/jupyter kernelspec install

from within the kernel directory <conda-dir>/share/jupyter/kernels/xeus-cling-cpp11.

Cheers,

[nthiery]

I forgot to mention: if the install went well, xeus-c++11 should appear in the Jupyter notebook's list of available kernels. Trying:

int i = 1;
i+3

in a cell should be good enough to test it.

[haraldschilly]

Hello, just a brief status update. All these Anaconda dependencies seem to start fighting with each other a little bit. Here is the issue with installing it right now:

$ conda install xeus-cling -c QuantStack -c conda-forge
Fetching package metadata ......................................................... 
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - r-fitdistrplus
  - r-irkernel
  - r-rematch
  - xeus-cling
Use "conda info <package>" to see the dependencies for each package.     

I'll see how it works without these R packages, and probably some follow up issues.

[haraldschilly]

Ok, I think I am at the point where I failed last time, too. After removing r packages, it ends up here:

ERROR conda.core.link:_execute_actions(339): An error occurred while installing package 'QuantStack::gcc-7-7.2.0-1'.                                                                                                                   
LinkError: post-link script failed for package QuantStack::gcc-7-7.2.0-1                                                                                                                                                               
running your command again with `-v` will provide additional information                                                                                                                                                               
location of failed script: /ext/anaconda3/bin/.gcc-7-post-link.sh                                                                                                                                                                      
==> script messages <==
<None>

Attempting to roll back.

LinkError: post-link script failed for package QuantStack::gcc-7-7.2.0-1
running your command again with `-v` will provide additional information
location of failed script: /ext/anaconda3/bin/.gcc-7-post-link.sh
==> script messages <==
<None>

and the error itself is from some sanity check: b"Installation failed: gcc is not able to compile a simple 'Hello, World' program.\n"

[SylvainCorlay]

@haraldschilly thanks for the details.

It would be awesome to see xeus-cling adopted in cocalc!

Quick question about your environment: which version of conda are you using? Someone else reported the post-link script failure earlier and we could not reproduce.

[slel]

In a CoCalc terminal:

$ /ext/anaconda3/bin/conda --version
conda 4.3.33

[haraldschilly]

@SylvainCorlay I have to add that I installed a lot into this anaconda environment and it didn't always went smooth. At some point in the future (during summer) it will all be stashed away and reinstalled from scratch. Maybe that's all it needs, not sure. However, you can try to add your own anaconda environment in cocalc and compile it. Here are some notes: https://github.com/sagemathinc/cocalc/wiki/Programming#anaconda-sage

edit: here is the output in the terminal

[slel]

The latest (64-bit Linux) Anaconda3 version available for download on the Anaconda download page is 5.0.1.

[slel]

Okay, I made some progress.

• installation instructions
• example worksheet

[SylvainCorlay]

@slel hey, I noticed that the "post-process the Jupyter Kernel" section is still in the instructions document. Would you be ok with removing it now that the issue is fixed?

[slel]

Thanks @SylvainCorlay for simplifying the installation process by removing the need for this post-processing. I updated the installation instructions.

[slel]

@haraldschilly hey, I hope with the release of Ubuntu 18.04 this will install seamlessly without conflicts with other installed packages.

[SylvainCorlay]

@haraldschilly, btw, we have fixed the link issue that you were seeing on installation of xeus-cling.

Let me know if this works better if you get back to this!

[slel]

@haraldschilly Now that CoCalc projects are based on Ubuntu 18.04, could you try again to install these kernels system-wide? The timing would be very good with JupyterCon this week (William's talk about CoCalc is coming in a few hours), and the fall term coming.

[haraldschilly]

I've searched via conda, and found a few bits and pieces. I could try installing the kernel, yes.

(base) /ext/anaconda5/lib$ conda list '.*xeus.*'
# packages in environment at /ext/anaconda5-py3:
#
# Name                    Version                   Build  Channel
xeus                      0.3.0                         0    conda-forge
(base) /ext/anaconda5/lib$ conda list '.*clang.*'
# packages in environment at /ext/anaconda5-py3:
#
# Name                    Version                   Build  Channel
clangdev                  3.9.1                   cling_4  [cling]  conda-forge

[SylvainCorlay]

@haraldschilly this version is very outdated and was very alpha-level work.

You need to install cling, xeus-cling and related channels from the QuantStack channel.

[haraldschilly]

Well, I just want to report some progress, such that nobody thinks this is a dead ticket :skull_and_crossbones:

(base) salvus@25957896d5e0:/ext/anaconda5-py3$ conda install xeus-cling -c QuantStack -c conda-forge   
Solving environment: failed                                                                            

UnsatisfiableError: The following specifications were found to be in conflict:                         
  - gcc -> isl=0.12                                                                                    
  - xeus-cling                                                                                         
Use "conda info <package>" to see the dependencies for each package.                                   

I am not sure what this isl means, and no idea which package is requiring it. Do I have start a new separate anaconda install just for xeus? Or should I attempt to compile the code in the default python3 environment? I have to look closely at the dependencies. … ubuntu at least offers gcc 7 out of the box :sun_with_face:

[slel]

Things work well as a user. Please find here installation instructions, quite simplified with respect to the February 2018 version. Hoping this can be of any help for the system-wide install...

• instructions: install Xeus-Cling C++ kernels for Jupyter in Ubuntu 18.04 CoCalc project
• example worksheet to use the Xeus-Cling C++ kernels for Jupyter in CoCalc

[haraldschilly]

ok, so, this looks like I have create a sub-environment in the global anaconda installation

[SylvainCorlay]

Hey, I am back in Paris tomorrow. Would be happy to help troubleshoot the installation when back.

[haraldschilly]

C++ 17 is now a jupyter kernel via xeus, using that idea of adding a nested environment. I don't know of any concrete use case, hence no idea if that's sufficient.

[SylvainCorlay]

Wow this is really great news!

[slel]

Would it make sense to also provide the C++ 11 and C++ 14 kernels from xeus?

[haraldschilly]

Please don't ask me about C++ ;-)

I'm trying to avoid clutter and supporting the one with the highest version number seems to make sense for me.