[discussion] Why do we need a kernel to do this instead of some cell magics?

[liudongqing]

I am wondering whether it is too heavy to implement a kernel to do tensorflow job? My understanding is that we can use a cell magic to achieve similar result: send the code to kubeflow and get back the logs.

Am I miss something? Thanks!

[gaocegege]

From the user level, we do just need some magic commands. But the magic commands are supported from the kernel, e.g. ipython or ir. Thus if we need to add magic commands, we could:

• Add the commands into ipython and implement the corresponding logics in ipython. In this way, we need to maintain a fork of ipython. I think we cannot merge it into ipython mainline.
• Implement a customized kernel. In this way, we can support some more use cases. In the future, maybe we could support model serving and hyperparameter tuning in the kernel.

I think maintaining a fork is not what we want, thus we decided to implement a kernel. But there is a problem in this way: How to run the python code natively, not via kubeflow.

To solve the problem, we use sos-kernel to support multiple kernels in one notebook. We have a demo for it:

You can select kernel in the right top sidebar of the cell. And run different cells using different kernels.

[gaocegege]

If you have any idea about it, we can discuss it here. Ciao is experimental now and welcome breaking changes :smile:

[gaocegege]

/close