What is kernel in jupyter lab

[Max-ChenFei]

ref: https://docs.jupyter.org/en/latest/projects/kernels.html

[xp1632]

By checking jupyter's lastest kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels, we find two more options for JAVA kernel:

Bacatá : Example Ganymede : Examples IJava: Binder online demo

[xp1632]

Ganymede based on JShell is most promising at first glance

[xp1632]

While Ganymede requires Java 11+, Fiji is bundled with Java version 8, but it works the same with Java 11:

Here is the reference link: https://forum.image.sc/t/run-fiji-or-imagej-headless-mode-with-java-11/73989

[xp1632]

Tutorial of Running Java in a Jupyter Notebook based on IJava or Ganymede.

IJava is also based on JShell

[Max-ChenFei]

Ganymede based on JShell is most promising at first glance

ganymeda is better

[xp1632]

We first learn the theory of kernel:

1. What is kernel in jupyter?

- Kernels:

- Kernels are programming language specific processes that run independently 
and interact with the Jupyter Applications and their user interfaces.

- the default kernel called ipykernel,is the reference Jupyter kernel built on top of IPython.
    which provides powerful environment for interactive computing in Python

- In addition to Python, other programming language can also be used in the notebook
    based on different language kernels

- jupyter-client

-jupyter-client contains the authoritative description of the Jupyter messaging protocol
    which is used by clients to communicate with the kernels

- here we have more details of how kernel works in Jupyter:
https://jupyter-client.readthedocs.io/en/stable/messaging.html

- by checking this we could understand how kernels work in Jupyter

- Xeus

- Xeus is a framework meant to facilitate the implemtation of kernels for Jupyter
- since I don't need to implement a kernel, I will pass this part

1.1 Architecture of Jupyter projects:

https://docs.jupyter.org/en/latest/projects/architecture/content-architecture.html

1.1.1 a high-level overview of jupyter projects:

-we also have our simple version as :

-

1.1.2 IPython Kernels

• The IPython kernel provides computation and communication with the frontend interfaces like notebook

-Terminal Python:

• the model REPL(Read-Eval-Print-Loop) of IPython works as:

  • prompt the user for some code
  • when user enter code
  • execute the code in the same processes

• REPL: -Stands for “read-eval-print-loop”.

  • An REPL is a program that reads user commands and inputs, evaluates them, and prints results for the user to view.

  • These steps runs in a perpetual(continuous) loop,

    • allowing the user to prototype code, interact with data, view results on the fly.
    • since the data and code stay open in memory after evaluation while the REPL is running

-The IPython Kernel:

• All the other interfaces such as the Notebook, the Qt console, ipython console in the terminal, and third-party interfaces -- uses the IPython Kernel.

• IPykernel is a separate process which is responsible for running user code

• Frontends such as notebook or qtConsole, communicate with iPython Kernel
  using JSON messages over ZeroMQ sockets(asynchronous I/O model) https://zeromq.org/

![image] (https://docs.jupyter.org/en/latest/_images/ipy_kernel_and_terminal.png)

• The protocol used between frontend and IPython Kernel is described in Messageing in Jupyter

https://jupyter-client.readthedocs.io/en/stable/messaging.html

1.2 So let's check jupyter-client focusing on how kernels work and deal with messaging:

• A single kernel can be simultaneously(at the same time) connected to one or more frontend

• to understand the communication between Front-end and kernels, we need to know basic socket type ROUTER,DEALER,PUB,SUB in ZeroMQ

1.4 Socket API in ZeroMQ

• I've summarize it here : https://github.com/Max-ChenFei/VPE_IP/issues/6

[Max-ChenFei]

@xp1632 Kernel Example using the command line https://github.com/gap-packages/JupyterKernel

This is a solution when we need to call the function of a close-sourced software

[xp1632]

After figuring ZeroMQ socket, we'll continue the rest part of jupyter-client focusing on how kernels work and deal with messaging

1.5 There are five functions of kernel based on ZeroMQ sockets:

Shell

• This single ROUTER socket allows multiple incomming connection from frontend
• And in this socket, frontend makes requests of code execution, promps, object informations
• Communication on this socket is a sequence of request/reply actions from each frontend and kernel

IOPub

• This socket is the broadcast channel where the kernel publishes all side effects

  • such as stdout, stderr,debugging events etc.

• It also publishes the requests from any client that we get through shell socket

• It also publishes its own requests on the stdin socket

• Additionally, in a multi-client scenario, this socket 'broadcast' the side effects and information taking place with one client, to all other clients over shell channel.

  • This can be useful in collaborative scenarios

stdin:

• this ROUTER socket is connected to all frontends,

• and for active frontend(when raw_input) is called

• it allows the kernel to request input from the active frontend

• DEALER socket

  • The active frontend that executed the code has a DEALER socket.
  • acts as a "virtual keyboard" for the kernel whild this communication is happenning

• Kernel request

  • Normally frontend may display such kernel request using a special input widget
  • to indicate the user that the type input is for kernel instead of normal commands

• Information tag

• All messages are tagged

  • to let the clients know which messages come from their interaction with kernel
  • and which ones are from other clients

Control

• The control channel is used for shutdown, restart, debugging messages
• this channel is identical to Shell
  • but operate on a separate socket to avoid ququeing behind execution requests
• The control channel is recommended to run in a separate thread from shell
  • so shutdown or debug message could be processed immediately

Heartbeat

• This socket is to ensure the connection between frontend and kernel
  • by allowing sending simple bytestring messages

[xp1632]

---

1.6 General Message Format:

• There are certain format for the message we passed via sockets
• A message is composed of five dictionaries

Message Header:

• The message header contains information about the message:

  • unique identifiers for
  • the originating session
  • the actual message_id
  • type of message
  • version of Jupyter protocol
  • date/timestamp when the message was created

  {
  'msg_id' : str, # typically UUID, must be unique per message
  'session' : str, # typically UUID, should be unique per session
  'username' : str,
  # ISO 8601 timestamp for when the message is created
  'date': str,
  # All recognized message type strings are listed below.
  'msg_type' : str,
  # the message protocol version
  'version' : '5.0',
  }

Parent header

• When a message is the result of another message

  • such as a side-effect (output or status) or direct reply -the parent_header is a copy of the header of the msg that cause the current message

• _reply messages MUST have a parent_header

• side-effects typically have a parent, if not, an empty dict should be used

• This parent header is used by clients to route message handling to correct place

  • such as outputs to a cell

{
    # parent_header is a copy of the request's header
    'msg_id': '...',
    ...
}

Metadata

• the Metadata dict contains information about the message that is not part of content-architecture
• This can be used to store information about request and replies
  • such as extesions adding information about context of request and execution

Content

• content dict is the body of the message
• its structure is determined by the msg_type in the header

Buffers

• In the final part of a message, it is a list of addition binary buffers
• No official messages make use of these buffers
• But they are used by extension messages, such as IPython Parallel's apply

---

A full message:

• Combining all of these, a complete message can be represented as :
• the following dictionary of dictionaries( and one list)

{
    "header" : {
        "msg_id": "...",
        "msg_type": "...",
        ...
    },
    "parent_header": {},
    "metadata": {},
    "content": {},
    "buffers": [],
}

[xp1632]

• The plan for tommorw is to finish the rest part of kernel communication: https://jupyter-client.readthedocs.io/en/stable/messaging.html#the-wire-protocol

• Understand

  • Wire Protocol
  • Python API
  • Msg on shell
  • Msg on Control
  • Msg on IOPub
  • Msg on stdin
  • Custom Messages for extension

---

• Then after having primitive understanding how kernel works and communicates,
  • we start to install Ganymede in jupyterlab
  • If more Java, JShell knowledge needed during installing, we 'll check that

[Max-ChenFei]

• The plan for tommorw is to finish the rest part of kernel communication: https://jupyter-client.readthedocs.io/en/stable/messaging.html#the-wire-protocol

• Understand

  • Wire Protocol
  • Python API
  • Msg on shell
  • Msg on Control
  • Msg on IOPub
  • Msg on stdin
  • Custom Messages for extension

• Then after having primitive understanding how kernel works and communicates,

  • we start to install Ganymede in jupyterlab
  • If more Java, JShell knowledge needed during installing, we 'll check that

After installing Ganymeda, try official examples first. If questions, you can create issues in that repo

[xp1632]

1.7 We continue with The Wire Protocol

Wire Protocol

• We check the general message format in jupyter:

A full message:

• Combining all of these, a complete message can be represented as :
• the following dictionary of dictionaries( and one list)

{
    "header" : {
        "msg_id": "...",
        "msg_type": "...",
        ...
    },
    "parent_header": {},
    "metadata": {},
    "content": {},
    "buffers": [],
}

• But this is only a logical representation of the contents

• the actual implmentation in ZeroMQ at wire level is different

• Every message is serialized to a sequence of at leat six blob of bytes:

[
  b'u-u-i-d',         # zmq identity(ies)
  b'<IDS|MSG>',       # delimiter
  b'baddad42',        # HMAC signature
  b'{header}',        # serialized header dict
  b'{parent_header}', # serialized parent header dict
  b'{metadata}',      # serialized metadata dict
  b'{content}',       # serialized content dict
  b'\xf0\x9f\x90\xb1' # extra raw data buffer(s)
  ...
]

Prefix

• The front part of message is the routing prefix of ZeroMQ
• which can be zero of more socket identities
• For IOPub, it should be one prefix, which is topic of subscribers, e.g. execute_result

Auchentication

• After the delimiter is the signature of message, used for authentication
• if authentication is disabled, this should be an empty string
• by default, we use hashing function sha256 to compute the signatures

Actual Message

• After the signatureis the actual message, in four frames of bytes

• The four dictionaries that compose a messgae are serialized separately

  • inthe order of header
  • parent header
  • metadata
  • content

• The default and most common serialization is JSON

• We can do the serialization by any functions that turns a dict into bytes

---

Python API

• All messages sent to or received by any IPython message handler:
• should have this extended struture after deserialization:

{
  'header' : dict,
  # The msg's unique identifier and type are always stored in the header,
  # but the Python implementation copies them to the top level.
  'msg_id' : str,
  'msg_type' : str,
  'parent_header' : dict,
  'content' : dict,
  'metadata' : dict,
  'buffers': list,
}

---

[xp1632]

1.8 Message on the shell (ROUTER/DEALER) channel

Request-Reply Pattern

• Client side send execute_request via its DEALER socket
• Kernel recieves the request and publish status:busy on IOPub
• Kernel process request and send back execute_reply
• After processing the request and publishing associated IOPub message
  • the kernel publish status: idle

Status

• All replt messages have a 'status' field, which will have one of following values:

  • status :ok means the request was processed successfully, the remaining content of reply is specified below

  • status: error means the request failed due to an error` the usual content of a successfully reply should be omitted

this is presented when status: error

{
   'status' : 'error',
   'ename' : str,   # Exception name, as a string
   'evalue' : str,  # Exception value, as a string
   'traceback' : list(str), # traceback frames as strings
}

Execute request type

• This message type is used by frontends to ask the kernel to execute code.

• Here's an example of execute_request

content = {

'code' : str,
# Source code to be executed by the kernel, one single string or more lines.

...

'user_expressions':dict.
# A dict mapping names to expressions to be evaluated in the user's dict
# this information could be retrieved to distinguish between:
# inner prompts, client prompts, different frontend prompts interacting with same kernel

'allow_stdin': True
# If true, code running in the kernel could prompt user for input
# with input_request message

...
}

Execute reply type

content ={

'status' : str,
# One of :'ok' OR 'error' OR 'aborted'

'execution_count': int,
# A global kernel counter that increased by one with each request
# Typically used by clients to display prompt numbers to the user

}

• When status is 'ok', the following fields are present:

{

'payload' : list(dict),
# Payloads are a way to trigger frontend actions from the kernel.

'user_expressions':dict
}

Payloads

• Payloads are a way to trigger frontend actions from the kernel.

  • page : display data in a pager

  • set_next_input : create a new output

    • used to create new cells in the notebook
    • or set the next input in a console interface -example:%load

  • edit_magic: open a file for editing

    • only QtConsole currently support edit payloads -%edit

  • ask_exit: instruct the frontend to prompt the user for exit -allows the kernel to request exit via %exit in IPython -only for console frontends

---

Other Message Type in shell channel:

• Introspection:

  • the kernel decide what information should be displayed to the user
  • inspect_request, inspect_reply

• Completion:

  • The code context in which completion is requested
  • complete_request, complete_reply

• History:

  • The kernel has all the actual execution history stored
  • Clients could explicitlt request history from a kernel when needed
  • history_request, history_reply

• Comm Info

  • When a client needs current open comms in the kernel
    • it can issue a request by comm_info_request
    • when target_name is specified, the reply will only contain target information

• Kernel Info

  • when client needs to know kernel info such as
    • language/ language version number/ IPython message spec version
  • kernel_info_request

---

• Code Completeness

• When the user enters a line in a console,

  • the console needs to decide whether to execute current code
  • Or showing a continuation prompt for further input

• There are four possible replies:

  • complete code is ready to be executed
  • incomplete code should prompt for another line
  • invalid code will be sent foe execution so user could see error soonest
  • unknown if kernel cound not determine, the code may defaultly be sent to execution

---

[xp1632]

---

1.9 Messages on Control(ROUTER/DEALER) channel

• Kernel shut

  • the client can request the kernel to shut itself down
  • on shutdown, client application will execute a last minute sanity check
  • and forcefully terminate any alive kernel

• Kernel Interrupt

  • In case a kernel can not catch OS's interrupt signals
  • kernel uses a message to be notified instead

• Debug request

  • This message type is used with debugging kernels to request specific actions:
    • dummyCell
    • debugInfo
    • inspectVariables
    • copyToGlobals

---

1.10 Message on the IOPub(PUB/SUB) channel

• Streams(stdout,stderr)

• Display Data

  • to bring back data should be displayed (html,svg,etc.) in the frontends
  • This data is published to all frontends
  • Each message can have multiple representation of data in a JSON'able structure
  • Frontend can decide which one to use and how

content = {

    # The data dict contains key/value pairs, where the keys are MIME
    # types and the values are the raw data of the representation in that
    # format.
    'data' : dict,

    # Any metadata that describes the data
    # In IPython, only width and height of images
    'metadata' : dict,

    # Information not to be persisted to a notebook or other documents.
    # Intended to live only during a live kernel session.
    # We could store transient key display_id for future update
    'transient': dict,
}

• Update Display Data

  • When a display_id is specified for a display in 'transient',
  • it can be updated later with an update_display_data message

• Code inputs

  • To let all frontend know
    • what code is being executed now
  • These messages contains a re-broadcast of code part of execuete_request

• Execution results

  • Result of an execution are published as execute_result
  • The data result is any JSON object

content = {

    # The counter for this execution is also provided so that clients can
    # display it, since IPython automatically creates variables called _N
    # (for prompt N).
    'execution_count' : int,

    # data and metadata are identical to a display_data message.
    # the object being displayed is that passed to the display hook,
    # i.e. the *result* of the execution.
    'data' : dict,
    'metadata' : dict,
}

• Execution errors

  • when an error occurs during code execution

• Kernel status

  • is used by frontends to monitor the status of the kernel

content = {
    # When the kernel starts to handle a message, it will enter the 'busy'
    # state and when it finishes, it will enter the 'idle' state.
    # The kernel will publish state 'starting' exactly once at process startup.
    execution_state : ('busy', 'idle', 'starting')
}

• Clear output: is used to clear visible output on the frontend

• Debug event: is used by debugging kernels to send debugging events to frontend

---

1.11 Messages on stdin(ROUTER/DEALER) channel

• In stdin channel, the request/reply pattern goes in opposite direction of most kernel comminication

• With stdin socketm Kernel makes the request, the single frontend provides response

• This allows code to prompt user for a line of input

  • which would normally be read from a terminal through stdin

---

1.12 Custom Messages

• A messaging system for developers to add their own objects with Frontend and Kernel -allow them to communicate with each other with notion of Comm

• Comm exists on both Kernel side and Frontend side, and can comminicate in either direction

• Opening a Comm

#Open Comm produce this 'comm_open' msg, to be sent to the other side
{
  'comm_id' : 'u-u-i-d',
  'target_name' : 'my_comm',
  'data' : {}
}

- Every Comm has an ID and target name.
- After `comm_open` messgae has been sent,
    there should be corresponding Comm instance on both sides

• Comm Message

#comm_msg
{
  'comm_id' : 'u-u-i-d',
  'data' : {}
}

• Comm messages are one-way communication to update comm State

• Tearing Down Comms

• since comms live on both side

• when one side destroys a comm

• the other side should be notified by comm_close message

[xp1632]

The message communication protocol between the frontend and kernel, jupyter client, is finished.

• What is kernel in jupyter lab?
  • To summarize, we can say, kernel is an important component in Jupyter, which takes charge of communication with frontend and code execution in backend.

• To develop kernels for other languages, there are three ways:

• 

• Wrapper kernel: reuse communication of IPykernel, implement only execution

• Native Kernel: implement both execution and communication in target language

• Xeus based kernel: wrapper with better protocol for c++

• Thus, by checking the dependency of the two java kernel we're going to investigate
• we could presume,
  • Ganymede is a native kernel
  • Bacatá is a wrapper kernel