Feature request: universal REPL

[benjaminrich]

I'm pretty impressed with this:

https://gitlab.com/Screwtapello/kakoune-repl-buffer

(also, watch the video here: https://asciinema.org/a/379916)

If something like this could be added to helix, it could potentially change my life. I know some other users are looking for this kind of feature as well (#1710).

Any thoughts?

[the-mikedavis]

I think this would be a cool feature that could fit into core. At first I thought it might be better as a plugin since it isn't based on some hypothetical REPL standard (something like LSP for REPL) but it looks like the plugin is just interfacing with stdin/stdout of the repl program which seems straightforward and consistent.

There are some rough edges to that plugin that I think could be designed around: the :prompt based capabilities at the end of the asciicast effectively turn kakoune into a line-editor. That fine for simple lines on a REPL but it's limiting for larger code fragments and it would be ideal to be able to use the full buffer editing power of Helix instead of prompts.

I don't have any ideas to get the ball rolling but it could be smart to canvas popular repl plugins from vim/emacs/etc and source ideas there.

[benjaminrich]

@the-mikedavis, thanks for your response.

To be clear, I'm not suggesting to re-implement that plugin exactly (I agree with you that it is rough and about the :prompt functionality, etc). I was just trying to point to an example that goes in the right direction. Ideally, I would like it to be a nice, well integrated feature with easy, consistent key bindings. I am thinking that it could be useful to have a REPL minor mode (with a sticky option) to "send code to REPL" and that sort of thing (admittedly I haven't quite thought this through, it's just an idea).

I mostly do work in R, and a bit of julia. For R, there is the very nice nvim-R plugin, that has REPL functionality, with lots of features. Unfortunately, it only works for R and can't be used for other interactive languages (like julia for instance), where you would have to rely on something like slime.vim, thus creating a major inconsistency in the workflows. That is the situation I would like to rectify.

[Sidenote: there is an official julia plugin for R, but it doesn't have any REPL features. It does however allow for nice entry of some Unicode characters, which can also be used in non-julia files, and is a useful feature in its own right. But that is a topic for a different discussion.]

Also, I am very aware of the "kitchen sink" and "bloat" concern. Indeed, this functionality goes beyond simple editing, but for interactive languages (and there are many) it can be considered "essential" (I would never go back to copy-pasting code into a different terminal/window). Then there are external tools like tmux that could be leveraged, but here we have the problem of catering to all kinds of users with their vastly heterogeneous environments, and so it really is better not to rely on the presence of external tools (approaches like that also always feel kind of kludgy). Obviously, there may be users that do not consider this feature important, but I hope there are enough who do.

Plugins create a plethora of fragmentation problems (as exemplified by nvim-R above), which is why I was very excited to hear that you feel this could fit into core. Consistent, well designed and well integrated with the rest of helix - that is the best solution. It could be a "killer" feature that distinguishes helix from other modal editors.

[Other sidenote: SpaceVim has built-in REPL functionality and defines some consistent bindings across different languages (layers in SpaceVim). Unfortunately they are pretty cumbersome because they require like 4 key presses, which is really too much].

Thanks, and sorry for the long post.

[dgkf]

Just wanted to add to the reference material. I'm most familiar with vim/neovim's Terminal mode, which is a full interactive terminal emulator rendered in a window. I use a plugin (vim-slime) to map a keybind to send code to the terminal. This is modeled after emacs' slime mode.

It's not pretty (I disabled most other plugins), but hopefully it's clear how it works.

As far as use cases go, I'm also coming from an R-for-analytics background. In the domain of scientific programming, interactive execution in a REPL is a very central part of development.

[riwsky]

Something like LSP for REPL

the Jupyter protocol fits this description, with plenty of implementations by this point

[benjaminrich]

Something like LSP for REPL

the Jupyter protocol fits this description, with plenty of implementations by this point

I always thought of Jupyter as something web based, but maybe I don't understand it very well. Could it work in a console editor like helix?

[riwsky]

Yeah.

While the web-based notebook is by now Jupyter's calling card, the project started life as a python console REPL. It still supports that console REPL, along with a few other frontends (a Qt REPL, alternative web UIs, etc). They all talk to backends over a ZeroMQ-based protocol.

Granted: these first-party UIs represent the overwhelming majority of Jupyter use, so their "first among equals" status would be even stronger than VSCode's is for LSP and DAP. That hasn't stopped people from trying, though:

jupyter-vim:

A two-way integration between Vim and Jupyter. Develop code on a Jupyter notebook without leaving the terminal. Send lines from Vim to a jupyter qtconsole. Have a MATLAB-like "cell-mode".

emacs-jupyter:

An interface to communicate with Jupyter kernels in Emacs.

(VS Code) Python Interactive window:

Jupyter (formerly IPython Notebook) is an open-source project that lets you easily combine Markdown text and executable Python source code on one canvas called a notebook. Visual Studio Code supports working with Jupyter Notebooks natively, as well as through Python code files.

(PyCharm) Scientific mode

Scientific mode in PyCharm provides support for interactive scientific computing and data visualization.

I haven't really used in-editor Jupyter support since a decade ago—but from what I can tell, a few features have emerged as conventional:

1. automatically launch a backend kernel if one isn't running
2. run selected code
3. separate code into cells with a magic comment (typically # %%), and make it easy to run entire cells at a time
4. automatically open a console buffer in a split
5. debugger support

helix already has many discussions in flight for 4 and 5, but for these other features:

1. Helix already launches LSPs automatically, so this seems uncontroversial.
2. This generalizes the existing shell-specific behaviors in | and friends, so the feature itself doesn't seem out of place. Evaluating code tends to be a pretty common action in read-eval-print-loops, though, so you'd want an easily-accessible key binding (ideally involving Enter)—and it's not clear whether dedicating such valuable real estate to REPLs makes sense for what is primarily a text editor.
3. This is a little uglier. The way other tools do it does not fit helix's select-then-act paradigm—and even if helix made it easy to select within a # %% cell (e.g. with multi-character surround, or defining text objects for them and supporting syntax surround), it still smells bad to make users change code to help the editor. It's clever, though, and it's not obvious what to do instead—maybe helix could track which selections on the jumplist were created explicitly (i.e. with save_selection), and support those as a text object in match mode? That'd give you, say, m<Ctrl-s> for "select current cell", which doesn't seem that bad—as well as let users recover the # %% convention with %S# %%<Enter><Ctrl-s> if they wanted.

[benjaminrich]

@riwsky Thanks a lot for your detailed response! In that case, it does sounds like a good approach. I agree with your comment about keybindings. Ideally, you want to be able to "spam" the evaluate-current-and-move-to-next command with something like ctrl-enter or shift-enter (i.e., hold down one modifier and press one key, not multiple keystrokes).

[danjenson]

I wrote a temporary workaround using tmux, but it would be nice if we could query helix internal state to get the file type as an argument, i.e. :sh my_script.sh (file-type) where file-type resolves to the current file type. Currently, this hack checks open helix processes, selects the first, gets the file extension, and opens a REPL based on that (lots of things could go wrong here, I know). Anyway, here is basic version in case anyone wants to use it:

#!/bin/bash

# copy text from system clipboard
tmux set-buffer "$(xclip -o -sel clipboard)"

# select bottom pane
tmux select-pane -t "{down-of}" 2> /dev/null

# if no bottom pane, then start a REPL based on file type
if [ $? -ne 0 ]; then
  # TODO(danj): replace this with ft from helix
  ft=$(ps -u | rg -m 1 -o '\.(py|jl|R)')
  declare -A repl
  repl['.py']='ipython'
  repl['.jl']='julia'
  repl['.R']='R'
  if [ ${repl[$ft]+_} ]; then
    tmux split-window -v -p 35 ${repl[$ft]} # automaticaly focuses new pane
    sleep 1 # sleep so tmux list-panes updates with this new pane
  fi
fi

# if there is a bottom pane, check that it is a valid REPL before pasting
cmd=$(tmux list-panes -f '#{pane_at_bottom}' -F '#{pane_current_command}')
if [[ $cmd =~ python|R|julia ]]; then
  tmux paste-buffer -p
  tmux send-keys C-m C-m
  tmux select-pane -t "{up-of}"
fi

An alternative might be to allow keymaps that are specific to languages in languages.toml, so you could do something like C-t = ":sh my_script python" under the [python] language.

[sourproton]

Consistent, well designed and well integrated with the rest of helix - that is the best solution. It could be a "killer" feature that distinguishes helix from other modal editors.

I definitely agree with that!

I just want to share my thoughts on the "launching a REPL if one doesn't exist":

What I like best about my current neovim/slime setup is that I'm free to easily setup the REPL I want, how I want it, in a terminal window or a tmux pane. The plugin simply sends the code to the target REPL, with the convenience of setting magic comments like # %% to run the entire cell with a single shortcut.

I don't think that the Helix REPL feature should setup a REPL for the user if one isn't active. With different languages requiring different setups, this level of abstraction can get difficult to implement very quickly. What if there are different active REPLs? How to choose the environment and working directory of the REPL?

The user can easily launch their REPL in the language they want, in the environment they want. Sourcing a Python environment or setting a Julia project is easy and I think that a user who needs REPL integration knows how to do it.

I think the REPL integration in Helix should focus on implementing a convenient way to target an existing REPL and reliably send the selected text or the current cell to that REPL.

Some people use different magic comments for their cells, so I think this could be a configuration option, that defaults to "# %%".

[sourproton]

Another benefit of the REPL being outside of the editor, on top of being easier to setup the desired environment, is that the REPL can be on a ssh connection. I myself run the editor locally and pass the code to a remote REPL on a server.

[zapazite]

Just adding some cool reference material. Inline result is a way of coding that is so satisfactory and productive.

-https://git.sr.ht/~detegr/nvim-bqn This way of doing it feel the best for me.

-https://github.com/mrossinek/deuterium Deuterium is a vim-implementation of the popular hydrogen plugin for Atom.

-https://github.com/dccsillag/magma-nvim Magma is a NeoVim plugin for running code interactively with Jupyter.

Hopefully someone talented can implement something like that. That would be so nice to have for data science workflow.

[theophilusx]

I'm very impressed with Helix and I would also find the ability to have basic repl integration extremely useful. However, I would suggest this will take considerable design effort to get right. Often I am a great believer in just building something and then refining it based on usage and feedback. However, in other systems I've used which adopted this approach with respect to interfacing with repls and sub-processes generally, the result has tended to be overly complex with way too many options and points of confusion with respect to correct behaviour in various corner cases.

The biggest source of problems seems to centre around definitions of output, return values, error codes and handling of stdout and stderr. Things become more complicated because different languages have different semantics here - some languages always return a value (Clojure, FP langs) , some only when the code explicitly does so (Python, many OOP langs) and some rely on stdout/sterr and process return codes (shells, CLI tools). Sometimes you may want the value returned by the repl or sub-process and sometimes you might want whatever was sent to stdout and/or stderr and sometimes you only care about the exit code. Ability to specify this in a clean and concise m,anner will be critical to a solid implementation.

The other things to possibly think about would be session management. Sometimes, you want all the code evaluation to happen within the same session context and other times, you mgiht want the code executed in a clean context or you may just want an easy way to clear the context.

The challenge is in how to provide this functionality in a flexible manner which is not complicated and which doesn't detract from its big strength - clarity and simplicity without sacrificing functionality. When I compare my neovim and helix configuration where I have largely equivalent functionality, the benefits of Helix are obvious. The ability to interface with a repl needs to maintain that clarity and simplicity if possible.

[dgkf]

I've been using a variation on the solution proposed by @danjenson above. I think this is a nice solution for a few reasons, and am increasingly in favor of a solution that brings this feature through composition with other tools, rather than being built in to helix itself.

To simplify things further, the whole interface is a simple keymapping. I just pipe the selection into tmux (assuming you have an active session called helix-target (ie have run tmux new -A -s helix-target) running your REPL.

[keys.normal."\\"]
space = [ ":pipe-to tmux load-buffer - \\; paste-buffer -t helix-target:0" ]

This is a super simple solution, but seems to be quite feature complete for my needs. Of course it's a one-way street and you don't get any output back in helix, but for me that's not a critical feature and I'm fine sacrificing it for the sake of narrowing the scope of each tool individually.

[jerabaul29]

This is really awesome and looks very promising. As far as I am concerned (and I am likely not alone, many people with some data science / ml / scientific programming background around me are in the same case), having a way to use functionality 'a la jupyter(lab)' is what will make me take the step to move to Helix.

Wondering: even if a native / official solution is still in the pipelines, would it be possible to have a repo or folder on this repo somewhere, either Helix or wider community maintained, that:

• list different alternatives available as of today for getting an experience 'a la jupyter(lab)'

• for (as many as possible of) these, provide a dedicated detailed tutorial or points to one, so that anyone with minimal technical proficiency is able to set up such an environment and use it

?

The idea would be that, in the meantime waiting for 'the ultimate Helix native solution', there may be 'perfectly good enough' temporary solutions, and that making these easy to use, and nurturing an ecosystem of users, and helping users migrate to better solutions as these are developed, would be very helpful and help the user community grow.

[jonas-w]

@jerabaul29 I don't know if this fits your needs, but I use euporie to edit and run jupyter python notebooks in my console, it also has a vim mode (i wish it had a helix mode). But I agree it would be really awesome to have something like this in helix itself, with more familiar keybindings and faster iteration while writing code.

[sourproton]

I would like to share a setup that is perfectly working for me.

TLDR

1. Add this to your Helix config:

   [keys.normal.'\']
   space = [''':pipe-to tmux load-buffer - \; paste-buffer -dpr -t '{right-of}' \; send-keys -t '{right-of}' Enter''']
   '\' = [''':sh tmux send-keys Q/# Space %% Enter O# Escape Escape Nvg.a BSpace BSpace Escape Escape \\ Space uuQ''']
   # the above is the quivalent of putting, in .tmux.conf: send-keys Q/#\s%%\rO#\e\eNvg.a\b\b\e\e\\\suuQ

2. Have a tmux session with a REPL in the pane directly to the right of the pane containing Helix
3. Have the cursor within a cell, delimited with the # %% header, call the \\ bind to run the first cell you want to execute (not necessarily the first cell of the notebook). This records a macro. Then use q to run any other cell, provided the cursor is within its limits. n and N will also jump to the next and previous cells.
4. Alternatively, select the text you want to send to your REPL and use the \-Space bind to send it.

Here's the result, with ipython and Julia REPLs:

Explanation

The coding setup is based on tmux, with Helix containing the code and a REPL on the pane directly to the right. The target REPL can be anywhere though, even in another tmux session, you just need to modify the -t flags in the tmux commands.

So, inspired by @dgkf, I added

[keys.normal.'\']
space = [''':pipe-to tmux load-buffer - \; paste-buffer -dpr -t '{right-of}' \; send-keys -t '{right-of}' Enter''']

to my Helix config. Once we select the text we want to send to the REPL with Helix's select mode, ::pipe-to tmux load-buffer - loads a tmux buffer with that text, and paste-buffer -dpr -t '{right-of}' pastes it to the pane immediately to the right. Notice the flags -dpr: they are not mandatory, but that way it works the best for me, specially the -p flag that pastes the text in bracketed mode, which allows better handling of blank lines and indentation in ipython, for example. Because of the bracketed paste, :send-keys -t '{right-of}' Enter is needed to send a Enter keystroke to the REPL, to execute the pasted code, otherwise some REPLs (like ipython) will just paste the code and not run it.

That would be enough for many, but I find it counter-productive to have the need to select the text to be sent every time, specially because in my REPL workflow I work with cell headers in my scripts, like # %%. Here's an example code:

# %% cell 1
print("foo")

# %% cell 2
def bar(x):
    # compute y
    y = 2 * x

    return y

print(bar(1))

# %% cell 3
print("baz")

The ideal scenario for me is to be anywhere in the file and, with a keystroke, execute the whole cell containing the cursor and go to the next cell. I would like to have this bound to a key, but could only achieve this with a macro. So, for now, I need to re-type the macro every time I start a new Helix instance. Inspired by this reddit comment, I arrived at

Q/# %%<RET>O#<ESC>Nvg.a<BS><BS><ESC>\<SPACE>uuQ

It seems long, but the logic is straight forward:

0. Have the cursor anywhere in the cell
1. Q to start recording the macro
2. /# %%<RET> searches for the next cell header (notice the space between # and %%, respecting the cell header)
3. O#<ESC> puts a character at the end of the cell we and to execute. This modifies the file and, once the cursor is at the start of the cell, we can use this to navigate to the end of the cell
4. N goes to the start of the cell. It works because of 2.
5. v to enter select mode
6. g. to select until the last modification, that is, the # inserted at 3.
7. a<BS><BS><ESC> to remove the #
8. \<SPACE> is the bind that calls [":pipe-to tmux load-buffer - \\; paste-buffer -dpr -t '{right-of}'", ":sh tmux send-keys -t '{right-of}' Enter"]. Modify this command to send the text to other terminal multiplexers.
9. uu is needed because of the added and after removed # mark, that modified the file and polluted the change history. Undoing that clears the history and, luckily, puts the cursor at the start of the next cell
10. Q to stop recording the macro

Now just type q to execute the macro and jump to the next cell. To execute it, just type q again. The cursor doesn't need to be at the beginning of the cell, it can be anywhere inside it. Also, because of 2., n and N will jump no the next and previous cell, too.

I was 90% happy with this because I still needed to setup the macro every time and it doesn't work for the last cell.

To mitigate the last cell problem, a simple solution is to have

# %% end

at the end of the file and n or N to jump it.

Finally, to avoid needing to re-type the long macro every time, a solution is to add another bind to Helix, that will call the send-keys tmux cli function to type the macro by itself. The resulting Helix config is:

[keys.normal.'\']
space = [''':pipe-to tmux load-buffer - \; paste-buffer -dpr -t '{right-of}' \; send-keys -t '{right-of}' Enter''']
'\' = [''':sh tmux send-keys Q/# Space %% Enter O# Escape Escape Nvg.a BSpace BSpace Escape Escape \\ Space uuQ''']
# the above is the quivalent of putting, in .tmux.conf: send-keys Q/#\s%%\rO#\e\eNvg.a\b\b\e\e\\\suuQ

Double Escapes are needed, I think because of some kind of delay. With a single Escape, Helix doesn't quit insert mode before the next send-keys command.

So now I can \\ to record the macro automatically and then use q to run it again as many times as needed.

[kshitijsachan]

@sourproton Thanks for sharing! Is there some way to adapt that script to pipe to Jupyter lab? I use Helix on a remote machine and would like to be able to display interactive plots (so can't use tmux).

[aberges-grd]

I wanted to barge in real quick to say that the "cell division standard" in buffers could be set as the "Form Feed" character, inserted through Ctrl+L in e.g. Emacs, which is a standard of the ASCII code for page break. As such, it perfectly delimits pieces of code larger than "blocks" but shorter than "buffers", would not collide with any other character nor interfere with other editors that do not use it.

[astro-ray]

@jerabaul29 I don't know if this fits your needs, but I use euporie to edit and run jupyter python notebooks in my console, it also has a vim mode (i wish it had a helix mode). But I agree it would be really awesome to have something like this in helix itself, with more familiar keybindings and faster iteration while writing code.

Yes, even a light weight cell input-output cell that is not as feature rich as euporie would be a pretty neat addition to helix

[ShalokShalom]

Couldnt we just integrate euphorie? I know, the core team wants everything to be done in Rust, and still, I see that this could do the job: https://rustpython.github.io/

[pascalkuthe]

the issue isn't just that that is not written in rust which is generally a blocker but also that its specific to python. We will not include any language-specific tools or integration in core. That kind ow integration has to wait for plugins

[ShalokShalom]

Its not specific to Python.

There are dozens of languages supported. Also Rust

[astro-ray]

@pascalkuthe When considering the comment about euporie, that is very much python oriented, but notebooks are not specific to a single language. Python, R, Julia, all have a popularity of being used with a notebook, particularly in scientific community. I regularly use a combination of Python and Bash in jupyter.

Even Matlab, Wolfram has notebook options

[pascalkuthe]

I think including something like https://gitlab.com/Screwtapello/kakoune-repl-buffer can make sense because that interfaces with an arbitrary shell command instead of integrating with one specific REPL. Maype something fancier beyond that may make sense but I don't think we should integrate language specific notebooks/repls into core (and I still count euporie as that, yes it suuports some other languages but its definitly primarily oriented towards python and not universal at all).

At most we could have an arbitrary REPL mode that can have an API that plugins could provide backends to (with the default backend just being stdout/stdin)

[huwaireb]

Just wanted to provide a heads up,

A protocol similar to that of LSP or DAP for repls already exists, has existed for almost a decade now and that is nrepl: https://nrepl.org/nrepl/1.1/index.html. Albeit majorly used in clojure(script) community it is language and editor agnostic as mentioned in the following page: https://nrepl.org/nrepl/beyond_clojure.html, stating and I quote:

As mentioned earlier the nREPL protocol is language-agnostic and can be leveraged for many languages that have the ability to evaluate code at runtime.

with numerous examples being provided for different languages e.g Python, Lua, R and others.

The only odd thing I found in the protocol personally is the usage of bencode for the data transport, but that is merely the default provided option, and the reasoning was provided in the author here: https://stackoverflow.com/questions/59594424/why-bencode-has-been-used-for-transporting-clojure-code-to-nrepl-in-cider.

What I and others particularly love about nrepl is its simplicity, it doesn't attempt to overcomplicate things while being easy to implement, even bencode the default format as mentioned in the reasoning is easy to implement manually. Other reasons include its stability, as it has had a decade to be rock-solid.

I understand the argument that it lacks in language support compared to others like euporie leveraging Juptyer and various backends implemented in it, but those solutions are limited such in a way highlighted prior, they aren't a real "fit" because they don't work over a protocol which is a general requirement to an editor that doesn't want to build language-specific features to core, and doesn't have a plugin system yet. Something like nrepl should've had similar spotlight and adoption to LSP or DAP years ago, but unfortunately that wasn't the case. However it's never too late.

And doing it over shell isn't going to give nobody a decent repl experience.

[theophilusx]

From reading some of the comments in this thread, I fear there is a risk of conflating two different user cases/ideas here. One the one side, we have the 'notebook' concept with approaches like Jupyter, euporie and literate style programming environments, similar to Emacs' org-mode etc. On the other side, you have the repl driven development approach, which is similar, but not the same. The differences are somewhat subtle and not obvious from the outside. You really need to have used both approaches to apprciate these differences. For this reason, I'm not convinced that incorporating something like euporie is appropriate. THe nrepl approach is interesting, but I'm not sure it is correct either. There are other alternatives, such as socket based repls, which are also popular and which apparently address some of the percieved weaknesses with nrepl. I"m also not sure how well nrepl scales to other languages (I've only used it with Clojure/Clojurescript).

My gut feeling here is that what Helix should really do is take the time to examine and assess how other editors have addressed both the two use cases (i.e. notebookand repl driven development), work out what has been done well and what could be done better and then see if an initial 'common ground' could be found which could be built into Helix as a core feature and which would then be leveraged on to develop more specific functionality for each of the use cases (something which would likely be part of the evenual plugin ecosystem?).

BTW with respect to repl driven development, I think the neovim conjure plugin may be a good source of ideas. It provides a framework for implementing repls in a number of languages and is able to take advantage of existing interfaces e.g. for Clojure, it communicates over a nrepl connection. For repl driven development, how it itnerfaces with the repl is quite nice and a model which would liekly work well with Helix (due to also being based around a modal interface).

[dhirschfeld]

People here seem to be conflating jupyter with a notebook environment. jupyter itself is a protocol for frontends to talk to kernels (repls) for any of the many languages implemented.

• https://jupyter-protocol.readthedocs.io/en/latest/index.html

All Helix needs to do is start a kernel process for a specified language and then pass messages to it via a ZeroMQ socket. A user can then take whatever frontend they want, a notebook, a jupyterlab session, a terminal session and talk to the kernel to run code or inspect the state. Multiple frontends can be connected to a single kernel process at the same time so you can be inspecting and modifying kernel state in a kernel process from a terminal session and a JupyterLab session simultaneously.

Helix would just be another frontend to the kernel process, able to talk to it via the specified jupyter protocol.

[ShalokShalom]

I can see, that the optional/additional integration by Microsoft - dotnet interactive - can be an advantage here.

Yes, this would pull dotnet in - but you do not depend on it for normal jupyter support. It is only to add a couple of really cool features, and comes with very little code to pay for:

The main benefit is probably the sharing of variables between languages.

So Python can bind the result of a calculation to a name, and F# can consume it. How to add a language to that support category? There is a really simple guide for it available.

Please note, that this is ultimately simply another implementation of the Jupyter protocol. The only difference is, that the dotnet core can spawn multiple sub-kernels.

That means, a dotnet core runs a python and a R core, who then can communicate. Very useful for data sharing, data analysis and data science.

Still compatible with regular old standards.

There is also a neat alternative to .ipynb called .dib, but I forgot what its purpose was, and I am currently struggling to find proper doc to it. But I remind it was a worthwhile consideration as well. https://github.com/dotnet/interactive/issues/3378

Much fun altogether

[sg1fan]

I think the easiest and most direct way to support this feature would be to implement a Jupyter client. By easiest, I mean all of the possible errors cases and rough edges of running arbitrary code have been smoothed out already. By direct, I mean there is no need to invent some bespoke way of running code and managing session state. Moreover, the ecosystem is massive as indicated by the over 100 kernels already implemented and the APIs provided by Jupyter are well-specified, stable and battle-tested.

In addition to supporting a basic REPL, this would would also naturally support the core pieces for full blow literate programming plugins. This isn't an "experimental" concept either. There are example implementations floating around, such as the Neovim plugin molten which runs can run code in any document and display the output to the user. No browser, no JSON, just fenced code blocks in Markdown and Neovim. And actually, this is document format agnostic as well. The markdown is not necessary, but just a common default.

Moreover, Jupyter has already specified a file format (i.e., notebooks) for which we can save code outputs, so the code outputs can be saved, exported, and then re-imported later just as is done inside the browser-based frontend, except this is all achieved outside of any browser. Not that I think this should be supported in the core, but rather just speaking to the feature-completeness of Jupyter.

[ShalokShalom]

Also notice, that Microsoft has implemented an improved notebook file format.

I guess it might be a bit out of scope, but it may prove useful.

https://github.com/dotnet/interactive/blob/main/docs/FAQ.md#whats-the-difference-between-a-dib-file-and-an-ipynb-file

[jerabaul29]

@sourproton many thanks, your solution works very well and is my daily driver :) .

Only one thing that is problematic: sometimes, I have a "multiple line selection", and I press q without collapsing the selection first with ,. This creates havoc, as parts of the same cell get copied several times, etc.

I tried to modify the config entries you provide:

[keys.normal.'\']
space = [''':pipe-to tmux load-buffer - \; paste-buffer -dpr -t '{right-of}' \; send-keys -t '{right-of}' Enter''']
'\' = [''':sh tmux send-keys Q/# Space %% Enter O# Escape Escape Nvg.a BSpace BSpace Escape Escape \\ Space uuQ''']

To a number of variations adding either , or keep_primary_selection to the macro definition but I am getting a number of config parsing error; any idea / recommendation how something to collapse to the main selection could be added at the beginning of your macro? :)

[peter-fm]

I found the above solutions work for the most part but had trouble with some valid python code causing issues (mainly classes and extra lines within functions). So I wrote a little rust tool to slot into the keyboard shortcut call that fixes most of the indentation issues. If anyone is interested you can find it here: pypaste.