Vim Mode

[curran]

I'd be interested in a "Vim Mode" plugin for CM6.

I wonder if the old Vim mode could be easily wrapped? My guess is likely not, it would need to be re-implemented using the new CM6 APIs.

I'd be interested in working on such a project, and I thought I'd create an issue here in case others are also interested in this. I do realize this would not be part of "core", it would be a separate package.

[marijnh]

Given that that existing vim code is huge (~5500 lines), I think trying to directly port it is the most reasonable approach. Wrapping it might also work, but introduce all kinds of inefficiencies.

The main tricky part would probably finding replacements for some of the weird core features that were motivated by the vim mode—fat cursors will work differently, since we can no longer override the style of the cursor div (it's a native cursor now), and enabling a keymap can definitely doesn't have side effects in CM6. But the new extension system is pretty powerful, so a reasonable replacement shoudn't be hard to find.

[curran]

Interesting that the vim mode motivated some core ceatures of CM5! Sounds like a vim mode for CM6 would be a good test case for the extension system.

My gut feeling is that it could be done with a lot fewer than ~5500 lines.

Possibly an interesting resource https://github.com/t9md/atom-vim-mode-plus . This contains ~13,000 lines of JS.

My gut feeling is probably wrong. Looks like a complex challenge.

[marijnh]

My gut feeling is probably wrong. Looks like a complex challenge.

There's definitely a lot in there. But almost all of it has associated tests, which will probably be a great help in making sure the port doesn't regress.

[nicholasbulka]

I'd be interested in trying to help accomplish this if anyone is still interested.

[taufik-nurrohman]

Fat cursors will work differently, since we can no longer override the style of the cursor div (it's a native cursor now).

Force to select one character on focus.

[nicholasbulka]

Yes, could maybe use CSS ::before and ::after pseudoelements to control the visuals

[curran]

@nicholasbulka You mentioned:

I'd be interested in trying to help accomplish this if anyone is still interested.

I'm still interested!

One way to help would be to investigate the approach that would need to be taken. A starting point at a custom plugin that just does one thing, for example, makes "h" behave like left arrow and "l" behave like right arrow. Also a starting point for tests would be amazing.

I might also be able to put some energy into this at some point, but right now I'm not sure where to even begin.

I wonder if there are any similar plugins that we could look to as starters...

[nicholasbulka]

I might suggest copying the emacs bindings extension and gutting it for a keybinding shell, then going through vimtutor and adding in bits of the functionality progressing through the tutorial. One could use a combination of the old vim code in CM5 along with the emacs implementation for CM6 and arrive at basic vim functionality pretty quickly, I would imagine.

[marijnh]

Which emacs bindings extension do you mean?

[benhormann]

I made a trivial plugin using default commands, but it is not a good starting point.

I propose a map engine and many motion/operator functions. Event handlers will translate input to keys (notation), queue them and start the engine. The engine will dequeue keys, map them to functions, pass state through those functions, then dispatch the result to the view. A simple map replaces a key in the queue. A macro injects keys into the queue.

My intention is to avoid some long standing issues in the previous vim emulation: Insert mode mapping not possible, macro/repeat not recorded as a string, default bindings getting in the way, functions not reusable in extensions, and so on... The previous code will make a good reference and, with care, parts can be ported.

I started working on this, here are some snags. Recursion - needs to be (partly) async so that it doesn't block the UI or blow the stack. IME - breaks the model of absorbing all input events into the queue (most problematic for Insert mode?).

Once the basics of the engine (or a test harness) is workable, it should be easy to contribute functions for motions/operators. A panel for the status line will be also required if anyone wants to have a go at that.

[nicholasbulka]

@marijnh sorry I should have said we could begin by creating a standalone extension, using the emacs keymap from CM6 as a guide:

import {emacsStyleKeymap} from '@codemirror/commands';

[curran]

Great idea! Let's do it!

creating a standalone extension

What does a standalone extension look like? Are there any examples we could reference as a project template?

[benhormann]

standalone extension

It's not feasible^* to load extensions outside of the script/bundle that imports @codemirror/*. ^* extremely impractical, if not impossible.

Forget emacsStyleKeymap, it's just a list of bindings. The keymap Facet is not practical for modal editing, use EditorView.domEventHandlers instead.

I'm working on proof-of-concept code. Full of hacks but almost useful, really wants some kind of yank and paste. In theory we could use CM6 for editing the Vim extension. (on-demand extension reloading!)

Project setup comes with many choices: test framework? linting? CI? guidelines? owner / maintainer? There are even more design choices to consider... State is the biggest one: StateField(s)? Compartment(s)? Immutable.js / Immer? Or stay with imperative mutation?

Do we discuss details here, on the forum, or elsewhere?

[nicholasbulka]

I’ve tried using the state library redux (and immer) with CM6 and the non serializable, read only types from CM6 proved to make it difficult, having to convert to JSON and back. What is lacking in the current dispatch functionality within CM6?

Perhaps the modes could be approached in a systematic way by passing annotations like the example given of syncing two editor views. Then we could stack the Insert mode view, Visual mode view, and Normal mode with clever CSS. Separate the concerns and share the state as a field within the editor.

I’d be down to talk it out, that seems more actionable. Feel free to reach out.

[benhormann]

It's just that Vim has so much state. Do we opt for many simple fields, or a few structured fields? If structured, that's where a library could help.

EditorView.editorAttributes can add a class to the editor, which would make it easy to do mode-specific CSS:

Example code

```js import {EditorView} from "@codemirror/view" import {Compartment, EditorState} from "@codemirror/state" let Mode = { compartment: new Compartment, of: mode => EditorView.editorAttributes.of({ class: `vim-mode-${mode}`, }), update: mode => view => view.dispatch({ effects: Mode.compartment.reconfigure(Mode.of(mode)), }), } let view = new EditorView({ state: EditorState.create({ doc: 'Vim!', extensions: [ EditorView.theme({ "&.vim-mode-normal": { color: "blue" }, "&.vim-mode-visual": { color: "green" }, "&.vim-mode-insert": { color: "red" }, }), Mode.compartment.of(Mode.of('normal')), ], }), parent: document.body, }) setTimeout(Mode.update('visual'), 1.2e3, view) setTimeout(Mode.update('insert'), 2.4e3, view) setTimeout(Mode.update('normal'), 3.6e3, view) ```

[benhormann]

One feature of CM5 that isn't really catered for is the swapDoc event. The old vim.js isn't using it, but it I do; it's handy for stashing marks.

CM6 has EditorView config.dispatch (and EditorView.setState).

I'm wondering how to avoid things like:

let view = new EditorView({
  config: new EditorState({ extensions: Vim() }),
  dispatch: (...args) => Vim.dispatch(view, ...args),
})
/* Or */
import {VimView as EditorView} from 'codemirror-vim'

We could try using PluginValue destroy to help join the dots. (@marijnh shouldn't it have a view parameter?) CM5 can distinguish "different documents" (i.e. not simply reloading a file) with cm.swapDoc vs. doc.setText.

Theoretically, setState can be used but with the identical doc (or same string). Should we attempt to map marks to the new state, or make the vendor tell us to do it? StateEffect.reconfigure should probably be used instead, but who knows what vendors will do.

Hang on... there's no setBookmark equivalent! If we are going to implement bookmarks, it should be a separate package. And it should have line-marks, since we'll probably want that too (vim.js calls cm.getLineHandle).

[marijnh]

In CM6 the way to swap out the 'document' is to swap out the entire state. This will, indeed, destroy and recreate view plugins. (View plugins usually save the view that was passed to the constructor as a property, so I haven't run into a situation where it's useful for destroy to take that as an argument.)

[benhormann]

Makes sense, I was looking at editorview.ts#L630 - but it's actually extension.ts#L242. Hopefully we can access state at that point. Sorry, off topic... I was more wondering about annotating the difference between switching to a different file and reloading a file. (way out of scope, even for Vim Mode...)

[curran]

I suppose a first working prototype could just map hjkl to the functionality of arrow keys. How might one do just that?

[curran]

I made a trivial plugin using default commands, but it is not a good starting point. I'm working on proof-of-concept code.

@benhormann Is it open source?

[benhormann]

The PoC a bit fiendish, featuring (simplified): motions, operators, counts, Visual, status-line, 1.5 registers, single key prefixes (g, z, \). Crammed into ~250 lines of JS. The most reusable part is toKey(event), about 30 lines worth. (I've also got a procedure for splitting a string into keys somewhere...) I need to start making modules, do state properly, etc. For now it's my Ship of Theseus.

A tiny hjkl example:

import {keymap, EditorView} from '@codemirror/view'
import {EditorState, StateEffectType, StateField} from '@codemirror/state'
import * as cmds from '@codemirror/commands'

let modeEq = ({state}, m) => /* TODO: StateField */ Math.random() < 0.5
let vimify = cmd => view => modeEq(view, 'N') && !void cmds[`cursor${cmd}`](view)

let view = self.view = new EditorView({
  state: EditorState.create({
    doc: 'Hello,\n  Vim!',
    extensions: keymap.of([
      { key: 'h', run: vimify('CharLeft') },
      { key: 'j', run: vimify('LineDown') },
      { key: 'k', run: vimify('LineUp') },
      { key: 'l', run: vimify('CharRight') },
    ]),
  }),
  parent: document.body,
})

^* !void is to return true, otherwise it inserts chars at the edges.

Exercise: Try adding a StateField for mode. Then try adding { key: 'g g', run: vimify('DocStart') },... You could add a Compartment and reconfigure keymap, but it's easier to use EditorView.domEventHandlers({ keydown }), (sig: keydown(event, view): boolean).

Once you get to implementing 5h, it'll be better to use view.moveByChar instead of cursorCharLeft. You'll be dispatching selections in no time!

The Examples are full of handy snippets, e.g. Configuration, and the Reference is essential. Read the Migration Guide if you know CM5 vim.js, otherwise read the System Guide.

[curran]

Amazing! Thank you so much. It is very exciting to see this germinate.

This Vim mode for CodeMirror 6 initiative is something I have very long term interest in, for use in VizHub. Therefore I took the plunge and set up a repository for this work:

https://github.com/vizhub-open-core/codemirror-vim-mode

I can't promise that much time, but I may be able to invest spurts of work here and there, and I definitely will keep an ear out for PRs and issues. I set up a Kanban board as well.

Here's hoping the repo sticks and we can get some traction!

[benhormann]

I'd love to work on this full time (dunno how to get sponsorship), so I'll probably make my own project (open source, of course). I haven't done that yet because I want to start with something that has shape? Something people can contribute to in a more straightforward way.

Or I could just push to yours... You should add GLP3+ to match codemirror.next. (It's a WordPress thing? Completely undermined by the MIT License anyway.)

---

I thought w would take 5 mins... 1 hour later:

  let left = cmds[`${prefix}GroupLeft`], right = cmds[`${prefix}GroupRight`], {head} = view.state.selection.main
  right(view) && left(view) && view.state.selection.main.head <= head && right(view) && right(view) && view.state.selection.main.head !== view.state.doc.length && left(view)

Eww! (and it needs to, maybe, go one char left in visual mode?)

[marijnh]

You should add GLP3+ to match codemirror.next.

(No, really, you don't. A dual license means you can pick, so I recommend everybody pick the MIT license.)

[benhormann]

I though it was a request from WordPress, so if they want to use Vim Mode shouldn't it be the same? Otherwise I agree, there is no need. It doesn't make sense to have copy-left and (compatible) permissive licenses.

On that subject, shouldn't it be (MIT OR GPL-3.0-or-later) in package.json? (Hmm, the npm docs should add -or-later too)

[nicholasbulka]

I am also excited to see this take root. Please let me know how I can help. I bet a brainstorming session would help everyone get on the same page architecturally. As for the repo, I don't think putting it under vizhub is necessarily helpful for anyone who needs the plugin for other use cases.

[curran]

I don't think putting it under vizhub is necessarily helpful for anyone who needs the plugin for other use cases.

It's just under that GitHub org. I could put it under my personal account if that would feel more comfortable. It's a standalone package regardless, and there's no mention of VizHub aside from the GitHub org (it won't be a namespaced package, for example). I'll consider moving it from https://github.com/vizhub-open-core/codemirror-vim-mode to https://github.com/curran/codemirror-vim-mode . Or maybe it could live at https://github.com/codemirror/codemirror-vim-mode . Happy to move it around once something is working.

@benhormann I was able to get your snippet to run! I also refactored it to simplify and split it into the "plugin" and the "example". Here's what that looks like:

import { keymap } from '@codemirror/view';
import {
  cursorCharLeft,
  cursorLineDown,
  cursorLineUp,
  cursorCharRight,
} from '@codemirror/commands';

export const vimMode = () =>
  keymap.of([
    { key: 'h', run: cursorCharLeft },
    { key: 'j', run: cursorLineDown },
    { key: 'k', run: cursorLineUp },
    { key: 'l', run: cursorCharRight },
  ]);

import { EditorState, EditorView, basicSetup } from '@codemirror/basic-setup';
import { javascript } from '@codemirror/lang-javascript';
import { vimMode } from './index';

const editor = new EditorView({
  state: EditorState.create({
    extensions: [basicSetup, javascript(), vimMode()],
  }),
  parent: document.body,
});

See complete source files.

(Note - I'm not a fan of import * as cmds from '@codemirror/commands' as it does not play well with tree shaking. My preference would be explicit named imports.)

Next up is setting up some sort of unit test infrastructure, then getting the mode to switch between "Normal mode" and "insert mode".

[benhormann]

I was using that because there are a lot of commands, and my version uses plenty of them. I'm using the key in two parts cursor to move, select in visual mode and operations. Using @codemirror/commands is a temporary solution.

I'd try out Marijn's setup for building: @codemirror/buildhelper (comes with Mocha / Selenium). Maybe use Standard for linting, though I prefer ES5 dangling commas.

BTW, you can paste my snippet into CodePen and click the "replace imports" autofix (use Skypack).

---

Worth setting up a separate org? Could potentially have more that one package?

[curran]

Nice.

Indeed, I just realized there is already a well established "project template" structure set up with all the CodeMirror packages. It would probably be best to use one of those as a starting point for the build and test setup. I sort of rolled my own without fully reviewing what's there first.

This, for example, looks like a good one to start from https://github.com/codemirror/basic-setup

[benhormann]

Another useful task could be to start breaking apart the old Vim Mode. We can categorize its parts into features we want to keep. (preferably just the unique / tricky ones)

Example:

[map types]
- motions
- motionOperators
- operators
- actions
- search
(some operators and actions have `isEdit` param)

[commands]
- colorscheme
- map, imap, etc.
- write?
- set, setlocal, setglobal
- substitute
- nohlsearch
- delmarks
- registers
- global

[actions]
- jumpListWalk
- scroll
- scrollToCursor
- replayMacro
- toggleOverwrite
- reselectLastSelection
- repeatLastEdit

... TBC ...

Wiki?

[curran]

Sounds good! I made that list into an issue.

I'm working on re-doing the project foundation based on codemirror/autocomplete as a project template.

I'm struggling to figure out how to set up an interactive demo that works in development. Are there any existing CM6 packages that have an interactive demo inside their repositories? If not, how is interactive testing done in development?

[marijnh]

My development setup uses esmoduleserve plus a @codemirror/buildhelper watch process. Other ES module-capable development tools should also work fine.

[curran]

Thanks for the tip! I found occurrences of esmoduleserve and it looks like the setup in the codemirror.next root package dev script could be the right path forward.

I suppose I am in uncharted territory, in a way, because so far the interactive tests for all CodeMirror modules (e.g. codemirror/autocomplete) live inside the codemirror.next package, and not in the packages that define them.

[thien-do]

Hi, sorry this is not a technical comment, but I just wanted to fund for the development for this specific issue. Anyone know a place to get started on the funding? It's personal so it's not much, really, but I hope it helps a little

[curran]

Nice! Well, @benhormann said

I'd love to work on this full time (dunno how to get sponsorship)

so there may be a good match. I'm not in a position to accept funding for this work myself. I'd be thrilled if @benhormann or others could take the initiative forward.

[thien-do]

@benhormann do you have a ko-fi or patreon? Or just contact me :D

Thanks everyone. And sorry for intercepting the technical discussion.

[benhormann]

@thien-do Thanks for showing interest. At this stage you could consider donating to Marijn, who has done all the hard work and likely to have ongoing involvement.

[marijnh]

I have no intent to work on the vim bindings in the near future, so I'm definitely not the correct person to sponsor for that.

[benhormann]

I meant that without CodeMirror, we wouldn't even be here. No doubt there will be plenty of questions and maybe weird bugs uncovered.

[nicholasbulka]

@benhormann are you still working on this? I have a POC editor able to switch between visual/normal/insert modes, but it uses keymaps and like you said earlier, they are fairly limited (for example, I don't believe they don't support sequences, like "dd"). How are you / do you propose we structure DOM event handlers to systematize sequences like "d4d"?

[nicholasbulka]

It seems like a tree could be useful here to track all "well-formed-formulas". We could abstract out parts of the syntax, I'm going to do some research into the grammatical types there.

[nicholasbulka]

This looks like a useful resource: https://rawgit.com/darcyparker/1886716/raw/eab57dfe784f016085251771d65a75a471ca22d4/vimModeStateDiagram.svg

[marijnh]

There is support for multi-stroke key bindings in CM key maps, but I think for VIM bindings you'll want your own key handling (the CM5 bindings definitely work that way) to easily support vim-style mapping and other idiosyncracies.

Again, I think trying to port the old code is a more promising approach than starting from scratch here. There's a lot of knowledge encoded in that 5750-line file, and adjusting the CM5-specific parts will be less work than redoing all that from scratch.

[benhormann]

@nicholasbulka I use one keydown handler. It converts the event properties into a Vim notation string. Actually handling the key is the tricky part...

That state diagram (nice find btw) mostly deals with mode. I had state for: count, mode, motion, operator and previous key. (A second count is needed for 2d3d). I used a map of maps (e.g. vim.keymap[vim.mode][key]), where key may have had previous key(s) prepended. There are drawbacks, you'd still need to search for partial bindings e.g. j pressed with jk mapped.

Digits are typically handled specially (e.g. zero is mentioned in :h map.txt). In the case of d4d the digit can be isolated to increment count. Maps like g0 or g8 should be allowed, so if you go down that route lookup such bindings first? I had coded dd as two parts, an operator and a motion, meaning d4d was handled the same as d4w or any other motion. (vim.js is similar, having no default mapping for cc, dd, or yy).

---

As for building a tree, you'd want to introduce cycles for digits. Zero is going to be a nuisance, but not insurmountable (otherwise you could do t['d'][0] = t['d'][1] = t['d'][2] ... = t['d']). Without cycles the tree would be infinite.

I'm not sure how you'd incorporate user mappings either, probably you'd rebuild the tree for each map command? Partial / overlapping mappings would be interesting (I keep bringing this up, it's an issue in vim.js).

[benhormann]

@marijnh I suspect we'd be rewriting huge portions and many small changes, touching thousands of lines. We'd also have to write replacements for APIs that don't exist. To top it off, we'd be porting all the old bugs.

Now is as good an opportunity to redesign as any. I'd rather have a small set of features that is easier to work on, than a port of the previous code. Note: I'll still help with porting if that's what people want, else I'll use the previous code as a guide for what's in scope.

I suppose it depends on how to port it... If we analyse each old line and deduce how to incorporate its purpose into the new code, that could work. Some lines/functions won't need much change, asides from splitting one huge closure into modules.

[thien-do]

This may be a helpful idea: the vim binding for monaco editor is actually CM5's vim binding, with an adapter: https://github.com/brijeshb42/monaco-vim/blob/master/src/cm_adapter.js

[squidbidness]

FWIW, Neovim (a great fork of vim) is designed to be an embeddable editor. I don't know the technical details of how its API works for that. I'm also not sure if CodeMirror depending on an outside, native program or library such as that would work for all of CodeMirror's intended environments and use cases. But if it's workable, writing an adapter layer to embed Neovim in CodeMirror would, I expect, be simpler than re-creating vim's functionality, and would have the advantage of fully supporting ALL vim features, allowing users to use their installed plugins, etc.

I mention this because with some searching around this conversation and this repo, I'm not sure I've seen it mentioned here yet.

I'm personally interested because the notes app that I use, Obsidian.md, currently relies on CodeMirror 5, from which it gets its vim emulation. There's active discussion in the Obsidian forums about what to do for a vim mode when CodeMirror 6 is ready.

[benhormann]

@squidbidness Firenvim already does that, depending on your definition of embed. There are also some node packages worth checking out to discover how to work with the API. WebSockets can connect to localhost, so users would just need to start the localhost server themselves and authenticate. I'd call that nearly workable?

WebAssembly is a more general solution, see vim.wasm or maybe libvim. (Or try BusyBox vi in JSLinux). Every origin has its own storage allocation, resulting in a separate vimrc for each website. Could add helpers to fetch plugins from GitHub or sync configs, but some sites will block cross-origin requests. An import/export helper would be simpler.

A potentially major problem with integrating Vim/Neovim would be getting it to play nice with other extensions, especially collab. Would it have to disable closebrackets? Are there other extensions that change the document? Do apps like Obsidian format as you edit? Wiring it up and keeping the document in sync could be challenging.

Vendor adoption is also a concern. Plain JS is a better default offering. Some vendors might provide both vim.js and wasm/remote Vim/Neovim options. A wasm blob shouldn't be a technical barrier, but you never know. Vim's license could be a factor. I didn't find any mention of converting libnvim to wasm, it'd be worth a try. Can libnvim function well enough without Lua?

Sounds like two or three different projects to me.

---

In an ideal world there'd be a libvim.js that the myriad of editors contribute to and benefit from. CodeMirror was filling this role to some extent, with at least Ace and Monaco using its vim code. A semi-agnostic rewrite could work out well in the long run.

[thien-do]

WebSockets can connect to localhost, so users would just need to start the localhost server themselves and authenticate. I'd call that nearly workable?

I believe there are some projects (hm maybe the promql) that uses this approach to take advantage of some Language Server (so basically web <-> server/host). However, I don't think it... effective, at least comparing to the current implementation :(

In an ideal world there'd be a libvim.js that the myriad of editors contribute to and benefit from. CodeMirror was filling this role to some extent, with at least Ace and Monaco using its vim code. A semi-agnostic rewrite could work out well in the long run.

Yes. This makes very much sense. Sad truth is that the "adapter" of Monaco is good enough that my company is considering moving to Monaco :(

WebAssembly is a more general solution, see vim.wasm or maybe libvim. (Or try BusyBox vi in JSLinux).

The wasm one seems to be great. Is it possible to write some sort of "adapter" so we can use it in CM6? Or is it too overkill?

[benhormann]

@thien-do Anything is possible, but as per my previous comment:

Wiring it up and keeping the document in sync could be challenging.

It's more than just sending keyboard events to Vim and translating Vim's messages into document updates. It requires reimplementing Vim's UI with CM6 panels, decorations and themes.

Line wrapping might be fine for ASCII, but Unicode could easily stop virtual lines from matching up. Vim doesn't support bidi the same as CM6 does, so that wouldn't match either.

CodeMirror doesn't have a direct analogue of Vim's windows (splits), some sites might not handle extra editor views being added on the fly. Even the concept of buffers doesn't translate well. Sites like Glitch have files, but most others have a singular unnamed document, i.e. the common denominator here is rather low. How do we tell them to switch buffer if they aren't listening? On the plus side, you could use commands like :find on sites/apps that are willing to integrate, e.g. CodePen could be keen to enhance their Project Editor.

Don't get me wrong, it'd be awesome to have Vim integrated. On the other hand, CM5's emulation is enough most of the time. The API allows me to do what I want, but there are issues that can't be fixed easily.

---

It really depends on priorities and expectations... Is Firenvim good enough for people who want more? What about mobile devices? What will vendors be willing to integrate? Do we need a JS implementation anyway, as the default / fallback? How complete does the emulation have to be? Are <C-n>, <C-t> and <C-w> too sorely missed?