A few different things have bugged me about the async abstraction in Plenary and
I've decided that it's sufficiently beneficial for us to roll our own.
The new async lib deviates from the behavior / design of Plenary's
implementation in a few different ways:
No implicit awaits.
Plenary's async lib conditionally uses implicit awaits. Meaning that, in
certain circumstances, calling an async function from an async context
will cause it to run synchronously.
More specifically, async.wrap-ed functions are blocking (within the
parent coroutine) when called without a callback. async.void functions
are always non-blocking, but can't be awaited.
In our implementation all async functions are non-blocking by default, and
can be made synchronous within the current coroutine with async.await().
Top-level awaits.
Async functions can be awaited from the main thread.
Better errors.
Whenever an error is raised in a Plenary async function, the resulting
stack trace is missing a lot of context. Important context from parent
coroutines + main thread is lost.
This is solved by using an error message handler, and re-raising errors in
yielding parent coroutines as well as the main thread such that we get a
far more detailed stack trace.
Smarter scheduler.
The async.scheduler() is frequently used in async code in order to yield
until the Neovim API is available. Plenary's scheduler always calls
vim.schedule() and yields until the callback is invoked, even when it's
unnecessary.
The new scheduler will only yield when the editor is actually constrained
by locks.
Now the only things left that we were using from Plenary were the jobs and the
logger. These are both things that I have long considered reimplementing for a
few different reasons. Let's start with the job abstraction.
Standard output in Plenary's jobs is streamed and processed line-by-line. This
certainly can be useful. In this plugin however, 99% of the time, we just want
to consume all the output from a job and we don't benefit from streaming.
The new job implementation has streaming disabled by default (we still use it
for file-history). Writing the job class alongside a logger also made it easy to
integrate better logging, which is incredibly useful for debugging.
There's now a way to configure custom fail conditions for different jobs, as
well as a mechanism for automatically retrying failed jobs. A multi-job
abstraction was included that enables simpler handling of complex fail
conditions for groups of jobs that rely on each other.
On to the logger. Plenary's file logger (which was what we were using) performs
a lot of I/O operations. Every time one of the log functions is called, the
logger 1. opens a file 2. appends data to the file, and 3. closes the file. This
makes the log functions expensive operations that causes a lot of I/O overhead.
This is easily solved by using a message buffer, and only writing to file every
few seconds when it's necessary. So that's what we do now.
A few different things have bugged me about the async abstraction in Plenary and I've decided that it's sufficiently beneficial for us to roll our own.
The new async lib deviates from the behavior / design of Plenary's implementation in a few different ways:
No implicit awaits.
Plenary's async lib conditionally uses implicit awaits. Meaning that, in certain circumstances, calling an async function from an async context will cause it to run synchronously.
More specifically,
async.wrap-ed functions are blocking (within the parent coroutine) when called without a callback.async.voidfunctions are always non-blocking, but can't be awaited.async.await().async.scheduler()is frequently used in async code in order to yield until the Neovim API is available. Plenary's scheduler always callsvim.schedule()and yields until the callback is invoked, even when it's unnecessary.Now the only things left that we were using from Plenary were the jobs and the logger. These are both things that I have long considered reimplementing for a few different reasons. Let's start with the job abstraction.
Standard output in Plenary's jobs is streamed and processed line-by-line. This certainly can be useful. In this plugin however, 99% of the time, we just want to consume all the output from a job and we don't benefit from streaming.
The new job implementation has streaming disabled by default (we still use it for file-history). Writing the job class alongside a logger also made it easy to integrate better logging, which is incredibly useful for debugging.
There's now a way to configure custom fail conditions for different jobs, as well as a mechanism for automatically retrying failed jobs. A multi-job abstraction was included that enables simpler handling of complex fail conditions for groups of jobs that rely on each other.
On to the logger. Plenary's file logger (which was what we were using) performs a lot of I/O operations. Every time one of the log functions is called, the logger 1. opens a file 2. appends data to the file, and 3. closes the file. This makes the log functions expensive operations that causes a lot of I/O overhead.
This is easily solved by using a message buffer, and only writing to file every few seconds when it's necessary. So that's what we do now.
TL;DR plenary.nvim is no longer a dependency.