Proper `if` support in kakscript

[andreyorst]

Feature

This is my solution from the discourse post Branching on a boolean option – without calling the shell. I've been testing it for a while and I'm convinced that this might be a good fit for Kakoune.

So I would like to propose very slim implementation for if statement in kakscript without need of forking shell:

define-command true -params 2 %{ eval %arg{1} }
define-command false -params 2 %{ eval %arg{2} }
define-command if -params 3 %{ eval -verbatim %arg{1} %arg{2} %arg{3} }

This is a lambda calculus approach, that seems to work really well. What exactly happens here is as follows:

true and false are commands, both of which take two arguments. true evaluates first argument and false evaluates second argument. So if you do true %{echo 1} %{echo 2} it will print 1, and if you use false it will print 2.

if is just another command, that accepts three arguments, first of which must evaluate to either true or false. Then it evals true or false command with the rest arguments passed to if.

So for example, doing this:

if %sh{ [ "a" = "b" ] && echo true || echo false } %{
    echo -debug "that's true"
} %{
    echo -debug "that's false"
}

Prints that's false in the *debug* buffer.

Usecase

Main usecace would be to provide a less destructive and more finite control flow to configurations and script writers. Right now the common approach is to use try that tests some option, and uses catch as a false branch. This is not a true if/else construct, because if any error happens anywhere inside try after the test, the whole catch branch gets executed, which would not happen in real if. It can be worked around by using nested try blocks, but it gets cumbersome pretty quickly.

Alternative approach is usually to use sh%{} block for the logic, but this has a problem of escaping Kakoune commands, sometimes exporting all needed variables, and echoing commands back either through pipe or directly, which makes simple if statement a pretty big boilerplate.

The solution I'm proposing doesn't have this problems, and would only evaluate one branch no matter what unlike try/catch.

Plugin authors will be able to create much simpler logic around boolean options they provide for altering plugin behaviors, and user configurations may have easier tests for some tools to be present like here:

when %sh{ [ -n "$(command -v rg)" ] && echo true || echo false } %{
    set-option global grepcmd 'rg -L --hidden --with-filename --column'
}

The only downside is that this solution works only and only when the first argument evaluates to true or false. This is not really an issue, but Kakoune also accepts yes and no, as valid Boolean values, so these also may need to be turned into commands.

Another small problem is that if expects exactly two branches, but this is easy enough to workaround, either by using nop as one of the branches, or by defining a when and unless commands:

define-command when -params 2 %{ eval -verbatim %arg{1} %arg{2} nop }
define-command unless -params 2 %{ eval -verbatim %arg{1} nop %arg{2} }

Or maybe there's a way to allow if, true, and false to accept as many args as needed to allow chaining. I've not explored this idea, as I needed two branches at most the entire time I've used my previous shell based implementation, that supported chaining.

I've ported my config to use these commands, and it seem to work well even for somewhat complex setups I do. Previously I've used shell escaping, and now my kakoune starts a little bit faster, as it doesn't have to do forking.

I understand that #2777 was closed because making kakscript into a real scripting language is not a goal for Kakoune, but having if in the language is a real quality of life improvement, and the suggested change is very minimalist, that it feels right to me. Of course one might argue, that since the needed change is so small, it should be kept as user config trickery, but I think this would prevent a lot of cool stuff appearing in the future, as some people are easily turned off writing plugins in pure kakscript, because of the lack of any conditionals.

[occivink]

I'm in favor. I know that the official stance is that the shell should be used for such logic, but in practice scripts just end up using try/catch blocks, because getting the escaping right is difficult and error-prone. Even some builtin scripts use try/catch as a poor man's if resulting in potential errors in the first block being interpreted as the 'false' case. Generating big blocks of kakoune script within a %sh block is also less ergonomic, since the highlighting of kakoune code is lost.

I would even go further and say that we could have support for basic boolean tests directly in this potential if command, such as string equality, string matches regex... but that's another can of worms.

[mawww]

Any of those requests to extend the command language to allow for more scripting functionality will need very strong motivating use cases. I'd like to see a real case where that feature would have made a big difference.

For this specific proposal, the weak typing of the command language where everything converts down to strings makes it quite weird, as if %opt{str_option} %{...} %{...} might end-up calling arbitrary command that are not true or false.

The fact that we dont have any way to compare/test means we still have to rely on the shell for the condition, which means we could already do

try %{
    evaluate-commands %sh{ [ foo ] || echo "fail" }
    true case
} catch %{
    false case
}

Using try/catch feels like a hack and that is a good thing, I do not want to encourage such use of the command language.

The question I have is why try to avoid the shell so much ? It is indeed slow, but is it really too slow ? Kakoune command language is not exactly fast either. I am not sure how much escaping is an issue, here-strings can help. Highlighting is a bit more painful to work with, agreed.

Kakoune command language shares a certain pragmatic approach with the shell, not trying to be too clean or perfect, making it easy to write some throwaway logic that solves the current problem. Kakoune has the chance not to have accumulated as much historical wrinkles as the shell did but they ultimately both favor heavily the quick and dirty and happily leave the clean and general solutions to more capable languages.

[andreyorst]

The question I have is why try to avoid the shell so much ? It is indeed slow, but is it really too slow ? Kakoune command language is not exactly fast either.

So combining slow shell and slow commands is double slow ;)

I am not sure how much escaping is an issue, here-strings can help.

Yeah, heredoc can defenitively help, and maybe we could even highlight something like

<<KAKHEREDOC
    kakscript highlighted here
KAKHEREDOC

It has a downside of not being a very good citizen - it is kinda ugly to store such thing to a variable for example.

Now to the main part.

The fact that we dont have any way to compare/test means we still have to rely on the shell for the condition, which means we could already do

try %{
    evaluate-commands %sh{ [ foo ] || echo "fail" }
    true case
} catch %{
    false case
}

This is exactly what motivated me, and everyone in the discource topic to find a safe way of branching on an option. The try block you have here is incorrect. The correct one would be this:

try %{
    evaluate-commands %sh{ [ foo ] || echo "fail" }
    try %{ true case }
} catch %{
    try %{ false case }
}

Without these extra try blocks, we're subject of executing the "false" branch if anything, and I mean, anything goes wrong in the "true case". I've had such bugs in plugins I wrote, and I suspect that others did too. The cost of these additional try blocks is that if you have a common handler you have to duplicate it. It's not that big of a deal, but if doesn't have this problem and can be wrapped into single try block.

I do agree that such if is a too open system, and it calling an arbitrary command may be an issue. Still it provides a more clean and maintainable way to do small logic, without relying on try or shell.

[Screwtapello]

Let's take a snippet like this:

echo a
try %{ do-thing-1; do-thing-2 }
catch %{ echo b }
echo c

We might represent that as a control-flow graph like this:

-a---1---2---c-
      \   \ /
       `---b

...where do-thing-1 and do-thing-2 can both fail, and hence can both jump to the catch block.

One weird quirk of Python's syntax is that it allows else blocks on many different control-flow statements, not just ifs. In particular, you can write try/catch/else:

echo a
try %{ do-thing-1 }
catch %{ echo b }
else %{ do-thing-2 }
echo c

...which results in a control-flow graph like this:

-a---1---2---c-
      \     /
       `---b

...so if do-thing-1 fails, it's handled by the catch block, and do-thing-2 is not executed (just as in the first example) but if do-thing-2 fails, the failure is not handled and it can bubble up.

You can kind of emulate that behaviour in Kakoune today with something like:

echo a
try %{
    do-thing-1

    try %{
        do-thing-2
    } catch %{
        fail a-very-unlikely-message-prefix %val{error}
    }
} catch %{
    eval %sh{
        kakquote() { ... } # the usual definition
        case "$kak_error" in
            a-very-unlikely-message-prefix *)
                echo fail "$(kakquote "${kak_error#* }")"
                ;;
        esac
    }
    echo b
}
echo c

...but that's kind of unwieldy and can't be nested or the message prefixes would collide.

Adding if TEST A B to Kakoune would be a lot simpler and clearer to newcomers, but spelling it try TEST catch B else A might make it easier to fit into the existing implementation?

Although Kakoune try/catch statements can have multiple catch blocks, I'm not sure if it makes more sense to have one else block total, or one else block per "catch".

[Calum-J-I]

The question I have is why try to avoid the shell so much ? It is indeed slow, but is it really too slow ?

I'd argue the slow down is quite noticable when you have shell calls per-selection, i.e. in a mapping or -itersel. The problem isn't the speed of the shell but the overhead. I implemented a kakscript if in c++ and in simple benchmarks with -n -e 'command' there was no difference in time between the built in if and just kak -n -e quit. Opening a few dozen shells meant an arguably noticable 10ms delay.

The major benefit of logic commands such as if is that they allow you to branch over "abstract" strings. Currently any test you put in a TRY refers to strings in a buffer barring some hacky use of eval or exec. Screwtapello's suggestion still has this issue.

I've spend some time thinking about this, even implementing a few different versions of different commands and I feel like I have a decent understanding of the problem. My prefferred implementation of for is:

for [<switches>] <variable> in <items> <cmd>
iterate over <items> running <cmds> each time updating <variable>
Switches:
    -regex <arg>    assign <variable> to each match of provided regex
    -pairs          interpret each <items> as a key=value pair

for buffer in %val{buflist} %{
    echo %val{buffer}
}
for -regex '\w+((?=,)|\k)' list_item in 'item1,item2,item3' %{
    echo %val{list_item}
}
for -pairs ui_option in %opt{ui_options} %{
    echo %val{ui_option_key} %val{ui_option_val}
}

This is simple enough to implement. The final item is the commands to execute, the first is the name of val the value gets set to. Every intermediate item is the "list" of items to iterate over. -pairs makes working with dictionarys easier and -regex covers a common use case. You could also expose a %val{index}. -regex checks matches, not captures which would be available same as %val{hook_param_capture_n}.

I have been less satisfied with different implementations for if. Ideally you'd want if elif else to chain conditions but if you use switches for different conditions (e.g. regex match vs string compare) then you can't have them for each condition. You could use keywords like a match b a equals b but having special keywords like this is unusual for kakoune commands. Also with the way kakoune parses if ... %{} else if ... %{} else %{} cannot be nested as in c-style languages.

In a roundabot way I think @Screwtapello post contains the answer, adding a test or assert command.

assert-equals [<switches>] <item1> <item2>
compares <item1> and <item2> and fails if they are not equal
Switches:
    -regex   parse <item1> as regex to match against <item2>
    -nocase  case insensitive string match
    -not     invert match

try %{ assert-equals -regex "^\\h*%opt{comment_line}" %opt{whatever}
    do-something-with-comment
} catch %{ assert-equals true %opt{readonly}
    do-something-in-buffer
} catch %{
    echo whatever...
}

define-command if -params 3..4 %{
    try %{
        assert-equals %arg{1} %arg{2}
        eval %arg{3}
    } catch %{
        eval %arg{4}
    }
}

Making each comparison it's own kakscript command is a little more verbose (but this can be remedied with shorthands like the if I showed) but solves the syntax issues. Additionally it's easy to add new assertion types.

Irregardless of the exact implementation I think the core issue to address is that kakscript has no way to inspect the value of a string, most often this comes up as people wanting to branch over an option.

[sidkshatriya]

My two cents:

I've been using kakoune for approximately a year. Overall, its been a joy to use. The design is consistent and clean and its object-verb model superior to anything vim has to offer (I used neovim before switching to kak). Kakoune is an editor that I feel passionate about. Only the very excellent software evokes such feelings in me.

As I've dived deeper into a kakoune and tried to write some kak scripts I do feel that there are some things that kakoune could improve on.

I agree with the overall goal that we should not "re-invent" the wheel. Hence the hesitation to add some more features to the kakoune script "language". But I do feel that we're re-inventing the wheel in other places: the removal of ncurses dependency is a good example. Now the battle hardened ncurses has been removed, kakoune needs to take on the whole burden of making sure the experience does not regress for a variety of terminals. I feel that we are possibly being too conservative on this axis (language features) while being quite ambitious on others.

I'm sure there is reasoning behind removing ncurses which makes eminent sense. Also I do appreciate that every creator has some internal design goals and preferences. I'm happy to follow along because the same design aesthetics have given us kakoune as it is today.

But I'm still personally not sure about the merits of "shelling out" to do simple arithmetic (like adding a 1) or doing a if statement (the equivalent try-catch if you want to "do it right" is quite convoluted as explained by @andreyorst above). Its a crazy amount of computation to do something simple (the Linux kernel is involved and many syscalls like exec, mmap, prctl, brk etc are issued -- all to do something really trivial like add 1).

I think everyone is very conscious of the energy efficiency of things around us and it I'm sure all these shell invocations do not have a good impact on our laptop batteries. I have not way of scientifically proving it but neovim just feels a bit lighter as I use it. Kakoune's frequent shell invocations (many of which can be avoided) probably don't help.

10 ms was a number that was mentioned above for a shell invocation. While 10ms may be small, it eventually adds up because kakoune makes many shell invocations (some are unavoidable of course).

So I guess the point I'm making is that as someone who has progressed from an ordinary user of kakoune to someone who is comfortable writing the odd kak script, I'd really like some small conveniences that will make my life simpler when it comes to writing scripts. We already have a DSL which is quite sophisticated -- lets give it some small features (that are even smaller than the features that already exist) that will make things a bit simpler for script writers.

Once again, this is my 2 cents.

[tototest99]

Something like this comment may be inspiring.

On the other hand this has been discussed many times, like here with various solutions, like Luar.

We already have a DSL which is quite sophisticated

It is not, and that's a "problem" raised by quite a few people, except it goes against stated design, and is reminiscent of the Vimscript history. IMHO if there should be any depart from design notes, It's for people to stop to constrain/force themselves to write non-trivial plugins in Bourne/Bash. It is understandable as a choice for the minimal standard library, but Kakscript is IMHO much more an interface language to trigger hooks and pass options to environment, and programming this "state" should be deal with more comfortable/powerful languages, as in practice Kakoune allow us to, which is more elegant than many other text-editors. Besides, as users, installing language toolchains required by plugins are not so difficult today.

[gknittl]

After considerable struggle and with big thanks to Screwtapello https://discuss.kakoune.com/t/command-fifo-explanation/1918/2 I end up with testing the error message (if any) in the shell. Sample code to extract the error message to the shell (no tests on the message actually performed) included below. I had to write end of file to $kak_response_fifo to cover the case where the command succeeds and there is no $kak_response_fifo output from catch. The first case fails and the second case succeeds. remove the exit to demonstrate the succeed case.

!/bin/bash

include kak_command_fifo kak_response_fifo on command line when invoking this script in kak

e.g. nop %sh(test2 kak_command_fifo kak_response_fifo}

https://discuss.kakoune.com/t/command-fifo-explanation/1918/2

cat > $kak_command_fifo <<EOF try fail\ err1 catch echo\ -to-file\ $kak_response_fifo\ %val{error} echo -to-file $kak_response_fifo "" EOF error=$(cat $kak_response_fifo) echo info 1$(echo "$error"|sed 's/ /\ /g') > $kak_command_fifo exit cat > $kak_command_fifo <<EOF try info\ bonjour catch echo\ -to-file\ $kak_response_fifo\ %val{error} echo -to-file $kak_response_fifo "" EOF error=$(cat $kak_response_fifo) echo info 2$(echo "$error"|sed 's/ /\ /g') > $kak_command_fifo

For the moment, I think the above code will meet my needs. Perhaps it would help to have a $kak_stderr_fifo available from kakoune that scripts could read from, Kakoune would put out error messages and empty messages for success. I guess the script would then have to read it after each command. Essentially this is FIFOing *debug*. Perhaps there is already a way to open a FIFO for *debug*?

try/catch has the effect of requiring extra commands to write error messages to *debug*. $kak_stderr_fifo could provide an option to continue to log the messages to *debug* or it could be the default

2024-04-24 additional thought: Maww writes above:

Using try/catch feels like a hack and that is a good thing, I do not want to encourage such use of the command language.

The question I have is why try to avoid the shell so much ?

I'm all in favour of scripting kakoune in the shell but as far as I know try/catch is the only way to get the error to the shell. Part of the struggle in writing the code snippet above was trying to write it without try/catch and having the $kak_command_fifo and $kak_response_fifo FIFOs block. It is documented but it still took me a while to realize that kakoune was blocking on error by design and not because I forgot to read from a pipe or wrote to a pipe incorrectly or messed up the escaping.

I'm still learning FIFO pipes but it seems to me that a $kak_stderr_fifo might be able to do away with try/catch. I.e. the default behaviour of blocking on error could be achieved by writing errors to $kak_stderr_fifo and only unblocking if the script reads the error off the FIFO. Possibly no need for try/catch at all. The only glitch is that the script doesn't know when kakoune is going to throw an error so I think the script would have to read $kak_stderr_fifo after each command. In my limited testing the shell blocks on reading a FIFO if there is nothing to read, which I got around by writing zero length strings to $kak_response_fifo, although there are apparently fancy shell tests to determine if a FIFO is empty... Perhaps try remains to flag the commands where the script wants to handle errors and $kak_stderr_fifo only replaces the catch block...