Integrating with external processes

[hovsater]

👋 @vamolessa, this is not so much an issue than it is a question.

I read that one of pepper's goals were Extensible through external cli tools on the Handmade Network's website. Is this true, and if so, how?

I'm coming from dte and was hoping pepper would provide some of the same features as dte provides when it comes to integrating with external tools. dte has a section on External Commands and I've found the following really useful:

command	description	pepper equivalent
filter	Filter selected text or whole file through external command.	?
run	Run external command.	?
exec-open	Run external command and open all filenames listed on its standard output.	find-file (sort of)

The main thing in dte I find powerful is the concept of exec. In Unify commands that run external programs into a single "exec" command this concept is explored further where a single construct, exec, can be used to implement a handful of other useful commands.

I really love being able to run :run tig blame $FILE to temporarily open tig's blame view on the current file and after exiting ending back in my editor. exec-open allows me to run an external program such as fzf and then see the selected files opened in dte.

[vamolessa]

Yeah, one of the goals is to, whenever it makes sense, be able to delegate tasks to external programs instead of reimplementing them.

At the moment, there are mainly 7 ways that we can interact with external tools:

on normal mode:

• | will prompt a readline in which you can typein a command. Then it will pipe in all selections (one for each invokation) to a program and substitute them to it's stdout;
• ! is similar to | but it won't pipe in the selections (stdin will be closed);
• $ will also prompt a readline for a command, but will only execute it with no piping. This is most useful for one shot things like "open a new terminal split with git on the same dir";

commands:

• find-file takes a command string param whose each stdout line is a file entry for the builtin fuzzy picker. When selected, that file is opened as a buffer
• find-pattern takes a template command string param. It will prompt you for a search pattern and then replace it in the template into a {}. Then it will call the resulting command and its stdout will be appended to a new buffer. From there you can use gf on normal mode to open the file under the cursor (or GF to also close the current buffer just before opening the new file);

also commands, but more of a way for me to implement clipboard support without having to link against other libs other than libc

• copy-command takes a command string param which, if non-empty, is invoked when trying to copy selections to the clipboard (using stdin). If empty, it only uses pepper internal clipboard storage;
• paste-command same as copy-command but for pasting;

This is not supposed to be the ultimate list, though. I'm open for more integrations suggestions (I didn't know dte until now!). Those were only the ones that I needed the most for my use cases.

How do usually use that filter when selecting selections? Would | or ! or $ suffice to replicate that run command? That exec-open is also interesting, but do you use it in the middle of a coding session or right after you open the editor?

[hovsater]

Appreciate the detailed reply, @vamolessa. I've been dealing with some personal matters, so sorry for the late reply.

$ will also prompt a readline for a command, but will only execute it with no piping. This is most useful for one shot things like "open a new terminal split with git on the same dir";

I have yet to see this work. Could you provide some examples of this? Whenever I press $ and type in a command (e.g., tig) nothing happens.

How do usually use that filter when selecting selections?

In dte, filter works on the current file (or the current line if -l is provided) when no selection is found. I tend to use it for things like formatting the file (e.g., :filter elm-format --stdin). I think it works similar to what | does in pepper, right?

Would | or ! or $ suffice to replicate that run command?

Maybe. I still need to understand $ better. I use run for all kinds of things. Opening tig in the root directory using :run tig, git blaming the current file using :run tig blame $FILE where $FILE is the path to the file in the active buffer. I use it to run a file manager using :run nnn and a lot of other different use cases.

That exec-open is also interesting, but do you use it in the middle of a coding session or right after you open the editor?

exec-open is just another command. I use it all the time to open files in the middle of a coding session. I have the following binding in dte:

bind C-P 'exec-open fzf --multi --preview="cat {}"'

This will spawn fzf and allow me to select files (with a preview). Whenever I press enter the selected files open in dte. It's really nice. Basically whatever the command to exec-open outputs to stdout, will be treated as file paths that dte will open (one file per line).

Hope this makes things more clear. 🙂

[vamolessa]

I have yet to see this work. Could you provide some examples of this? Whenever I press $ and type in a command (e.g., tig) nothing happens.

Were you expecting tig to take over pepper's terminal window? Because that's kinda hard since it's the server (a background process) that spawns the processes so it can't 'see' its clients ttys. What I usually do is to communicate with the terminal telling it to spawn a new split with the program I want to run there. It's also what Kakoune does if I'm not mistaken.

• Here you can see an example of such integration with Kitty: https://github.com/vamolessa/pepper-config/blob/master/linux.pepper
• And here with Windows Terminal: https://github.com/vamolessa/pepper-config/blob/master/windows.pepper

In both examples (both first line), I tell the terminal to spawn the program verco in another split.

In dte, filter works on the current file (or the current line if -l is provided) when no selection is found. I tend to use it for things like formatting the file (e.g., :filter elm-format --stdin). I think it works similar to what | does in pepper, right?

With | you can pipe all selections individually to the stdin of a program and replace them with its stdout. You could integrate the elm formatter like so: alias elm-format [[enqueue-keys "<esc>aa|elm-format --stdin<enter>"]] (The <esc>aa in the beginning will clear all selections then select the whole buffer). This will create the command elm-format (actually an alias) which you can then latter map to some key chord.

Maybe. I still need to understand $ better. I use run for all kinds of things. Opening tig in the root directory using :run tig, git blaming the current file using :run tig blame $FILE where $FILE is the path to the file in the active buffer. I use it to run a file manager using :run nnn and a lot of other different use cases.

So what's probably happening with tig is that it simply quits itself once it sees there is no stdin or tty (nor stdout) to attach to (it's what $ does). So the solution is really to instead communicate directly with your terminal and tell it to spawn such process in a new split or tab.

Also, there is no variable of sort that expand into bufferpath or something. Pepper commands are really simple, it just interprets a line and that's it. However, it's possible to implement more complex behaviors by integrating a plugin. However that would mean coding in rust and using the pepper crate and then building your own code editor.

exec-open is just another command. I use it all the time to open files in the middle of a coding session. I have the following binding in dte:

If you're only opening a single file, you could use the builtin find-file command which takes an external command that output file paths (like fd and find). It then presents you with a fuzzy file picker menu like fzf. It's the default way of quickly opening a new file. Here's the macos default binding (which uses find): https://github.com/vamolessa/pepper/blob/master/pepper/rc/default_macos.pepper#L4

However it should also be possible to actually use fzf by using the 'communicate with the terminal' trick. You can invoke pepper like pepper --as-focused-client --quit <files to open...> which will connect to the server acting like the currently focused client, send it commands to open all the files listed and then quit right after. By using something like xargs you are able to redirect the fzf selections as file to open in the current client.

Alternatively, you should also be able to do this whith the help of a little rust code as a plugin.

Hope this helps you better configure it for your needs. Some solutions may seem kinda convoluted right now, but it's what you CAN do right now. In the future plugins may be easier to write and consequently make it easier to hack pepper.

Thanks for your feedbacks :)

[hovsater]

Were you expecting tig to take over pepper's terminal window? Because that's kinda hard since it's the server (a background process) that spawns the processes so it can't 'see' its clients ttys. What I usually do is to communicate with the terminal telling it to spawn a new split with the program I want to run there. It's also what Kakoune does if I'm not mistaken.

Right, that makes sense. Thank you for the clarification. 🙂

Also, there is no variable of sort that expand into bufferpath or something. Pepper commands are really simple, it just interprets a line and that's it. However, it's possible to implement more complex behaviors by integrating a plugin. However that would mean coding in rust and using the pepper crate and then building your own code editor.

Would it make sense to support some kind of expansion out of the box? Initially, perhaps the path to the current file. In Vim, there's filename modifiers and in Kakoune there's expansions. For instance, if you want to interactively blame the current file using tig blame, run the tests for the current file, or run any CLI command on the current file really. It's not always possible to just work with stdin and writing specific plugins for one-off use cases feel a bit over the top.

However it should also be possible to actually use fzf by using the 'communicate with the terminal' trick. You can invoke pepper like pepper --as-focused-client --quit which will connect to the server acting like the currently focused client, send it commands to open all the files listed and then quit right after. By using something like xargs you are able to redirect the fzf selections as file to open in the current client.

Right. I keep forgetting that we have a server/client architecture. This is really helpful, thanks! 👍

Hope this helps you better configure it for your needs. Some solutions may seem kinda convoluted right now, but it's what you CAN do right now. In the future plugins may be easier to write and consequently make it easier to hack pepper.

Absolutely. I appreciate you're taking the time to help me. I really like the overall approach pepper takes! ❤️

[vamolessa]

Right, that makes sense. Thank you for the clarification. 🙂

No problem!

Would it make sense to support some kind of expansion out of the box? Initially, perhaps the path to the current file. In Vim, there's filename modifiers and in Kakoune there's expansions. For instance, if you want to interactively blame the current file using tig blame, run the tests for the current file, or run any CLI command on the current file really. It's not always possible to just work with stdin and writing specific plugins for one-off use cases feel a bit over the top.

That would make sense, yes! It's in fact something I tried doing before a couple of times but was never satisfied with the design. Which is why I ended up removing it entirely. However, that document about Kakoune's expansions is something that I did now find at the time and already inspired me some solution for pepper :)

Right. I keep forgetting that we have a server/client architecture. This is really helpful, thanks!

Alright! Cool!

Absolutely. I appreciate you're taking the time to help me. I really like the overall approach pepper takes! ❤️

Thank you so much! I'm glad it's of use for other people and thanks for the reports so far :)