[QUESTION] Potentially used to speed `.t` tests

[quark-zju]

The .t tests used by Mercurial is basically a shell script with fancy input output checks. It was extracted to https://pypi.org/project/cram/.

While it works reasonably fine, there are some issues:

• Very hard to setup correctly on Windows.
• Dependencies are unclear - can run anything in $PATH.
• High overhead for repetitive commands, especially Python ones with high startup cost. Even more overhead if it has to go through some buck wrappers to setup Python or ld library paths.

I have been thinking about a shell interpreter that accepts a subset of the shell language for the .t case, and avoids spawning processes. It needs a shell-ish features but with one extra feature:

• Optionally redirect command executions to in-process Rust functions. for example, cat will be redirected to a builtin Rust function fn cat(in: impl Read, out: impl Write, err: impl Write) -> ExitCode.

That would allow us to explicitly control dependencies, and potentially reduce the Python startup overhead (by pre-import things).

Now the question is, do you think wzsh can be purposed in this direction? I think the questions are around:

• Abstract process model so it could be driven by threads with Rust io abstractions for stdio, and pipes, redirections etc still work.
• Maybe implement some missing subsets like for, while used in .t tests.
• Have some sort of "library" interface that can be embedded and used in other places.

[wez]

I think wzsh could be used for this, and maybe without a huge effort; the loop constructs are probably the most difficult part, and those shouldn't be all that hard.

In theory, you'd target the shell_vm crate for the library interface; you'd provide a ShellHost impl for spawn_command to control what/how things are spawned, and in there you could decide how to handle the "spawned" command; the interface is intended to allow for immediately processing the args/environment, or spawning a thread or a child process with some captured information.

Some of the shell builtins live in the top level wzsh command crate (https://github.com/wez/wzsh/tree/master/src/builtins) rather than shell_vm, so it will probably make sense to move those around; perhaps it is worth making a wzsh-embed (or insert a better name) crate that is a slightly augmented version of shell_vm with some more general useful helper functions included.

[quark-zju]

It seems another difficult area is parsing. Some tests use syntax like $((x+1)), or [ a = b ]. Support seems missing. I found the lexer and parser are hand-written. Have you considered tools like lalrpop?

[quark-zju]

I found 2 Rust projects:

• https://github.com/ipetkov/conch-parser
• https://github.com/ipetkov/conch-runtime

They seem quite complete for the .t use-case: Namely, it has $((x+1)), heredoc support commonly missing elsewhere. And it has abstractions to customize process execution. I will probably try to build something on top of them. Thanks anyway!