PyCmd on Linux (running on top of bash)

[horeah]

As an experiment, PyCmd can now also run on Linux (or WSL) as a front-end for bash; this issue is for collecting and tracking related feedback.

Rationale: while shells focused on user interaction do exist in the Linux world (e.g. fish, and also zsh in some regards) and they are way more advanced than PyCmd, there are still a few areas that PyCmd/Linux covers better (at least in my opinion):

• Runs straight on top of bash, and therefore can directly run bash commands, source bash scripts etc. This means bash users (the vast majority of shell users in the world, for what it's worth) can use it straight-away with no additional learning or adaption.
• Offers consistent interaction (appearance, completion, shortcuts etc.) across Windows (running on top of cmd) and Linux (running on top of bash). For users who need both (like I do for my $DAYJOB) this makes for reduced cognitive overhead.
• The command search (both immediate filtering with the "Up" arrow and the modal "Ctrl-R" search) is more powerful and intuitive than what I (for one) have seen in other shells.
• Completing wildcards is much more useful than the typical shell: the matching files are listed and can be selected interactively from the completion list.
• The directory history (fish also offers some of this, but in a more rudimentary form).
• Git & SVN-aware prompt out of the box
• Friendlier editing (selection, undo, token expansion with Alt-/)
• Pressing Esc pushes the command to the history; this doesn't sound like a lot but for me it is very helpful in practice (I often start typing a command, then realize I am in the wrong folder; I can press Esc, then cd to the correct directory, then use the Up arrow to retrieve the half-typed command).

There are a lot of things that are currently not working or need to be improved, of course; the most important ones are listed here:

• Multi-line commands are not really supported
• Running commands that produce a lot of output is much slower compared to a typical shell; this is because the output is intercepted by PyCmd and therefore processed in Python (this does open opportunities for interesting new features though; e.g. a shortcut that would copy the output of the previous command to the Clipboard).
• Using the clipboard directly from PyCmd doesn't work for now; one must use the terminal copy/paste functionality like with any other shell
• Shell variables (as opposed to exported variables) are not expanded when completing
• Internal commands are not completed
• Quoting is not properly accounted for when completing
• Bracketed paste is disabled
• ... more will probably be discovered along the way

One can try it out using release bundle "20230424-linux64", or by checking out branch "mesh" -- note that I consider this a "private" branch for now and might be still amending/rebasing; if someone wants to contribute, some prior coordination will be needed.

[horeah]

@santagada I believe you are a Linux/shell user -- I would really appreciate your feedback on this one.

[santagada]

Indeed it seems interesting, having most features of fish without changing your shell syntax. But I would say there's still plenty to do to make PyCMD even more awesome.

By the way, PyCMD has been my daily driver for a long time, I forget how good it is until I have to use a machine without it.

[ufo]

In a WSL/Ubuntu shell I cloned the repo into ~/PyCmd and checked out 'mesh'. When I manually run it via python3.11 ~/PyCmd/PyCmd.py then everything works ok. But when I place python3.11 ~/PyCmd/PyCmd.py at the bottom of ~/.bashrc, in order to use it as the default shell, then PyCmd endlessly loops printing all output vertically thus only one character per line:

W
e
l
c
o
m
e

t
o

P
y
C
m
d

<
n
o

b
u
i
l
d

d
a
t
e
>
-
x
6
4
!

[
u
f
o
@
H
P
4
7
0
]

~
>

Maybe somebody has an idea about this behavior?

/EDIT: It actually works when placing it into ~/.profile instead, which would be more correct anyway.

[ufo]

Running commands that produce a lot of output is much slower compared to a typical shell; this is because the output is intercepted by PyCmd and therefore processed in Python

I was curious and also did some measuring. With PyCmd it was indeed 4 times slower on Linux compared to bash w/o PyCmd. But how come that PyCmd is not slower on Windows at all compared to a classic Windows terminal w/o PyCmd? On Windows the test command dir /s /b C:\Windows\* took 30 seconds, no matter if with or without PyCmd.

[horeah]

In a WSL/Ubuntu shell I cloned the repo into ~/PyCmd and checked out 'mesh'. When I manually run it via python3.11 ~/PyCmd/PyCmd.py then everything works ok. But when I place python3.11 ~/PyCmd/PyCmd.py at the bottom of ~/.bashrc, in order to use it as the default shell, then PyCmd endlessly loops printing all output vertically thus only one character per line

The mechanism for wrapping bash is kind of hacky, PyCmd actually creates a customized bashrc based on the user's bashrc and passes it to a newly started bash; this is not very robust, and can create weird behavior as you just noticed. I'm not entirely happy with the whole thing, but bash is devilishly hard to control from an outside program so I have to accept whatever way I was able to come up with :) -- at least for now.

I was curious and also did some measuring. With PyCmd it was indeed 4 times slower on Linux compared to bash w/o PyCmd.

I am surprised that you only see a 4x slowdown -- in my tests, I sometimes see up to 20x (e.g. for commands which produce large amounts of output with very little processing or I/O, such as find /usr/lib). I am currently testing an improvement that makes it twice as fast; this is still far from great, but since PyCmd is targeted at interactive usage it is not a huge problem in practice (running find /usr/lib in an interactive session is not something people do often; for most interactive commands, the output-printing overhead is negligible compared to the actual "work" that the command performs).

But how come that PyCmd is not slower on Windows at all compared to a classic Windows terminal w/o PyCmd? On Windows the test command dir /s /b C:\Windows* took 30 seconds, no matter if with or without PyCmd.

The explanation is that the mechanism of wrapping around cmd.exe is completely different (just as hacky, but in a different way) from the mechanism used for wrapping around bash. Both could probably be improved, but it would take a lot of work and I cannot afford to spend that much time :(