Feature suggestion: Interactive Shell

[catpea]

I am talking about a proper Bash substitute a marriage of good ideas between Node.js REPL and https://www.npmjs.com/package/vorpal (or "Tiny CLI")

It will make you famous, a bit more handsome, and you'll be the talk of the town.

Yes.

[gcperrin]

Bump on this idea.

[antonmedv]

UPDATE: Now zx supports a repl via --repl flag.

zx --repl

[catpea]

As a replacement for Bash. And it is not just me, all people who work with node.js need it.

zx already postulates that shell scripting is not perfect, it is not something we want to do in bash.

The next step is to replace the interactive shell, so that we may write our commands and one-liners in JavaScript.

I think everyone is ready to:

sudo dnf install zxsh

and

cat /etc/shells

/bin/sh
/bin/bash
/bin/zxsh

and

chsh -s /bin/zxsh

and forget about shell scripting in /bin/sh ... I hate to say it, but, this is the middle of 2021 we have react and svelte we need more /etc/shells, we need modern languages all over, we can't do this anymore:

yourfilenames=`ls ./*.txt`
for eachfile in $yourfilenames
do
   echo $eachfile
done

Again, I hate to say it, but the 80's are over, and we need to leave dusty shells behind.

Anybody else with me on this one? I don't think antonmedv wants to read my comments anymore let me have some +1 here, halp.

[biels]

Yes, IMHO this can be an excellent project if it becomes a proper Bash substitute with the exact same features (or more) and same or similar performance.

[Phate6660]

And it is not just me, all people who work with node.js need it.

You don't speak for everyone. I work with node.js and I would NOT want a JS shell.

and forget about shell scripting in /bin/sh ... I hate to say it, but, this is the middle of 2021 we have react and svelte we need more /etc/shells, we need modern languages all over, we can't do this anymore:

yourfilenames=`ls ./*.txt`
for eachfile in $yourfilenames
do
   echo $eachfile
done

for i in *.txt; do echo "${i}"; done

Maybe you should really learn a language, or at least practice it, before criticizing it? Implying that a language is old and (presumably) too verbose / complicated, and then giving a bad example is ironic when you're trying to make a point.

But for the sake of argument, please enlighten me on what you find to be so bad about bash. And I will in return refute each and every point you make.

Yes, IMHO this can be an excellent project if it becomes a proper Bash substitute with the exact same features (or more) and same or similar performance.

So... what? Are you saying it's not an excellent project if it doesn't get an interactive shell?

[gcperrin]

The following is just my opinion, take it for what you will.

While Bash has no inherent problems (frankly I love Bash for certain things), the concept of a REPL-style shell is interesting for a couple of reasons...

1. Developer productivity -- Context switching between languages has proven to drastically decrease developer productivity. Any tooling that helps minimize this is worth the price of admission.

2. Use-case -- While I write bash line-for-line in a script no problem, I use it in the shell format about 100x more often in my daily workflow. I.E. I've written many more lines of Bash in the shell itself, rather than scripting it for later use. I suspect many developers are the same. Augmenting this workflow with Node (for me, at least) has a certain attractiveness to it, if done correctly.

I suspect dropping into a node shell and importing a globally installed version of ZX would yield some interesting results, and help conceptualizing ideas on if/how to proceed. It'd be nice to have total Bash compatibility...you can use ZX features, but aren't tied down to using them exclusively.

It certainly might be beyond the scope of this project and warrant another new body of code that leverages the library.

[Phate6660]

And that is both a respectable, and agreeable, answer.

What I was mad about before was not only the presumptuousness of the commenter (e.g. speaking for everyone), but also the fact that they tried to slander a language they obviously have little to no experience in.

I mean, in defense of the commenter, I shouldn't've lost my cool like that. But there are also better ways to reason for something like this (your comment gcperrin is a shining example).

[catpea]

@gcperrin Thank You. Phate6660 I do not mean to offend, remember that some people can't handle others getting mad at them, as anger is such a frightful thing. So, be good to strangers, inspire them, rather than punish them.

3. (to continue your points gcperrin from my own perspective as you wrote so well). Hackability is another big deal. Clicking "View Source" is what brought many of us here. Bash is not some profound truth or standard, it is a shell written in C. A JavaScript shell would be written in JavaScript. https://www.npmjs.com/package/cash is a good way of visualizing that (they have a fun little gif movie on their page). This means that users would be able to look under the hood, and learn what makes it tick.

4. There is a right language for every scenario. Brian explains this beautifully: https://www.youtube.com/watch?v=xnCgoEyz31M Also, he kept his youth: https://youtu.be/tc4ROCJYbm0?t=248 hasn't aged at all :) How do great beings do this? Ergo, a shell in a cross platform language, is also a cross platform shell (no cross compiling, no futile soon to be short lived kernel abstractions, no multiple binaries - it just works) And yes, WEB is a platform! sprinkle some mem-fs and you have a web terminal going, a wesh?. We must not underestimate small ideas, it is the small ones that change the world.

5. Every epoch needs new shells. We are in a new epoch. What is beautiful about this is that the fundamental language of the shell does not change. It is still a | b the perfect reduction of the infinite complexity of the beauty OF ALL programming languages (especially COBOL, but never mind that) Some may not realize that a|b is not a one dimensional thing. As a can have multiple output streams to make a tree. It is a graph, my goodness! You can see a visual representation of the tree nature of a | b, in programs like Node-RED: https://www.youtube.com/watch?v=ksGeUD26Mw0 you see a | b represents a thing that has legs, and a thing grew on top of it, a certain class of visual programming languages - that one can't help but to appreciate.

Anton the patient person that he is for tolerating shell talk in a program about scripting, is coding in that ancient-computer space. zx has the making of an important component of a new timeless shell, zx lives within that majestic a | b world that made Unix worth the while to so many of us.

And with each new epoch new things must come.

I think the beauty of the EventEmitter will somehow feature in the new js shell once it happens, we will see things born in the DOM (document object model) come to the shell born of ancient UNIX. Wow, I say, wow.

And if the magic of Life and Cyberspace taught me anything, then it will be the person that complains about all this, and gets mad at everything the most, that will eventually create the shell that we will all, without exception, end up using.

Also for i in *.txt; do echo "${i}"; done is actually the magic that we want to keep, it is not a burden, for i in *.txt; do echo "${i}"; done if of the same ilk-and-magic as a | b it is that perfect reduction that makes things worth the while for the future programmer. We just need to adjust the grammar a tiny bit and make do and done keywords optional - humbly and respectfully if I may.

[Aheal]

Hi 👋 not related to the main topic of discussion but this isn't descriptive enough

for i in *.txt; do echo "${i}"; done

Something like this will make more sense to the point of "practicing and bla, bla ..."

for file in *.txt; do echo "${file}"; done

Also, I agree with @catpea about being polite on the internet, remember that behind each monitor there is a human being 🙂

[dospunk]

A possible compromise could be adding a file called something like interactive.js that could be required in an existing node shell that adds all the zx globals. Something along the lines of this stackoverflow answer could work.

edit: an example of interactive.js:

const zx = require('zx');
module.exports = function(){
    this.$ = zx.$;
    this.chalk = zx.chalk;
    this.cd = zx.cd;
    // and so on and so forth
}

And then in the shell, one could just call require('zx/interactive')()

Edit 2: Or to make it even easier to maintain,

const zx = require('zx');
module.exports = function(){
    for(const funcName of zx){
        this[funcName] = zx[funcName];
    }
}

[biels]

IMO @dospunk way is totally a very good idea and clean way of doing it.

[catpea]

I have been thinking about this line: for file in *.txt; do echo "${file}"; done

In my experience, one word is still not enough, it should be fileName which by forcing you to use a pair of identifiers actually helps in debugging, so I recommend: for fileName in *.txt; do echo "${fileName}"; done

But that is just the surface issue.

What do we do with the beauty of Command Line as we know it from UNIX?

Above I mention, that we want to preserve a | b and now for fileName in *.txt; do echo "${fileName}"; done and I still believe that to be true, but not in context of zx, zx should be about JavaScript.

What I was originally thinking about is using Sweet.js, and moving on to PEG.js or Jison or the Example Rich (see fun-lang.ne) nearley.js and just get the best of Bash into JavaScript, that would then be used in the Interactive Shell. This is not a great programming language idea, but it is pretty useful for the Interactive Shell.

But, zx is great at being smol. For example Sweet.js brings in Babel, and we don't need that. So, Anton's take on a | b is very nice, it is powerful and flexible, and plays nice with JavaScript:

let {stdout} = await $`echo "hello"`
  .pipe($`awk '{print $1" world"}'`)
  .pipe($`tr '[a-z]' '[A-Z]'`)
assert(stdout === 'HELLO WORLD\n')

(Anton's snippet taken from #91)

as to for fileName in *.txt; do echo "${fileName}"; done

Maybe iterating over files should become another built-in so we would end up with:

$.glob("**/*.js").map(fileName=>console.log(fileName))
// weeee you don't have to add .bind() to console.log, it just works now

(I hint on using isaacs's glob in sync mode or just await it as it is usual in zx)

So we are not really giving anything up by not bringing bash into the Interactive Shell. I hope I didn't create too much confusion, and I apologize.

biels and dospunk are bringing in the zx built-ins to the REPL with their thinking here, and they are correct. REPL way is the right way, forget Tiny CLI we don't want any extra layers lets just JavaScript because that is why we are here.

As to integration, I think Anton may wish to create a separate project, to keep things simple. But if one was to add it directly into zx then I would recommend an -i argument to zx, short for --interactive.

Those of us who are not interested into being dropped into an Interactive JavaScript Shell at login, would simply run: zx -i at the Bash command line.

In my research I have come across xtermjs which interprets rich ANSI command output into HTML. This means that the next step after having an Interactive JavaScript Shell could be an electron driven JavaScript terminal very similar to Hyper (as mentioned on xterm's website) but this terminal could tightly integrate with zx and perhaps even allow for using a GUI similar to Node-Red which after all is the GUI to creating rich CLI Pipes!

Finally, I would like to invite you all to practice creating your own programming languages with programs like PEG.js, Jison, and nearley.js. At the very least, at least try making your own operator with Sweet.js (see bottom of tutorial) I swear this is as healthy as eating vegetables, and has been medically proven to delay pirate scurvy (citation needed).

[dospunk]

The repl api could be used to create a zx -i command that starts a node repl with zx already imported

[catpea]

Are we going to introduce ZX Internal REPL/Shell Variables? (give thumbs up) Should we just ignore this entirely, and let the user take care of this? (Thumbs down)

And should we have a little configuration file, or better yet module that gets executed at startup, say from ~/.zx/zx_profile.js And if we are going to have a config file will there be a master config file next to which we could have a node_modules that zx's npm command could install things into?

Examples (I am using bash syntax here )

$? Exit code of the last process $_ Last argument. $! PID of last process.

Ideas

$ZXSH Command that launches the shell $ZXPID Process ID of the current instance $ZX_VERSION $EUID Identification number of whatever identity the current user has assumed, perhaps by means of su. $GLOBIGNORE (when using glob() that I mentioned above) $GROUPS Groups current user belongs to

See the full list here for more ideas: https://tldp.org/LDP/abs/html/internalvariables.html

[NullVoxPopuli]

Pls no. JavaScript is way too slow to be used broadly for CLI utils... Especially when it comes to I/o.

Even if I use sub_process and use the faster bash equivalent, parse that, etc, native bash is still significantly faster.

Like.. I'm talking close to 1s for js to load run find on a couple directories and present a prompt. In native bash, it's almost instant.

We can go faster though.. by using a language that compiles to machine code and has actually good performance. Example: NVM vs volta.sh. volta is written in rust, and blows the pants off any node version /toolchain manager out there. Golang would also be sufficient..

[Perdiga]

First I liked this project idea, Kudos for the idea. I also like the interactive mode.

Most of the time I need to do scripts to set up things or CLIs, for most of the developers doing this in bash is hard, but doing in js would be easier. For most of these works I don't need to be faster than bash I need to be productive, and be easier for new developers. If you need performance you can always use the native bash.

Thanks again for this amazing project

[webduvet]

Sure, as an alternative why not. But replacing bash? no thanks. I know it is going to end up like - so do you want to tail the stream? no problem, just run npm i -g and it will provide you with 127 dependencies, 23 with vulnerabilities and 6 obsolete.

[catpea]

Daniel Shiffman when talking about evolution of Processing describes an important process: https://www.youtube.com/watch?v=8j0UDiN7my4&t=149s He asks the key question at 4:00

I propose that we by subtle analogy ask the following:

What if BASH was designed with today's technologies.

I suspect that Bash is very small, its potential "127 dependencies", can actually be reduced by just programming in the reusable functions, creating a wee mono-repo. Though we should prefer npm, and active npm projects. This is just the first version of the shell, let us make no assumptions yet. Perhaps it will become a mere REPL that you launch from Bash, it will become whatever is most fantastic to have.

Which leads me to the following.

Critics should see this as a small research project that is yet to identify the weaknesses of JavaScript, in context of shells. Nothing negative you say about npm or JavaScript comes with any verifiable proof in context of shells, and don't forget about programs like nexe which invalidate a lot of initial assumptions. This is the project that will bring in results and numbers for all tho see. And I assure you that many will enjoy having a slower but more hackable shell, in a language that they prefer and feel passionate about.

Above all, we are programmers, we know no bounds. There are no limits, we are inventors, we thrive on breaking them.

Many of us have outgrown saying things like JS, or ASP, or Perl, or BASIC, or COBOL, or Michigan Algorithm Decoder Symmetric LIst Processor... is bad. Because we do things to study, to learn, to master, and sometimes to prove ourselves wrong, though only sometimes.

Sometimes crafting a beautiful shell that does end up being slow, will teach a programmer what they need to do in GO, or C. But also at the same time, make someone extremely happy; that they have a shell they can hack on, and extend, and make it their own, and learn from, and hug and squeeze, and kiss, and listen to the sea from :shell: (don't ask)

These are the people that make programming so beautiful, they appreciate the program for its beauty.

If we do this shell right, retired coders will reminisce about it 30 years from now, same way that some people still talk about Bill Atkinson's HyperCard (and please don't do drugs, Jesus Bill, now is not the time.)

So, Anton stands at the cross-roads of the future, he is now scanning for wisdom to make the right choices.

If he makes the right choice, he will make an unforgettable program.

Right now he can replace REPL with Ink and freakin' give us a responsive prompt that will always show up-to date information on remote git repositories, CPU temperature, and usage, all in the same line:

pull (3) 62° 8% zerocool@gibson cicada3301>

(and probably by means of a cute DSL prompt.gitPull().cpuTemp().cpuUsage() or something.)

And as much as I understand the negativity for replacing bash, that is what the beautiful and enchanting Raspberry PI Zero W needs, a modern language, to make the tiny little itsy bitsy SBC even more useful.

Anton, what you have is revolutionary, prepare for making 5 reference implementations and three shells from a hard core one that uses await grok('-bork', '${FLARP}').pipe(...) JavaScript functions, to your Perl backticks, and maybe even one that supports modes, because ZOMG vi's modes is the thing we don't yet know we need, how dare we assume it is bad (The left "menu" on https://console.cloud.google.com may reveal that we might have done something wrong, but let's just assume I am wrong.).

And maybe consider something with curses, take a look at https://www.npmjs.com/package/blessed which would let you explore Norton Commander like UI, where you navigate the FS/VFS on the left, and get contextual options on the right. This could simplify git npm/yarn, updates, and would be EXTREMELY useful for devices like the RPi Zero W.

Don't worry about negative comments, because they only consider one selfish target implementation, they can't conceive of your shell running on a proper linux version of https://www.pine64.org/pinetime/ or PinePhone, or in the King's Cross terminal on Librem 5.

I suggest purchasing an under-powered SBC of your choice, and seeing the future development of your shell from the perspective of your little device; just to experience how much changes.

In closing, it is good to be a critic, it is good to stand up for the languages and programs that are important to you, don't worry about the people that make you feel bad for feeling strongly about something that you are passionate about.

And just like you weave a library of top modules with good support, make a library of Tools For Thinking. Tools like the process Daniel Shiffman explains, and the vision you get from approaching your interactive shell development by means of a small and slow Single Board Computer.

I'll leave you with another one of Bret Victor's talks, The Future of Programming

And thank you all for all your thoughts and observations, all genius is equally precious.

[linonetwo]

electron driven JavaScript terminal @catpea

If someone is interested in creating this, I have a similar example https://github.com/tiddly-gittly/TiddlyGit-Desktop/releases/tag/v0.4.5

I have successfully make zx work in an electron note taking app, now I can execute note as script, and use git sync to backup those scripts.

https://user-images.githubusercontent.com/3746270/133672483-6188dd11-4c53-4a3a-a5fe-468bc1f15220.mov

[antonmedv]

It is done! Zx now comes with a REPL! https://github.com/google/zx/releases/tag/7.0.0