[Proposal] Using POSIX shell commands in tasks chain

[hzlmn]

It is just a weird idea and I want to know what do you guys think.

Basically I like Make due to it's "UNIXy", like you have a lot of tools for text wrangling and you can do almost everything you can imagine without any hustle, just by using well known shell power tools.

But when we speak about node build systems, we have some limitations. We run builds usually in *nix env, but what if I want to make this steps.

• read file
• get subset of it
• replace character with other one
• save to file

This is a trivial task using tools like tails, sed or grep, but it involves writting custom plugins with Fly

Basically what I propose is to add method shell or something similar to Fly instance that will look like

x['get-subset'] = function () {
  yield this.source(someFile)
    .shell('head -10 | sed -i s/test/changed $file')
    .target('/dist')
}

How this can be acomplished

.shell accepts raw string of shell command parse it and generate proper calls using shelljs and returns execution result.

It can be useful for additional text transformation or some file system manipulations before or after task.

cc @jbucaran @lukeed

[lukeed]

I'm not sure if I value it as a core API.

In my mind, I see Fly as the office manager. Takes requests (source) from the boss, then it tells its workers to do things, when to do them, and based on the completion of some tasks, tells other workers to finish their jobs. At the end of it all, Fly then takes the results and places them on the desk (target) of the boss.

(Concat is the only exception here, though it could be argued that concat is like merging the stack of results into a single manilla folder.)

Each worker is a fly-plugin who specializes in a task. This also means each office (project) is or could be different. The only aspect of an office that remains a same is the boss, the manager, and the need for workers.

With all that said, I think shell should be a (useful) plugin. All it has to do is wrap/expose co-exec or execa. Those two have already done all the heavy lifting.

[lukeed]

@hzlmn @jbucaran

Made a plugin that makes Oleh's example possible. Check out fly-exec 😄

("fly-shell" was taken 😞)

[hzlmn]

@lukeed neat work 🎉 , but plugin with exec name, will conflict with fly.exec, isn't it ?

Also, there is one "small" problem called Windows , that is why I mentioned shelljs.

As we have some amount of users on Windows(due to opened issue) , I was thinking about cross platform way of executing shell commands.

[devmondo]

yeah please please keep cross platform comparability.

after working with Fly, i removed Gulp completely out of my life, Fly is so beautiful, simple and how build system should be!

my whole company is moving all projects to use Fly, and all of us use Windows.

[lukeed]

@hzlmn You're right; I quickly renamed it from fly-shell to fly-exec without giving it a test run. It's not called fly-execa.

And I don't use Windows or regularly have a Windows machine available. I went with execa because it's yieldable, straightforward, and seemed to include Windows support. I know that you can also pass a shell path within your options object. It'll be passed to child_process.exec

TBH, I really don't know & haven't used Win in the last 5 years, so I am speaking from ignorance, but I'd be really surprised if it didn't work.

[lukeed]

@devmondo Happy to hear! Welcome aboard 🚂

[hzlmn]

@lukeed It will work, but what execa do is just normalize Node's spawn with cross-spawn. What I actually meant, is that I want Win users to be able to use something like sed or grep.

For example, I was forced to use Win when I worked with .Net guys, and even simple text transforming was a pain for me (cmder and cash trying to fix it now) and basically I want build tool to allow me to use shell tools in cross-platform manner, even when I on Win machine.

[lukeed]

@hzlmn Ah, ok.

Well, the one slight caveat I see to using Shell JS is that it requires Windows users to know the UNIX commands in order to run a shell command on their own system. They might not appreciate the roundabout method of interacting with their own machine, despite the added bonus of cross compatibility. (Especially more novice devs)

What if there were 2 plugins:

• fly-shell (.shell) for basic, command line interaction
• fly-shelljs (.shelljs) for cross-compatible, POSIX interaction

[hzlmn]

@lukeed IMO, I see your shell plugin as a core method (method's name for executing single task should be revisited) because ,basically, before reading code, I thought that current exec do what your shell plugin do. I think it's valuable to have ability to execute shell commands, without any additional plugins.

We are not forcing Win users to use nix shell cmds, we are just adding helper for nix users that work in foreign env for some reasons, to do this kind of things more natively. I know that porting all commands tricky but basic text transforming should works fine. This one could be a plugin.

[lukeed]

I thought that current exec do what your shell plugin do.

Yeah, I'm going to get rid of it (fly.start in #203)

I'll have to think about integrating it as a core API. I'm still mixed, leaning against it. I've gone a full year with Fly without the need to execute a shell command once, so I don't think others, or the majority, will need this included within each installation. For those who do, a minor plugin is a breeze to install.

@jbucaran ?

[jorgebucaran]

@lukeed Agreed.

[devmondo]

i may not be qualified to elaborate on this, but shell commands is something as @lukeed said is not used that often, and i believe it is not the responsibility of Fly to provide that, it should be a plugin or direct use of the required Lib. same like fly does not provide browser-sync, or babel compiling as built in, otherwise it would defeat the purpose of Fly being small easy and flexible.

in my experience, i wanted to use nodemon, babel, and browser-sync, and even though there are Fly plugins for this, i did not even need it, i just used them directly as if i would in any node.js application and man this was a breath of fresh air to escape from Gulp plugins, which you have to wait for developers to fix bug or upgrade to latest babel for example.

Fly workflow for me is awesome, it gives you the simple nature of NPM scripts with the power of Gulp as task runner, but with really minimal rules and open freedom to compose your code or use any Node.js package.

[jorgebucaran]

Fly workflow for me is awesome, it gives you the simple nature of NPM scripts with the power of Gulp as task runner, but with really minimal rules and open freedom to compose your code or use any Node.js package.

@devmondo I couldn't have put it better! Thanks.

[lukeed]

Update: My plugin was renamed to fly-shell. Thank you to @danny-andrews for relinquishing the name.

Pinging @hzlmn since you probably interested.

[lukeed]

Discussion seems to have dissipated. Closing since it appears as though we've come to an agreement.

Ping or reopen if this was not the case 👍