shenwei356 / rush

A cross-platform command-line tool for executing jobs in parallel
https://github.com/shenwei356/rush
MIT License
846 stars 63 forks source link

Unable to run executable within a folder #50

Closed danielsamuels closed 10 months ago

danielsamuels commented 10 months ago

Prerequisites

Describe your issue

I'm trying to run a series of scripts in parallel as a Git pre-push script (code linting, etc), but I'm having trouble getting anything to run.

/c/dev/bash-scripts/binaries/rush --verbose -I -D ';' ' {} ' <<- "DOC"
    venv/scripts/python manage.py makemigrations --check --dry-run;
DOC

Gives this output:

+ /c/dev/bash-scripts/binaries/rush --verbose -I -D ';' ' {} '
09:55:59.801 [INFO] start cmd #1: venv/scripts/python manage.py makemigrations --check --dry-run
(1/1/1): 'venv' is not recognized as an internal or external command,
(1/1/2): operable program or batch file.
09:55:59.851 [INFO] finish cmd #1 in 49.7856ms: venv/scripts/python manage.py makemigrations --check --dry-run
09:55:59.851 [ERRO] wait cmd #1: venv/scripts/python manage.py makemigrations --check --dry-run : exit status 1

I can, of course, run this command by itself without issue:

$ venv/scripts/python manage.py makemigrations --check --dry-run
No changes detected

What am I doing wrong here?

shenwei356 commented 10 months ago

I tried to reproduce this, but it works fine.

$  mkdir copy bin
$ cp /bin/echo bin/echo1
$ rush  -D ';' ' {} ' <<- "DOC"
    bin/echo1 $PWD;
DOC
/home/shenwei/tmp/rush

Also tested with wrapping the command with sh -c

$ rush  -D ';' ' {} ' <<- "DOC"
    sh -c "bin/echo1 $PWD";
DOC
/home/shenwei/tmp/rush

$ rush  -D ';' 'sh -c "{}" ' <<- "DOC"
    bin/echo1 $PWD;
DOC
/home/shenwei/tmp/rush

I've never tried with here doc. If it's me, I should do some thing like:

fd -H -t d '^.git$' \
      | rush -v 'd={//}' \
             'echo repo: {d}; \
              echo  do something in {d}'

Where fd -H -t d '^.git$' is for searching directories containing a .git directory.

danielsamuels commented 10 months ago

Strange, it's pretty consistent in not working for me, even with different approaches:

echo '
venv/scripts/python manage.py makemigrations --check --dry-run;
' | /c/dev/bash-scripts/binaries/rush --immediate-output -D ';' -T b --verbose -q ' {} '
+ echo '
venv/scripts/python manage.py makemigrations --check --dry-run;
'
+ /c/dev/bash-scripts/binaries/rush --immediate-output -D ';' -T b --verbose -q ' {} '
11:32:44.588 [INFO] start cmd #1: venv/scripts/python manage.py makemigrations --check --dry-run
(1/1/1): 'venv' is not recognized as an internal or external command,
(1/1/2): operable program or batch file.
11:32:44.630 [INFO] finish cmd #1 in 41.0876ms: venv/scripts/python manage.py makemigrations --check --dry-run
11:32:44.630 [ERRO] wait cmd #1: venv/scripts/python manage.py makemigrations --check --dry-run : exit status 1

The sh method is interesting, as it does get me a little further:

/c/dev/bash-scripts/binaries/rush --verbose -I -D ';' ' {} ' <<- "DOC"
    sh -c "venv/scripts/python manage.py makemigrations --check --dry-run";
DOC
+ /c/dev/bash-scripts/binaries/rush --verbose -I -D ';' ' {} '
11:36:13.794 [INFO] start cmd #1: sh -c "venv/scripts/python manage.py makemigrations --check --dry-run"
(1/1/1): No changes detected
11:36:16.587 [INFO] finish cmd #1 in 2.7926444s: sh -c "venv/scripts/python manage.py makemigrations --check --dry-run" : exit status 0
+ exit 1

Until I then try to do something like this:

/c/dev/bash-scripts/binaries/rush --verbose -I -D ';' ' {} ' <<- "DOC"
    sh -c "venv/scripts/python manage.py makemigrations --check --dry-run";
    sh -c "yarn run translate:check | grep '\[error]' | exit $(wc -l)";
DOC

When it then exits the 2nd task without doing anything (it should take ~11s):

+ /c/dev/bash-scripts/binaries/rush --verbose -I -D ';' ' {} '
11:37:04.023 [INFO] start cmd #2:
    sh -c "yarn run translate:check | grep '\[error]' | exit $(wc -l)"
11:37:04.023 [INFO] start cmd #1: sh -c "venv/scripts/python manage.py makemigrations --check --dry-run"
11:37:04.063 [INFO] finish cmd #2 in 40.5382ms:
    sh -c "yarn run translate:check | grep '\[error]' | exit $(wc -l)" : exit status 0
(1/1/1): No changes detected
11:37:06.821 [INFO] finish cmd #1 in 2.7988391s: sh -c "venv/scripts/python manage.py makemigrations --check --dry-run" : exit status 0
+ exit 1
shenwei356 commented 10 months ago

How about remove -D ';', the default value is \n. Also use -T b to trim white space (" \t\r\n") in input.

$ rush --verbose <<- "DOC"
> /bin/echo1 abc;
> /bin/echo1 123
> DOC
19:47:02.524 [INFO] start cmd #2: echo "/bin/echo1 123"
19:47:02.524 [INFO] start cmd #1: echo "/bin/echo1 abc;"
19:47:02.528 [INFO] finish cmd #2 in 4.253549ms: echo "/bin/echo1 123": exit status 0
/bin/echo1 123
19:47:02.529 [INFO] finish cmd #1 in 4.359987ms: echo "/bin/echo1 abc;": exit status 0
/bin/echo1 ABC;
danielsamuels commented 10 months ago

Same as the original problem:

/c/dev/bash-scripts/binaries/rush --verbose -I -T b ' {} ' <<- "DOC"
    venv/scripts/python manage.py makemigrations --check --dry-run;
    yarn run translate:check | grep '\[error]' | exit $(wc -l);
DOC
+ /c/dev/bash-scripts/binaries/rush --verbose -I -T b ' {} '
14:42:09.963 [INFO] start cmd #2: yarn run translate:check | grep '\[error]' | exit $(wc -l);
14:42:09.963 [INFO] start cmd #1: venv/scripts/python manage.py makemigrations --check --dry-run;
(1/1/1): 'venv' is not recognized as an internal or external command,
(1/1/2): operable program or batch file.
14:42:10.019 [INFO] finish cmd #1 in 56.66ms: venv/scripts/python manage.py makemigrations --check --dry-run;
14:42:10.019 [ERRO] wait cmd #1: venv/scripts/python manage.py makemigrations --check --dry-run; : exit status 1
14:42:21.262 [INFO] finish cmd #2 in 11.2992515s: yarn run translate:check | grep '\[error]' | exit $(wc -l); : exit status 0
shenwei356 commented 10 months ago

Are you using Windows? cygwin? or git for Windows? The error message seems to be only shown in Windows.

The way of executing commands in Windows is different from that in *nix.

danielsamuels commented 10 months ago

Yes, I'm using Git Bash on Windows

danielsamuels commented 10 months ago

Ah, I see what's happening then. Rush spawns a new shell using cmd, which means that the path separators change. Changing to a format that cmd understands, it works:

/c/dev/bash-scripts/binaries/rush --verbose -I -T b ' {} ' <<- "DOC"
    venv\\scripts\\python manage.py makemigrations --check --dry-run
    yarn run translate:check | grep '\[error]' | exit $(wc -l)
DOC
+ /c/dev/bash-scripts/binaries/rush --verbose -I -T b ' {} '
15:54:45.751 [INFO] start cmd #2: yarn run translate:check | grep '\[error]' | exit $(wc -l)
15:54:45.751 [INFO] start cmd #1: venv\\scripts\\python manage.py makemigrations --check --dry-run
(1/1/1): No changes detected
15:54:48.759 [INFO] finish cmd #1 in 3.0076311s: venv\\scripts\\python manage.py makemigrations --check --dry-run : exit status 0
15:54:56.750 [INFO] finish cmd #2 in 10.9990875s: yarn run translate:check | grep '\[error]' | exit $(wc -l) : exit status 0
shenwei356 commented 10 months ago

Right, I'm saying that too, a single \ is enough.

danielsamuels commented 10 months ago

Yes - you're right, one is enough. Thank you, I think that's enough for me to get somewhere now.