Closed yarikoptic closed 4 years ago
I think it should be the GNU one
Why do you prefer that one rather than moreutil's (which is the current target)?
I like joey and his tools and collections. But as far as I see it moreutils is a Unix/Linux specific collection of small tools and is less likely to be installed on a random box. Not sure if works on OSX at all. Can't login to our test OSX box (probably died), can't check if OSX comes with any "natively"... but I see that in conda world: moreutils is present only in bioconda and only for linux, while parallel for both linux and osx from generic conda-forge (I have not yet tried to install both on conda and see what happens ;) - probably 2nd one would beat the first one)
Originally observed in https://github.com/ReproNim/reproman/issues/511#issuecomment-634914499 on smaug:
and full stack would be (from a different run):
, while working locally.
It might lead to by problem running parallel
```shell (git)smaug:…im/reproman-master/docs/usecases/bids-fmriprep-workflow-NP/out1[master]git $> cat /home/yoh/.reproman/run-root/83d02dde-a0db-11ea-8df0-002590f97d84/.reproman/jobs/local/20200528-083028-15fd/stderr Unknown option: c $> cat /home/yoh/.reproman/run-root/83d02dde-a0db-11ea-8df0-002590f97d84/.reproman/jobs/local/20200528-083028-15fd/stdout Usage: parallel [options] [command [arguments]] < list_of_arguments parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))... cat ... | parallel --pipe [options] [command [arguments]] -j n Run n jobs in parallel -k Keep same order -X Multiple arguments with context replace --colsep regexp Split input on regexp for positional replacements {} {.} {/} {/.} {#} {%} {= perl code =} Replacement strings {3} {3.} {3/} {3/.} {=3 perl code =} Positional replacement strings With --plus: {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} = {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...} -S sshlogin Example: foo@server.example.com --slf .. Use ~/.parallel/sshloginfile as the list of sshlogins --trc {}.bar Shorthand for --transfer --return {}.bar --cleanup --onall Run the given command with argument on all sshlogins --nonall Run the given command with no arguments on all sshlogins --pipe Split stdin (standard input) to multiple jobs. --recend str Record end separator for --pipe. --recstart str Record start separator for --pipe. See 'man parallel' for details Academic tradition requires you to cite works you base your article on. When using programs that use GNU Parallel to process data for publication please cite: O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login: The USENIX Magazine, February 2011:42-47. ```and somehow then not having those references setup.
Note that above parallel is GNU's parallel (from
parallel
package on debian). Locally (where execution worked) I hadparallel
from moreutils.So I guess the issue is multifold
.status
to raise a dedicated exception to indicate that job tracking is "errored" so--follow
would just fail andjobs
would report some ERROR