search

ReproNim / reproman

ReproMan (AKA NICEMAN, AKA ReproNim TRD3)
https://reproman.readthedocs.io
Other
24 stars 14 forks source link

run: local submitter fails (due mismatch of `parallel`?) but we fail to detect that and crash due to missing ref #522

Closed yarikoptic closed 4 years ago

yarikoptic commented 4 years ago

Originally observed in https://github.com/ReproNim/reproman/issues/511#issuecomment-634914499 on smaug:

2020-05-27 16:07:27,208 [ERROR] CommandError: command 'git cat-file -p refs/reproman/20200527-160657-0490:.reproman/jobs/local/20200527-160657-0490/status.0' failed with exitcode 128
| Failed to run 'git cat-file -p refs/reproman/20200527-160657-0490:.reproman/jobs/local/20200527-160657-0490/status.0' under '/home/yoh/.tmp/rm-tfbYoMv/HOME/.reproman/run-root/940832c2-a055-11ea-a8dd-002590f97d84'. Exit code=128. out= err=fatal: Not a valid object name refs/reproman/20200527-160657-0490:.reproman/jobs/local/20200527-160657-0490/status.0
|  [orchestrators.py:_execute_in_wdir:641] (OrchestratorError)

and full stack would be (from a different run):

(Pdb) bt
  /home/yoh/proj/repronim/reproman-master/venvs/dev3/bin/reproman(11)<module>()
-> load_entry_point('reproman', 'console_scripts', 'reproman')()
  /home/yoh/proj/repronim/reproman-master/reproman/cmdline/main.py(250)main()
-> ret = cmdlineargs.func(cmdlineargs)
  /home/yoh/proj/repronim/reproman-master/reproman/interface/base.py(287)call_from_parser()
-> return cls.__call__(**kwargs)
  /home/yoh/proj/repronim/reproman-master/reproman/interface/run.py(420)__call__()
-> orc.follow()
  /home/yoh/proj/repronim/reproman-master/reproman/support/jobs/orchestrators.py(367)follow()
-> .format(self.jobid, self.status, self.working_directory))
  /home/yoh/proj/repronim/reproman-master/reproman/support/jobs/orchestrators.py(637)status()
-> self.working_directory)))
> /home/yoh/proj/repronim/reproman-master/reproman/support/jobs/orchestrators.py(617)_execute_in_wdir()
-> str(exc) if err_msg is None else err_msg)

, while working locally.

It might lead to by problem running parallel ```shell (git)smaug:…im/reproman-master/docs/usecases/bids-fmriprep-workflow-NP/out1[master]git $> cat /home/yoh/.reproman/run-root/83d02dde-a0db-11ea-8df0-002590f97d84/.reproman/jobs/local/20200528-083028-15fd/stderr Unknown option: c $> cat /home/yoh/.reproman/run-root/83d02dde-a0db-11ea-8df0-002590f97d84/.reproman/jobs/local/20200528-083028-15fd/stdout Usage: parallel [options] [command [arguments]] < list_of_arguments parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))... cat ... | parallel --pipe [options] [command [arguments]] -j n Run n jobs in parallel -k Keep same order -X Multiple arguments with context replace --colsep regexp Split input on regexp for positional replacements {} {.} {/} {/.} {#} {%} {= perl code =} Replacement strings {3} {3.} {3/} {3/.} {=3 perl code =} Positional replacement strings With --plus: {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} = {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...} -S sshlogin Example: foo@server.example.com --slf .. Use ~/.parallel/sshloginfile as the list of sshlogins --trc {}.bar Shorthand for --transfer --return {}.bar --cleanup --onall Run the given command with argument on all sshlogins --nonall Run the given command with no arguments on all sshlogins --pipe Split stdin (standard input) to multiple jobs. --recend str Record end separator for --pipe. --recstart str Record start separator for --pipe. See 'man parallel' for details Academic tradition requires you to cite works you base your article on. When using programs that use GNU Parallel to process data for publication please cite: O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login: The USENIX Magazine, February 2011:42-47. ```

and somehow then not having those references setup.

Note that above parallel is GNU's parallel (from parallel package on debian). Locally (where execution worked) I had parallel from moreutils.

So I guess the issue is multifold

kyleam commented 4 years ago

I think it should be the GNU one

Why do you prefer that one rather than moreutil's (which is the current target)?

yarikoptic commented 4 years ago

I like joey and his tools and collections. But as far as I see it moreutils is a Unix/Linux specific collection of small tools and is less likely to be installed on a random box. Not sure if works on OSX at all. Can't login to our test OSX box (probably died), can't check if OSX comes with any "natively"... but I see that in conda world: moreutils is present only in bioconda and only for linux, while parallel for both linux and osx from generic conda-forge (I have not yet tried to install both on conda and see what happens ;) - probably 2nd one would beat the first one)