reana-client --file foo/reana.yml does not work (relative paths)

[lukasheinrich]

it would be nice if one could submit a workflow from a different pwd but right now it seems due to the relative paths declared in reana.yml

reana-client run --name recast-pmsssm-1234 --file reana-demo-atlas-recast/reana.yaml

gives

urllib.error.URLError: <urlopen error [Errno 2] No such file or directory: '/some/path/../..//workflow/workflow.yml'>
==> ERROR: Cannot create workflow recast-pmsssm-1234: 

Lukas

[tiborsimko]

(1) Are you trying to put reana.yaml inside some subdirectory of the analysis main tree, while being located there? That might work, for example:

$ reana-client validate -f docs/reana.yaml                                                                         
==> Verifying REANA specification file... /home/tibor/private/project/reana/src/reana-demo-bsm-search/docs/reana.yaml
  -> SUCCESS: Valid REANA specification file.
==> Verifying REANA specification parameters... 
  -> SUCCESS: REANA specification parameters appear valid.
==> Verifying workflow parameters and commands... 
  -> WARNING: Yadage parameter "workspace" found on step "makews" is not defined in input parameters.
==> Verifying dangerous workflow operations... 
  -> SUCCESS: Workflow operations appear valid.

(2) Are you trying to call reana-client from whatever directory that is located outside the analysis main tree? (Judging based on -f reana-demo-atlas-recast/reana.yaml.) That might lead to some troublesome situations, e.g. analysis folder in an analysis folder... We always assumed people would cd there first, as for git commands for example. Would it be possible to simply do cd dir && reana-client run ...?

(3) BTW one thing I see not working is launching reana-client from within subdirectory:

$ reana-demo-bsm-search/code
$ reana-client validate -f ../reana.yaml 
...
==> ERROR: Something went wrong when trying to validate ...

So if we touch the problem of launching reana-client command from non-root directories, we may look at improving this as well, so that one could use reana-client as one uses git wherever in the source code tree...

[lukasheinrich]

I'm looking at (2) (or (3) which for me are basically the same) - the use acse is submitting many different workflows

reana-client validate -f analyis_one/reana.yml reana-client validate -f analyis_two/reana.yml reana-client validate -f analyis_three/reana.yml

without having to jump directories

[tiborsimko]

For validating or submitting, wouldn't a tiny outer shell one-liner be sufficient?

$ for analysis in $(ls -d analysis_*); do cd $analysis; reana-client validate; cd ..; done

Regarding (2) or (3), I see a difference. E.g. if these analyses are governed by git source code management, you cannot do git status without jumping into directories... So implementing (3) but not (2) would akin to making REANA client equivalent to Git client, as it were, which would make the behaviour more understandable to users by close analogy with the Git client. Would it make sense?

(E.g. for detecting the last commit in each analysis_one repository, you would have to jump there to be able to do git log -1 --format=oneline, so it could use the same shell one-liner?)

[lukasheinrich]

I think generally it's a nice property for CLIs to not depend on where the $PWD is. Could reana-client not accept the "base directory" as an option (with a sensible default) ? From a user story perspective I think it'd be nice to also include this "batch handling" of multiple analyses as a use-case