search

reanahub / reana-client

REANA command-line client
http://reana-client.readthedocs.io/
MIT License
10 stars 46 forks source link

allow passing workflow parameters as a file (separate from reana.yml) #492

Open lukasheinrich opened 3 years ago

lukasheinrich commented 3 years ago

via -p/--parameters I can do e.g.

reana-client run --name recast-pmsssm-1234 --file reana-demo-atlas-recast/reana.yaml -parameter foo=bar

but sometimes workflows have nested parameters / many parameters and listing them individually might be difficult. Being able to pass them via a file (keeping the rest of reana.yml fixed) would be good

reana-client run --name recast-pmsssm-1234 --file reana-demo-atlas-recast/reana.yaml --parameter-file pars.json

is that possible?

mvidalgarcia commented 3 years ago

CWL supports setting an input file in the reana.yaml. E.g. reana-demo-cdci-crab-pulsar-integral-verification. Would that fulfil your need? I guess we can easily support it for Yadage workflows here.

lukasheinrich commented 3 years ago

is that a parameter names input with value inputs.yaml i.e. equivalent to -p inputs=inputs.yaml or JSON {"inputs":inputs.yaml or does this have special semanticcs within CWL?

mvidalgarcia commented 3 years ago

I think it must be set inside the reana.yaml at the moment (so no -p inputs=inputs.yaml yet), and as far as I can see it's only supported for CWL workflows.

lukasheinrich commented 3 years ago

it seems like this could be a source of confusion (e.g. I would have def. expected -p inputs=inputs.yaml equivalence) .. would adding a parameter_file entry in reana.yml or similar be feasible?

mvidalgarcia commented 3 years ago

it seems like this could be a source of confusion (e.g. I would have def. expected -p inputs=inputs.yaml equivalence) ..

I agree.. it looks like this feature was asked for CWL workflows and remained there. In addition, the translation of parameters from the yaml file to the workflow spec is happening at a creation time, and one can only pass runtime parameters -p at start time. So probably that why this equivalence you mention does not exist...

Said that, in my opinion, it makes total sense to add a parameter file option to reana-client start, and consequently to reana-client run.