ga4gh / task-execution-schemas

Apache License 2.0
80 stars 27 forks source link

Clarification required: executor command line variable interpolation #164

Open markjschreiber opened 2 years ago

markjschreiber commented 2 years ago

In the spec it appears that inputs are only allowed to be of the form:

{

    "url": "s3://my-object-store/file1",
    "path": "/data/file1"

}

Implying inputs must only be files.

Presumably non file variables required by the executors need to be provided either in file form or as environment variables for the appropriate executor?

If it is the former, is there a convention to be followed - like some kind of inputs.json standard?

If it is the latter, is there an assumed SHELL or simply the default shell of the container?

uniqueg commented 2 weeks ago

I think the executors model includes a field for specifying env vars to be defined in the container. Is that what you mean?

I don't think there's an assumed shell in the container - good point!

markjschreiber commented 2 weeks ago

I thinking of things like WDL tasks which allow inputs like Int, String, Float, Struct etc as well as File. The spec seemed to only allow URL and Path (effectively File).

uniqueg commented 2 weeks ago

I think the idea is that the TES client (e.g., the workflow engine) should provide any non-file inputs as part of the command line. In fact, that is expected for files as well. The only reason tesInput and tesOutput are different is because files somehow need to be staged into/out of the executor environment.

What do you think @kellrott?