common-workflow-language / cwltool

Common Workflow Language reference implementation
https://cwltool.readthedocs.io/
Apache License 2.0
335 stars 231 forks source link

Changing docker read-only=true #1395

Open mr-eyes opened 3 years ago

mr-eyes commented 3 years ago

I was wondering how to change the default read-only=true option to false. I am trying to write files inside the container without an expected exact output to be defined in the workflow.

I think this line is related https://github.com/common-workflow-language/cwltool/blob/ac60dc1df0c23e54ecee99bc0d989da410851d2e/cwltool/docker.py#L381

mr-c commented 3 years ago

Hello @mr-eyes and thank you for sharing your issue.

Can you provide the CWL description that needs this?

Often this can be fixed there by redirecting output to $(runtime.outdir) or $(runtime.tmpdir) via an additional command line parameter or setting an application specific environment variable.

While you can add --no-read-only to your cwltool invocation to skip --read-only=true being passed to Docker, we do not recommend that as other CWL runners do not have this behaviour and your CWL description will not be portable.

mr-eyes commented 3 years ago

Hi, @mr-c thanks for your quick response.

my CWL description

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
baseCommand: [make, test]

hints:
  DockerRequirement:
    dockerPull: "mabuelanin/genome-grist:tag01"

inputs:
  grist_directory:
    type: string
    inputBinding:
        prefix: --directory

outputs:
  grist_stdout:
    type: stdout
stdout: grist_stdout.txt

I am basically trying to run make test inside the docker container, which writes a lot of files/directories inside the container (Testing purposes).

Passing --no-read-only to the cwltool worked perfectly. But I could not use the $(runtime.outdir). Should I put that line under the outputBinding?

Thanks!

mr-c commented 3 years ago

@mr-eyes Ideally grist_directory should be of type: Directory (not type: string) and then staged into the working directory using https://www.commonwl.org/user_guide/15-staging/index.html