Configure docker executor volumes from redun.ini

[aksarkar]

I am using redun to run a workflow manually on a single EC2 instance (via ssh and manual invocation, not Batch) using a Docker container to ensure reproducibility.

Currently, I need to specify the volumes to be made available to the container for each task, for example

import glob
import redun

@redun.task(volumes=[('foo', 'foo')])
def my_task():
  return glob.glob('foo/*')

otherwise, I will get FileNotFoundError when trying to read/write from the volume.

This does not appear to be configurable in redun.ini, since including volumes there and removing it from redun.task() does not work

[executors.default]
type = docker
image = my-image:latest
include_aws_env = false
scratch = /mnt/fsx/scratch/asarkar/
volumes = [("foo", "foo")]

It would be helpful to be able to configure volumes in redun.ini.

redun version 0.16.1

[mattrasmus]

Thanks @aksarkar for posting this issue. Adding a volumes configuration option to the Docker executor config in redun.ini sounds like a good idea and should be straightforward to implement. Thanks for the suggestion.

[mattrasmus]

We have now added a volumes config option to the redun.ini config file. Thanks for posting this issue.

https://insitro.github.io/redun/config.html#volumes