search

insitro / redun

Yet another redundant workflow engine
https://insitro.github.io/redun/
Apache License 2.0
517 stars 43 forks source link

Docker executor cannot recognize the volumes option. #88

Open danielecook opened 11 months ago

danielecook commented 11 months ago

I am trying to get a local setup running for debugging purposes. Data will be staged locally to reduce latency. Unfortunately, the volumes option for the docker executor does not appear to work.

Current config

[executors.docker]
type = docker
max_workers = 20
image = ####
scratch = /tmp/scratch
include_aws_env = False
volumes = [["/home", "/home"]]
interactive = False

The get_docker_executor_config does not retrieve volumes currently.

https://github.com/insitro/redun/blob/bcc337003923b21ef096c813a32c6af70e559129/redun/executors/docker.py#L38-L43

It may be helpful to throw an error for additional options specified here that cannot be used with the Docker executor.

Additionally, I wonder whether mounting the workflow working directory would be beneficial?

https://github.com/insitro/redun/blob/bcc337003923b21ef096c813a32c6af70e559129/redun/executors/docker.py#L278-L285