Add possibility to add multiple volumes

[jhaezebr]

Other than the working directory, one might need to add more volumes to the job (eg: for reference data). This would require the following:

• be able to add more than one volume in the workflow config
• be able to define the path where to mount the volume?

[jagedn]

As a starting point I've working on this DSL

If only 1 volume is required, use the current implementation:

jobs {
        deleteOnCompletion = false
        volume = { type "host" name "scratchdir" }
}

if you want to use multiple volumes:

jobs {
        deleteOnCompletion = false
        volumes = {
            volume { type "host" name "scratchdir" },
            volume { type "host" name "scratchdir" path "/var/data" },
        }
    }

In this implementation volumes is a closure

or maybe we can use as a list without the volume keyword

jobs {
        deleteOnCompletion = false
        volumes = [
            { type "host" name "scratchdir" },
            { type "host" name "scratchdir" path "/var/data" },
        ]
    }

I prefer the closure implementation, because I like closures, but second option maybe is more expresive

What do you think @jhaezebr @abhi18av @matthdsm ?

[jhaezebr]

To me the second option seems cleanest, because there is no redundant 'volume' keyword?

[jagedn]

as an "extra ball" you'll be able to mount a volume multiple times in different paths

not sure is util but ... how knows

volumes = [
            { type "host" name "scratchdir" },
            { type "host" name "scratchdir" path "/var/data" },  // can mount same volume in different path
        ]

[jagedn]

Also you'll be able to "mix" both configurations

jobs {
        volume = { type "host" name "scratchdir" }

        volumes = [
            { type "host" name "one" path "/var/data1"},
            { type "host" name "two" path "/var/data2" }
        ]
    }

in this case, scratchdir will be mounted as workDir by default

Or you can specify which volume mount as workdir :

{ type "host" name "second" workdir true }