[Compose file] allow names for non-external volumes

dnephin commented 7 years ago

Proposed syntax:


volumes:
  data:
    name: something
    driver: drivername

This will fix a bunch of issues:

Probably others I haven't found yet.

We should also deprecate external: name: foo (external: bool would remain undeprecated).

cc @shin- What do you think?

Also, should we go ahead and do this with secrets, configs and networks as well, to keep them consistent?

dnephin commented 7 years ago

cc @vdemeester as well

shin- commented 7 years ago

Seems reasonable. external is ignored in docker stack anyway, right?

xask00 commented 7 years ago

In reference to https://github.com/moby/moby/issues/30770

Will this change make templated names for volumes possible in the compose YML file used by docker stack deploy like below -

volumes:
  data:
    name: {{.Service.Name}}-{{.Task.Slot}}-vol
    driver: cloudstor:aws

equivalent of the command line -

docker service create \
  --replicas 5 \
  --name ping2 \
  --mount type=volume,volume-driver=cloudstor:aws,source={{.Service.Name}}-{{.Task.Slot}}-vol,destination=/mydata \
  alpine ping docker.com

dnephin commented 7 years ago

Yes, that is one of the problems that will be fixed by this change.

xask00 commented 7 years ago

@dnephin thanks for confirming

eliaoggian commented 7 years ago

I Think this fix will be great! When will this be implemented?

Thanks!

shouze commented 7 years ago

@eliaoggian yes see https://github.com/docker/cli/pull/306 need to wait for 3.4 version of the schema with some other changes. I guess we can't expect a GA until 17.09 @dnephin right?

vieux commented 7 years ago

@dnephin @thaJeztah should this one be closed ?

dnephin commented 7 years ago

Yes, this issue is closed, fixed by #306

alculquicondor commented 7 years ago

I'm trying to use version 3.4 but it doesn't seem to work.

The relevant configuration is:

services:
    hdfsdata:                                                                                                                                                                                                                           
        image: quay.io/geomesa/hdfs:geomesa-1.3.2-accumulo-1.8.1                                                                                                                                                                        
        command: data                                                                                                                                                                                                                   
        depends_on:                                                                                                                                                                                                                     
            - hdfsname                                                                                                                                                                                                                  
        environment:                                                                                                                                                                                                                    
            - HADOOP_MASTER_ADDRESS=hdfsname                                                                                                                                                                                            
        volumes:                                                                                                                                                                                                                        
            - hdfsdata:/data/hdfs                                                                                                                                                                                                       
        deploy:                                                                                                                                                                                                                         
            mode: global                                                                                                                                                                                                                
            endpoint_mode: dnsrr

volumes:
    hdfsdata:
        name: '{{.Service.Name}}-{{.Task.Slot}}'
        driver: cloudstor:aws
        driver_opts:
            backing: relocatable
            size: 20

However, only a single volume is created with the task.slot from the container in the manager and all tasks attempt to attatch to it. Am I doing something wrong?

shouze commented 7 years ago

It depends if the docker release you run on aws, which one are you running?

alculquicondor commented 7 years ago

Here

~ $ docker version
Client:
 Version:      17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:39:28 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.09.0-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:45:38 2017
 OS/Arch:      linux/amd64
 Experimental: true

docker / cli

[Compose file] allow names for non-external volumes #274