Timed out waiting for dataset to mount

[sirlp2]

When running the tests for flocker, I am getting the error Error response from daemon: error while mounting volume '/': Timed out waiting for dataset to mount.

In the logs I do not see any error's reported, but I do see this entry begin repeated over and over since the container failed to run.

06bc3bab-ace2-4f1a-984a-f7d81a4e6763 -> /3
2017-03-29 23:45:38.907647Z
  message_type: 'flocker:failure-retry:success'
  result: {'AppArmorProfile': '',
        |  'Args': ['-c',
        |           'echo hello > /data/file.txt'],
        |  'Config': {'AttachStderr': True,
        |             'AttachStdin': False,
        |             'AttachStdout': True,
        |             'Cmd': ['sh',
        |                     '-c',
        |                     'echo hello > /data/file.txt'],
        |             'CpuShares': 0,
        |             'Cpuset': '',
        |             'Domainname': '',
        |             'Entrypoint': None,
        |             'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'],
        |             'ExposedPorts': None,
        |             'Hostname': '55f75287f6fd',
        |             'Image': 'busybox',
        |             'Labels': {},
        |             'MacAddress': '',
        |             'Memory': 0,
        |             'MemorySwap': 0,
        |             'NetworkDisabled': False,
        |             'OnBuild': None,
        |             'OpenStdin': False,
        |             'StdinOnce': False,
        |             'Tty': False,
        |             'User': '',
        |             'VolumeDriver': 'flocker',
        |             'Volumes': None,
        |             'WorkingDir': ''},
        |  'Created': '2017-03-29T23:43:29.351199072Z',
        |  'Driver': 'overlay',
        |  'ExecIDs': None,
        |  'GraphDriver': {'Data': {'LowerDir': '/var/lib/docker/overlay/969023c9caac6a280d18465374638113a13c18f86e7f932529dd81e59155275a/root',
        |                           'MergedDir': '/var/lib/docker/overlay/340d2804eddbad7c22aaa63ddf376808b3797223f4ce5de29e1d4cdf6e4057e7/merged',
        |                           'UpperDir': '/var/lib/docker/overlay/340d2804eddbad7c22aaa63ddf376808b3797223f4ce5de29e1d4cdf6e4057e7/upper',
        |                           'WorkDir': '/var/lib/docker/overlay/340d2804eddbad7c22aaa63ddf376808b3797223f4ce5de29e1d4cdf6e4057e7/work'},
        |                  'Name': 'overlay'},
        |  'HostConfig': {'AutoRemove': False,
        |                 'Binds': ['apples:/data'],
        |                 'BlkioDeviceReadBps': None,
        |                 'BlkioDeviceReadIOps': None,
        |                 'BlkioDeviceWriteBps': None,
        |                 'BlkioDeviceWriteIOps': None,
        |                 'BlkioWeight': 0,
        |                 'BlkioWeightDevice': None,
        |                 'CapAdd': None,
        |                 'CapDrop': None,
        |                 'Cgroup': '',
        |                 'CgroupParent': '',
        |                 'ConsoleSize': [0, 0],
        |                 'ContainerIDFile': '',
        |                 'CpuCount': 0,
        |                 'CpuPercent': 0,
        |                 'CpuPeriod': 0,
        |                 'CpuQuota': 0,
        |                 'CpuRealtimePeriod': 0,
        |                 'CpuRealtimeRuntime': 0,
        |                 'CpuShares': 0,
        |                 'CpusetCpus': '',
        |                 'CpusetMems': '',
        |                 'Devices': [],
        |                 'DiskQuota': 0,
        |                 'Dns': [],
        |                 'DnsOptions': [],
        |                 'DnsSearch': [],
        |                 'ExtraHosts': None,
        |                 'GroupAdd': None,
        |                 'IOMaximumBandwidth': 0,
        |                 'IOMaximumIOps': 0,
        |                 'IpcMode': '',
        |                 'Isolation': '',
        |                 'KernelMemory': 0,
        |                 'Links': None,
        |                 'LogConfig': {'Config': {},
        |                               'Type': 'json-file'},
        |                 'Memory': 0,
        |                 'MemoryReservation': 0,
        |                 'MemorySwap': 0,
        |                 'MemorySwappiness': -1,
        |                 'NanoCpus': 0,
        |                 'NetworkMode': 'default',
        |                 'OomKillDisable': False,
        |                 'OomScoreAdj': 0,
        |                 'PidMode': '',
        |                 'PidsLimit': 0,
        |                 'PortBindings': {},
        |                 'Privileged': False,
        |                 'PublishAllPorts': False,
        |                 'ReadonlyRootfs': False,
        |                 'RestartPolicy': {'MaximumRetryCount': 0,
        |                                   'Name': 'no'},
        |                 'Runtime': 'runc',
        |                 'SecurityOpt': None,
        |                 'ShmSize': 67108864,
        |                 'UTSMode': '',
        |                 'Ulimits': None,
        |                 'UsernsMode': '',
        |                 'VolumeDriver': 'flocker',
        |                 'VolumesFrom': None},
        |  'HostnamePath': '/var/lib/docker/containers/55f75287f6fdfc7437dd8220c3f40438e179748996bfe557db48f62ebff74a21/hostname',
        |  'HostsPath': '/var/lib/docker/containers/55f75287f6fdfc7437dd8220c3f40438e179748996bfe557db48f62ebff74a21/hosts',
        |  'Id': '55f75287f6fdfc7437dd8220c3f40438e179748996bfe557db48f62ebff74a21',
        |  'Image': 'sha256:00f017a8c2a6e1fe2ffd05c281f27d069d2a99323a8cd514dd35f228ba26d2ff',
        |  'LogPath': '',
        |  'MountLabel': '',
        |  'Name': '/infallible_roentgen',
        |  'NetworkSettings': {'Bridge': '',
        |                      'EndpointID': '',
        |                      'Gateway': '',
        |                      'GlobalIPv6Address': '',
        |                      'GlobalIPv6PrefixLen': 0,
        |                      'HairpinMode': False,
        |                      'IPAddress': '',
        |                      'IPPrefixLen': 0,
        |                      'IPv6Gateway': '',
        |                      'LinkLocalIPv6Address': '',
        |                      'LinkLocalIPv6PrefixLen': 0,
        |                      'MacAddress': '',
        |                      'Ports': None,
        |                      'SandboxID': 'aa78d6cdb25d63fff4419ab6351b9663afea5797b8415286e6aed030dc8c241d',
        |                      'SandboxKey': '/var/run/docker/netns/aa78d6cdb25d',
        |                      'SecondaryIPAddresses': None,
        |                      'SecondaryIPv6Addresses': None},
        |  'Path': 'sh',
        |  'ProcessLabel': '',
        |  'ResolvConfPath': '/var/lib/docker/containers/55f75287f6fdfc7437dd8220c3f40438e179748996bfe557db48f62ebff74a21/resolv.conf',
        |  'RestartCount': 0,
        |  'State': {'Dead': False,
        |            'Error': "error while mounting volume '/': Timed out waiting for dataset to mount.",
        |            'ExitCode': 128,
        |            'FinishedAt': '0001-01-01T00:00:00Z',
        |            'OOMKilled': False,
        |            'Paused': False,
        |            'Pid': 0,
        |            'Restarting': False,
        |            'Running': False,
        |            'StartedAt': '0001-01-01T00:00:00Z',
        |            'Status': 'created'},
        |  'Volumes': {'/data': '/'},
        |  'VolumesRW': {'/data': True}}

[pdhamdhere]

@sirlp2 Have you tried vSphere Docker Volume Plugin Flocker driver will be deprecated and we advise you to give vSphere Docker Volume Plugin a try. Feel free to reach out to us @ containers@vmware.com if you have any questions.

[sirlp2]

I have looked at it but I specifically need support for Docker swarm service volumes. All I have seen in the documentation is Docker host specific volumes and have seen no support for swarm and movement of containers with their volumes among the swarm nodes.

[pdhamdhere]

@sirlp2 vSphere Docker Volumes are "global" in scope and fully compatible with Swarm. If Swarm fails over container/service to another node, vDVS will make volume is available on new node. Are you up for a quick call to discuss use case? Drop us a note @ email.

Good feedback on Documentation. We need to add section on Docker Swarm Interop. CC/ @tusharnt

[sirlp2]

I have sent an email with contact info. Appreciate the opportunity to discuss my use case.