search

vmware-archive / vsphere-flocker-driver

vSphere Flocker driver
Other
24 stars 16 forks source link

vsphere_blockdevice.GetDevicePathFailure #36

Open commandantmarcos opened 7 years ago

commandantmarcos commented 7 years ago

First of all, sorry for my english ;)

I'm facing issues while creating a volume with this driver with flockerctl command (even with docker but for this fresh and third install i've tried to create it directly with flockerctl create command)

On first attempt, a volume was created with a pending status. After restarting flocker-dataset-agent, the volume has a detached status and was created on my datastore, but i had the following errors on flocker-dataset-agent log : 

2016-12-13 14:02:09.669674Z
  action_type: 'agent:blockdevice:register'
  action_status: 'failed'
  exception: 'twisted.internet.error.ConnectionLost'
  reason: 'Connection to the other side was lost in a non-clean fashion: Connection lost.'

7a5b8357-a755-48bb-b0f6-ffbbdae5df68 -> /1
2016-12-13 14:02:09.670130Z
  message_type: 'eliot:traceback'
  exception: 'twisted.internet.error.ConnectionLost'
  reason: 'Connection to the other side was lost in a non-clean fashion: Connection lost.'
  traceback: "Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
           |  "

9c0c0bdf-73f7-4b8d-ab81-e661e333a0d2 -> /2/6/3
2016-12-13 14:02:09.670597Z
  action_type: 'flocker:node:in_parallel'
  action_status: 'failed'
  exception: 'twisted.internet.defer.FirstError'
  reason: "FirstError[#0, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
        |  ]]"

9c0c0bdf-73f7-4b8d-ab81-e661e333a0d2 -> /2/7
2016-12-13 14:02:09.671062Z
  message_type: 'eliot:traceback'
  exception: 'twisted.internet.defer.FirstError'
  reason: "FirstError[#0, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
        |  ]]"
  traceback: "Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#0, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
           |  ]]
           |  "

47f27fea-4ab0-452f-a2d0-b6eb6739b3b6 -> /2/2/3
2016-12-13 14:02:25.281602Z
  action_type: 'flocker:agent:discovery'
  action_status: 'failed'
  exception: 'vsphere_flocker_plugin.vsphere_blockdevice.GetDevicePathFailure'
  reason: 'No device path found'

On second attempt, no volume created on datastore and i can read the following error on my log : 

dfd0f7c2-00c9-4ec0-b7ca-9790ac8f3724 -> /2/2/2/2/2/2/3
2016-12-13 15:08:40.135983Z
  action_type: 'flocker:agent:discovery'
  action_status: 'failed'
  exception: 'vsphere_flocker_plugin.vsphere_blockdevice.GetDevicePathFailure'
  reason: 'No device path found'

dfd0f7c2-00c9-4ec0-b7ca-9790ac8f3724 -> /2/2/2/2/2/3
2016-12-13 15:08:40.136837Z
  message_type: 'eliot:traceback'
  exception: 'vsphere_flocker_plugin.vsphere_blockdevice.GetDevicePathFailure'
  reason: 'No device path found'
  traceback: "Traceback: <class 'vsphere_flocker_plugin.vsphere_blockdevice.GetDevicePathFailure'>: No device path found
           |  /opt/flocker/local/lib/python2.7/site-packages/machinist/_logging.py:102:receive
           |  /opt/flocker/local/lib/python2.7/site-packages/machinist/_fsm.py:534:receive
           |  /opt/flocker/local/lib/python2.7/site-packages/machinist/_fsm.py:592:output
           |  /opt/flocker/local/lib/python2.7/site-packages/flocker/node/_loop.py:529:output_CONVERGE
           |  --- <exception caught here> ---
           |  /opt/flocker/local/lib/python2.7/site-packages/twisted/internet/defer.py:150:maybeDeferred
           |  /opt/flocker/local/lib/python2.7/site-packages/flocker/node/agents/blockdevice.py:1798:discover_state
           |  /opt/flocker/local/lib/python2.7/site-packages/flocker/node/agents/blockdevice.py:1345:_count_calls
           |  /opt/flocker/local/lib/python2.7/site-packages/flocker/node/agents/blockdevice.py:1764:_discover_raw_state
           |  /opt/flocker/local/lib/python2.7/site-packages/flocker/node/agents/blockdevice.py:2035:get_device_path
           |  /opt/flocker/local/lib/python2.7/site-packages/vsphere_flocker_plugin/vsphere_blockdevice.py:752:get_device_path
           |  "

Now i'm unable to create / remove volume with flocker, they are all on a pending status.

This is my config :

root@flockercontrol:~# /opt/flocker/bin/pip show pyvmomi

Metadata-Version: 2.0 Name: pyvmomi Version: 6.5

root@flockercontrol:~# /opt/flocker/bin/python --version Python 2.7.12

root@flockercontrol:~# /opt/flocker/bin/pip show vsphere-flocker-driver

Metadata-Version: 1.1 Name: vSphere-Flocker-driver Version: 1.0

Installed on Ubuntu 16.04. VCenter Version = 6.0.0, cluster installed on a OVH dedicated cloud.

My agent.yml : 

version: 1
control-service:
  hostname: "192.168.0.55"
dataset:
  backend: "vsphere_flocker_plugin"
  vc_ip: "X.XXX.XXX.XXX"
  username: "VCUser"
  password: "VCPassword"
  datacenter_name: "pcc-X-XXX-XXX-XXX_datacenterXXXX"
  datastore_name: "pcc-00XXXX"
  ssl_verify_cert: no
  ssl_key_file: ""
  ssl_cert_file: ""
  ssl_thumbprint: ""

Does anyone can help me? This issue seems to be similar with the #18. I've searched on all issues but 'im still unable to find a solution...

aliouba commented 7 years ago

Hello, Can you show "lsscsi -v" and/or "sudo fdisk -l" results?

commandantmarcos commented 7 years ago

Hello @aliouba

sovinty@flockercontrol:~$ lsscsi -v

[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0
  dir: /sys/bus/scsi/devices/1:0:0:0  [/sys/devices/pci0000:00/0000:00:07.1/ata2                                                                                        /host1/target1:0:0/1:0:0:0]
[2:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sda
  dir: /sys/bus/scsi/devices/2:0:0:0  [/sys/devices/pci0000:00/0000:00:10.0/host                                                                                        2/target2:0:0/2:0:0:0]
[2:0:1:0]    disk    VMware   Virtual disk     1.0   /dev/sdb
  dir: /sys/bus/scsi/devices/2:0:1:0  [/sys/devices/pci0000:00/0000:00:10.0/host                                                                                        2/target2:0:1/2:0:1:0]

and 

sudo fdisk -l
[sudo] Mot de passe de sovinty :
Disque /dev/ram0 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram1 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram2 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram3 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram4 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram5 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram6 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram7 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram8 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram9 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram10 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram11 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram12 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram13 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram14 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/ram15 : 64 MiB, 67108864 octets, 131072 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disque /dev/sda : 25 GiB, 26843545600 octets, 52428800 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x588ba7d4

Périphérique Amorçage   Start      Fin Secteurs  Size Id Type
/dev/sda1    *           2048   999423   997376  487M 83 Linux
/dev/sda2             1001470 52426751 51425282 24,5G  5 Étendue
/dev/sda5             1001472 52426751 51425280 24,5G 8e LVM Linux

Disque /dev/sdb : 4 GiB, 4294967296 octets, 8388608 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disque /dev/mapper/ubuntu--vg-root : 22,5 GiB, 24150802432 octets, 47169536 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disque /dev/mapper/ubuntu--vg-swap_1 : 2 GiB, 2143289344 octets, 4186112 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
aliouba commented 7 years ago

I mean "lsscsi -v" and/or "sudo fdisk -l" of your Agent nodes (or Agent node doing the task).

kerneltime commented 7 years ago

@commandantmarcos I am curious if you have given https://github.com/vmware/docker-volume-vsphere a try. Feel free to reach out containers@vmware.com or on gitter

commandantmarcos commented 7 years ago

@aliouba my control-service is also one of my two agent node, as mentioned in the Doc. I have exactly the same issue when trying to create a volume from my second node but If you want i'll send you lsscsi and fdisk results on tomorrow.

@kerneltime unfortunately, ovh dedicated cloud solutions have a bad restriction on VCenter server they rent: i can't ssh my cluster hosts, so i cant install any VIB package :'(