UnfullfillableDiskAffinities when trying to provision a disk.

tactical-drone commented 7 years ago

12s 12s 1 persistentvolume-controller Warning ProvisioningFailed Failed to provision volume with StorageClass "shared-vmfs": photon: Task 'c71d4695-0df1-4a04-987a-6765527fc91c' is in error state: {@step=={"sequence"=>"0","state"=>"ERROR","errors"=>[photon: { HTTP status: '0', code: 'UnfullfillableDiskAffinities', message: 'Check disk(s) affinity. In case of more than one disk affinities, disk flavors may be incompatible.', data: 'map[]' }],"warnings"=>[],"operation"=>"RESERVE_RESOURCE","startedTime"=>"1495206886000","queuedTime"=>"1495206886000","endTime"=>"1495206886000","options"=>map[50a3c40c-4d8e-4bd2-8763-f6c3cd296ce5:persistent-disk]}}

Spesifically this part: 'UnfullfillableDiskAffinities', message: 'Check disk(s) affinity. In case of more than one disk affinities, disk flavors may be incompatible

I am following this guide

All quotas are correctly setup because if they weren't it would not get to this stage. Kubernetes keeps on failing and then after a while photon disk list shows many disks in ERROR state. photon disk info does not reveal any clues.

I removed all flavor spam of service-shared-vmfs-persistent-disk as I thought this 1.2.0 flavor spam issue might be causing this. But not. Maybe it has to do with my 4x ESXi server it does not know where to put the disks? I donno. I need help here please.

tactical-drone commented 7 years ago

noob@photon-dev:~/src/osprey-solution/DevOps/kube/mailserver$ photon tenant show ctlab
Tenant ID:  fd3adcbd-a9e9-4450-b5e0-2415fcaffcb4
  Name:               ctlab
    SecurityGroups 1:
      Name:           photon.ctlab.com\Administrators
      Inherited:      true
    SecurityGroups 2:
      Name:           photon.ctlab.com\Administrators
      Inherited:      true
    Limits:
      ephemeral-disk    200 COUNT
      ephemeral-disk.capacity   2000    GB
      persistent-disk   200 COUNT
      persistent-disk.capacity  8000    GB
      storage.LOCAL_VMFS    200 COUNT
      vm.memory 1200    GB
      storage.NFS   200 COUNT
      storage.SHARED_VMFS   200 COUNT
      storage.VSAN  200 COUNT
      vm    200 COUNT
      vm.cpu    144 COUNT
    Usage:
      persistent-disk   100 COUNT
      persistent-disk.capacity  4000    GB
      storage.LOCAL_VMFS    0   COUNT
      vm.memory 600 GB
      ephemeral-disk    100 COUNT
      ephemeral-disk.capacity   2000    GB
      storage.VSAN  0   COUNT
      vm    100 COUNT
      vm.cpu    72  COUNT
      storage.NFS   0   COUNT
      storage.SHARED_VMFS   100 COUNT

noob@photon-dev:~/src/osprey-solution/DevOps/kube/mailserver$ photon project quota show a86a0f3e-c74d-4474-9005-6ef9b91b3671
    Limits:
      sdn.floatingip.size       0          COUNT
      ephemeral-disk            100        COUNT
      persistent-disk           100        COUNT
      vm.cpu                    72         COUNT
      vm                        100        COUNT
      persistent-disk.capacity  4000       GB
      storage.SHARED_VMFS       100        COUNT
      vm.memory                 614400     MB
      ephemeral-disk.capacity   2.048e+06  MB
    Usage:
      sdn.floatingip.size       0       COUNT
      ephemeral-disk            9       COUNT
      persistent-disk           0       COUNT
      vm.cpu                    55      COUNT
      vm                        6       COUNT
      persistent-disk.capacity  0       GB
      storage.SHARED_VMFS       0       COUNT
      vm.memory                 110592  MB
      ephemeral-disk.capacity   359424  MB

noob@photon-dev:~/src/osprey-solution/DevOps/kube/mailserver$ photon disk create
Disk name: test
Disk Flavor: service-shared-vmfs-persistent-disk
Disk capacity in GB: 1
Creating disk: test (service-shared-vmfs-persistent-disk)
Tenant: ctlab, project: dev
Are you sure [y/n]? y
2017/05/22 15:00:11 Error: photon: Task '9d4c6207-465f-4915-8db5-b8a10ebc18ef' is in error state: {@step=={"sequence"=>"0","state"=>"ERROR","errors"=>[photon: { HTTP status: '0', code: 'UnfullfillableDiskAffinities', message: 'Check disk(s) affinity. In case of more than one disk affinities, disk flavors may be incompatible.', data: 'map[]' }],"warnings"=>[],"operation"=>"RESERVE_RESOURCE","startedTime"=>"1495457889000","queuedTime"=>"1495457889000","endTime"=>"1495457889000","options"=>map[734201da-5a1c-4b3a-8ab3-51fc5c685a70:persistent-disk]}}
API Errors: [photon: { HTTP status: '0', code: 'UnfullfillableDiskAffinities', message: 'Check disk(s) affinity. In case of more than one disk affinities, disk flavors may be incompatible.', data: 'map[]' }]

photon disk show 734201da-5a1c-4b3a-8ab3-51fc5c685a70
Disk ID:  734201da-5a1c-4b3a-8ab3-51fc5c685a70
  Name:        test
  Kind:        persistent-disk
  Flavor:      service-shared-vmfs-persistent-disk
  CapacityGB:  1
  State:       ERROR
  Datastore:   
  Tags:        []
  VMs:         []

noob@photon-dev:~/src/osprey-solution/DevOps/kube/mailserver$ photon flavor show edeb3fdd-2d95-412c-9799-26ddb39e6183
Flavor ID: edeb3fdd-2d95-412c-9799-26ddb39e6183
  Name:  service-shared-vmfs-persistent-disk
  Kind:  persistent-disk
  Cost:  [storage.SHARED_VMFS 1 COUNT]
  State: READY

Could it be because I have sas & sata vmware storage datastores?

tactical-drone commented 7 years ago

Ok I see my mistake,

noob@photon-dev:~/src/osprey-solution/DevOps/kube/mailserver$ photon datastore list
ID                                   Type        Tags
58dc020a-8263054c-4590-b8ac6f7efbf6  LOCAL_VMFS  LOCAL_VMFS
58dd1e46-a27215a0-3244-0024e83e0daf  LOCAL_VMFS  LOCAL_VMFS
58e75cbe-b473b3d7-f209-0024e83e0daf  LOCAL_VMFS  LOCAL_VMFS
58e7743e-66c3605a-a324-b8ac6f7efbf6  LOCAL_VMFS  LOCAL_VMFS
58ec94cf-b98470c9-dfe7-0026b974422f  LOCAL_VMFS  LOCAL_VMFS
58ec94e0-faa87091-db80-0026b974422f  LOCAL_VMFS  LOCAL_VMFS
58ecb98b-47440a42-1286-b8ac6f7f33e2  LOCAL_VMFS  LOCAL_VMFS

Total: 7

I dont have any SHARED_VMFS volumes. Switching to local_vmfs seems to get rid of that error. It works now.

vmware-archive / photon-controller

UnfullfillableDiskAffinities when trying to provision a disk. #123