NetApp / netappdvp

A Docker volume plugin for NetApp storage
96 stars 33 forks source link

E-Series: MediaType on volume creation defaults to hdd even if disk pool only contains ssds #108

Closed Stephen001 closed 6 years ago

Stephen001 commented 6 years ago

I am attempting to install trident against an EF540, that contains only SSDs. During installation, trident-ephemeral is trying to create a volume 'trident', which is failing as the requested MediaType is hdd, but the backend is only offering ssd.

Installer logs:

ephemeral log:
time="2017-11-10T17:32:59Z" level=info msg="Running Trident storage orchestrator." build_time="Wed Oct 18 19:55:39 UTC 2017" version=17.10.0
time="2017-11-10T17:32:59Z" level=info msg="Starting REST interface on :8000"
time="2017-11-10T17:32:59Z" level=warning msg="Unable to find key Backends.  Continuing bootstrap, but consider checking integrity if Trident installation is not new."
time="2017-11-10T17:32:59Z" level=warning msg="Unable to find key StorageClasses.  Continuing bootstrap, but consider checking integrity if Trident installation is not new."
time="2017-11-10T17:32:59Z" level=warning msg="Unable to find key Volumes.  Continuing bootstrap, but consider checking integrity if Trident installation is not new."
time="2017-11-10T17:32:59Z" level=warning msg="Couldn't retrieve volume transaction logs: Unable to find key"
time="2017-11-10T17:32:59Z" level=info msg="trident bootstrapped successfully."
time="2017-11-10T17:33:00Z" level=info msg="API server REST call." duration=77.292µs method=GET route=ListBackends uri="/trident/v1/backend"
time="2017-11-10T17:33:00Z" level=debug msg="Setting default volume size." size=1G
time="2017-11-10T17:33:00Z" level=debug msg=">>>> Initialize" Method=Initialize Type=ESeriesStorageDriver
time="2017-11-10T17:33:00Z" level=debug msg="Reparsed into ESeriesStorageDriverConfig" DebugTraceFlags=map[] DisableDelete=false StorageDriverName=eseries-iscsi StoragePrefix=trident Version=1
time="2017-11-10T17:33:00Z" level=debug msg="Connected to Web Services Proxy." AlreadyRegistered=true ArrayID=CHAN01-G20-EF540
time="2017-11-10T17:33:00Z" level=debug msg="Read controllers." Count=2
time="2017-11-10T17:33:00Z" level=debug msg="Read serial numbers." Count=2 SerialNumbers=[SQ35100877 SQ34500226]
time="2017-11-10T17:33:00Z" level=info msg="Controller serial numbers." serialNumbers=[SQ35100877 SQ34500226]
time="2017-11-10T17:33:00Z" level=debug msg="<<<< Initialize" Method=Initialize Type=ESeriesStorageDriver
time="2017-11-10T17:33:00Z" level=debug msg="Set default E-series HostGroup to trident"
time="2017-11-10T17:33:00Z" level=warning msg="Please ensure all relevant hosts are added to Host Group trident." controller=10.209.114.4 driver=eseries-iscsi hostGroup=trident
time="2017-11-10T17:33:00Z" level=debug msg="Considering pool." Name="Disk_Pool_1" Pool={false 60080E50003249B8000000A65326D473 0400000060080E50003249B8000000A65326D473 Disk_Pool_1 2516850835456 ssd}
time="2017-11-10T17:33:00Z" level=debug msg="EseriesStorageDriver#GetStorageBackendSpecs : Added pool for E-series backend." attributes="map[backendType:{Offers: eseries-iscsi} media:{Offers: ssd} snapshots:{Offer:  false} encryption:{Offer:  false} provisioningType:{Offers: thick}]" backend="eseries_10.209.1114.6" pool="Disk_Pool_1"
time="2017-11-10T17:33:00Z" level=debug msg="Adding backend." backend="eseries_10.209.1114.6" backendUpdate=false
time="2017-11-10T17:33:00Z" level=debug msg="EseriesStorageDriver:StoreConfig"
time="2017-11-10T17:33:00Z" level=debug msg="Sanitizing common config." name=eseries-iscsi
time="2017-11-10T17:33:00Z" level=debug msg="Setting nil raw storage prefix to contain an empty set." name=eseries-iscsi
time="2017-11-10T17:33:00Z" level=info msg="Newly added backend satisfies no storage classes." backend="eseries_10.209.1114.6"
time="2017-11-10T17:33:00Z" level=debug msg="EseriesStorageDriver:GetExternalConfig"
time="2017-11-10T17:33:00Z" level=debug msg="Sanitizing common config." name=eseries-iscsi
time="2017-11-10T17:33:00Z" level=info msg="Added a new backend." backend="eseries_10.209.1114.6" handler=AddBackend
time="2017-11-10T17:33:00Z" level=info msg="API server REST call." duration=154.046763ms method=POST route=AddBackend uri="/trident/v1/backend"
time="2017-11-10T17:33:00Z" level=debug msg="EseriesStorageDriver:GetExternalConfig"
time="2017-11-10T17:33:00Z" level=debug msg="Sanitizing common config." name=eseries-iscsi
time="2017-11-10T17:33:00Z" level=info msg="API server REST call." duration=172.337µs method=GET route=GetBackend uri="/trident/v1/backend/eseries_10.209.1114.6"
time="2017-11-10T17:33:00Z" level=info msg="Storage class satisfied by 1 storage pools." storageClass=trident-basic
time="2017-11-10T17:33:00Z" level=debug msg="Creating new storage pool list for backend." Backend="eseries_10.209.1114.6" Method=ConstructExternal pool="Disk_Pool_1" storageClass=trident-basic
time="2017-11-10T17:33:00Z" level=info msg="Added a new storage class." handler=AddStorageClass storageClass=trident-basic
time="2017-11-10T17:33:00Z" level=info msg="API server REST call." duration=145.929µs method=POST route=AddStorageClass uri="/trident/v1/storageclass"
time="2017-11-10T17:33:00Z" level=debug msg="Looking through 1 storage pools." volume=trident
time="2017-11-10T17:33:00Z" level=debug msg="Attempting volume create." size=2000000000 storagePool="Disk_Pool_1" volConfig.StorageClass=trident-basic
time="2017-11-10T17:33:00Z" level=debug msg="EseriesStorageDriver#GetInternalVolumeName : Created Base64 UUID for E-series volume name." Base64=lmEp3sxnTj2TX1S4gDz8hA Name=trident UUID=966129de-cc67-4e3d-935f-54b8803cfc84
time="2017-11-10T17:33:00Z" level=debug msg="EseriesStorageDriver#GetVolumeOpts" opts=map[pool:Disk_Pool_1] pool=&{Disk_Pool_1 [trident-basic] map[] 0xc420189260 map[media:{Offers: ssd} snapshots:{Offer:  false} encryption:{Offer:  false} provisioningType:{Offers: thick} backendType:{Offers: eseries-iscsi}]} requests=map[] volConfig=&{1 trident lmEp3sxnTj2TX1S4gDz8hA 2GB      trident-basic  {{  0   []} { }}  }
time="2017-11-10T17:33:00Z" level=debug msg="Filesystem format." fileSystemType=ext4 name=lmEp3sxnTj2TX1S4gDz8hA
time="2017-11-10T17:33:00Z" level=debug msg="Considering pool." Name="Disk_Pool_1" Pool={false 60080E50003249B8000000A65326D473 0400000060080E50003249B8000000A65326D473 Disk_Pool_1 2516850835456 ssd}
time="2017-11-10T17:33:00Z" level=debug msg="Pool does not match requested media type." MediaType=ssd Name="Disk_Pool_1" RequestedMediaType=hdd
time="2017-11-10T17:33:00Z" level=debug msg="Read volumes." Count=6
time="2017-11-10T17:33:00Z" level=warning msg="Failed to create the volume on this backend." backend="eseries_10.209.1114.6" error="Create failed. No storage pools matched specified parameters." storagePool="Disk_Pool_1" volume=trident
time="2017-11-10T17:33:00Z" level=warning msg="Failed to create the volume on this backend!" backend="eseries_10.209.1114.6" error="Create failed. No storage pools matched specified parameters." pool="Disk_Pool_1" volume=trident
time="2017-11-10T17:33:00Z" level=error msg="Encountered error(s) in creating the volume: [Failed to create volume trident on storage pool Disk_Pool_1 from backend eseries_10.209.1114.6: Create failed. No storage pools matched specified parameters.]" handler=AddVolume
time="2017-11-10T17:33:00Z" level=info msg="API server REST call." duration=41.641467ms method=POST route=AddVolume uri="/trident/v1/volume"

launcher log:
time="2017-11-10T17:38:38Z" level=info msg="Launcher successfully retrieved the version of Kubernetes." gitCommit=fff65cf major=1 minor=6
time="2017-11-10T17:38:38Z" level=info msg="Launcher created the pod." pod=trident-ephemeral
time="2017-11-10T17:38:41Z" level=info msg="Launcher detected the IP address for the pod." ipAddress=10.1.2.54 pod=trident-ephemeral
time="2017-11-10T17:38:41Z" level=debug msg="Launcher validated pod trident-ephemeral has bootstrapped after 2.022418ms."
time="2017-11-10T17:38:42Z" level=info msg="Launcher successfully added backend eseries_10.209.1114.6 to pod trident-ephemeral."
time="2017-11-10T17:38:42Z" level=info msg="Launcher successfully added storage class trident-basic to pod trident-ephemeral."
time="2017-11-10T17:38:42Z" level=error msg="Launcher failed in adding volume trident: Encountered error(s) in creating the volume: [Failed to create volume trident on storage pool Disk_Pool_1 from backend eseries_10.209.1114.6: Create failed. No storage pools matched specified parameters.]"
time="2017-11-10T17:38:42Z" level=debug msg="Launcher is starting the cleanup after failure." deploymentCreated=false error="Encountered error(s) in creating the volume: [Failed to create volume trident on storage pool Disk_Pool_1 from backend eseries_10.209.1114.6: Create failed. No storage pools matched specified parameters.]" pvCreated=false pvcCreated=false tridentEphemeralCreated=true volumeCreated=false

Speaking to ardalan on slack, and they identified the following:

It seems the culprit is the following line that sets mediaType to hdd when this parameter is not specified during volume creation (Trident itself correctly discovered the media type for your E-Series backend: map[media:{Offers: ssd} snapshots:{Offer: false} encryption:{Offer: false} provisioningType:{Offers: thick} backendType:{Offers: eseries-iscsi}]} ): https://github.com/NetApp/netappdvp/blob/master/storage_drivers/eseries_iscsi.go#L195.

kangarlou commented 6 years ago

@clintonk This issue should have been created under the Trident repo as the cause seems to be the interactions between Trident and nDVP.

kangarlou commented 6 years ago

Closed per 434e31cf7382d5d04f28fa20870876a779dafb9c.