Unable to create new volume type

[millingw]

We'd like to be able to use multi-attach volumes on Somerville, to allow us to mount the GAIA DR3 dataset (10 TB) from ceph with read-only permissions into kubernetes pods. I've tried following the instructions here https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html to create a multi-attach volume type, but get the following error both using a normal project credential and with an unrestricted credential when trying to create the new volume type:

openstack volume type create gaia_multiattach --os-cloud somerville-jade Policy doesn't allow volume_extension:type_create to be performed. (HTTP 403) (Request-ID: req-4d337c64-9ca7-41f7-8750-38fe25dc82be)

I guess another way to do this could be via a CephFS share?

I'm assuming a volume created with a ceph-hdd type does not have multi-attach set to true, so we can't mount it directly into a pod with cinder.

[GregBlow]

cephFS would provide a file system that can be mounted on multiple servers. What is the use case you're trying to solve for here, please?

[millingw]

For our kubernetes-based version for the GAIA data mining platform, we want to be able to mount the GAIA dataset simultaneously into many kubernetes pods as opposed to OpenStack VMs. It looks like the cinder driver will let us just specify the volume id and access credentials in the pod configuration and it will take care of the details. In order to do that, it looks like we need to have the volume created with the multiattach property set. I've subsequently reread the instructions, and it looks like we can actually set the property on the volume post-creation, eg openstack volume --set multiattach=True --os-cloud somerville-jade malcolm_test_volume. So I think this ticket can actually be closed. One thing I'm a little bit unclear on is whether we should be using a volume in this way to expose the data into the pods, or using a share, given we expect the data to be read only and a large dataset size.

[GregBlow]

I think the remaining question is one of relative performance, as it sounds like both would be valid. I'm not sure I have the experience necessary to give an indication of which would be more suitable. Closing this ticket for now, but feel free to re-open if you'd wish to raise the matter again.

[GregBlow]

Hi Greg,

We were discussing multiattach volumes in issue 185 https://github.com/lsst-uk/somerville-operations/issues/185

Subsequently, appears that setting the multiattach property post-creation has no effect and we do need to have a new volume type with multiattach set to True.

Should I create a new issue, or can you reopen 185?

Thanks,

Malcolm

[millingw]

Could you create a new volume type for us in the jade project, with multattach set to true as per https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html? Creating the new volume type seems to require admin-level permissions, which I understand you would not want us to have

[GregBlow]

I've created a new volume type as per instructions. Seemed a bit too easy, so give it a go and see if it works as you'd expect please?

[GregBlow]

p.s. could you let us know if you cannot close/re-open tickets yourself?

[millingw]

It seems to be working suspiciously well with the new volume type. We've been able to simultaneously access the same volume from multiple VMs and a k8s instance.