markus-hentsch
commented
2 months ago Possible changes to disk_format
Initially, we proposed to introduce a new container_format to represent encrypted images.
Glance was not a fan of this and requested to not introduce new formats and instead solely rely on metadata for this^1.
Due to the recent OSSA-2024-001 and OSSA-2024-002 CVEs, the proper marking and scanning of image formats has gained new importance.
This now leads to a rethinking and new requests for changes in our approach.
On 2024-08-28 in the weekly Glance IRC meeting^2, @josephineSei and I couldn't participate due to a conflicting meeting but the image encryption patchsets were discussed nonetheless and an interesting point was raised:
14:08:35 <pdeore> I have added few suggestions on parameter change patch but I request other cores to have a look at those patches
14:08:39 <dansmith> I feel like we need to revisit a couple things about how we store these images
14:08:49 <dansmith> in light of the giant CVE recently
14:09:07 <dansmith> in that I think we need to have a specific disk_format for luks-encrypted images,
14:09:31 <dansmith> so that we can inspect them with a known target format and reject things that are supposed to be encrypted but aren't (and v-v)
14:10:06 <dansmith> that goes with my proposal to also basically stop using "raw" to mean "image of a PC-like disk or partition"
14:10:17 <dansmith> (in my defender spec)
14:10:36 <dansmith> so I feel like we probably need to discuss that with glance, cinder, and nova people together
14:11:21 <dansmith> much of the complexity in the recent CVE came around the fact that we can never trust the disk_format in glance, and many of the side attack vectors came by putting one format in glance but calling it something elseThen we received the following comment on the Glance patchset from Dan Smith (Nova):
This is sort of an awkward place to raise this, but:
I think something we learned from the recent mega-CVE is that I think we need a new disk_format for luks-encrypted images. So much of the complexity of handling that CVE came from all the side vectors by which you can fool nova, glance, and cinder into doing something bad by saying an image is in one format, but actually sending another. Specifically, I think we have got to stop allowing raw to be both a catch-all format, as well as the thing we use when we really mean "an image of a PC-like disk".
Having to probe raw images to see if they smell like a LUKS disk, and if not, assume it's a regular raw is inviting more possibility for issue here I think.
I've got a spec proposed to make glance inspect and reject uploads that do not conform to the stated disk_format (i.e. you said it was raw, but uploaded a vmdk -> fail). It's here:
https://review.opendev.org/c/openstack/glance-specs/+/925111
and I have a LUKS inspector proposed against format_inspector:
https://review.opendev.org/c/openstack/oslo.utils/+/926809
those two things together will basically require that you declare a thing as luks before uploading it. One could also make the argument that we should use container_format=luks and disk_format=gpt for the typical arrangement, but I think that's more complicated. Either way, some discussion is required, IMHO.
As a result, we once again need to rethink and rework our encryption format approach in the Cinder and Glance patchsets to address this, it seems.
josephineSei
Currently there is the possibility in Cinder to encrypt volumes and in Nova to use qcow2 encrypted images (still under development). Both can lead to and use LUKS-encrpyted images, but those are different and not aligned:
@markus-hentsch also found out in https://github.com/SovereignCloudStack/standards/issues/541 that uploading a LUKS-encrypted image (that was created from a volume) to another cloud in combination with setting a few parameteres (cinder_encryption_key, etc...) will result in an image that can be used to create an encrpyted and functional volume.
As a user it would be good to have a streamlined operation to use encrypted images in openstack for both volumes and ephemeral storage and to also allow interoperability between clouds. Therefore we need and will propose standardized parameters to describe and detect an encrypted image, which might be similar to the parameters described here: https://specs.openstack.org/openstack/cinder-specs/specs/zed/image-encryption.html But will use the LUKS encryption. So those encrypted images could be natively mounted in Nova or just formed into a volume (raw LUKS images can be directly used, qcow images need to be flattened).
With such a way encrypted backup images can be easily downloaded and transferred to another cloud.
This is a result from a lengthy discussion at the PTG with Nova, Cinder and Glance ( https://etherpad.opendev.org/p/dalmatian-ptg-cinder#L376 )
Followup tasks may be to implement re-encryption to fully change keys for LUKS volumes and images.