Is it planned to support functionality corresponding to DirectStorage in Vulkan?

[janekb04]

Some time ago a feature called DirectStorage was announced to be added to the DirectX API. In July 2021 Microsoft apparently released a preview version of the API to limited partners.

Is it planned for Vulkan to support functionality that is supposed to be supported by DirectStorage? By this, I mostly mean GPU decompression of assets, as I imagine disk IO would be out of scope for Vulkan.

I know that DirectStorage is in an early phase and that publically available information about it is limited, so I wanted to ask if there have been any discussions yet regarding a possible introduction of similar functionality to Vulkan?

[Tobski]

Hi @janekb04, thanks for your interest! We are aware of the functionality but are not currently doing any active development. As the Microsoft API is not public, we can't really comment on what we would or would not add to Vulkan either.

Any information you can provide about use cases or that would help us prioritise this feature request would be most welcome.

[APeasantFarmer]

ditto what @janekb04 said. Seems like this API is now published. Annoucement: https://devblogs.microsoft.com/directx/directstorage-api-available-on-pc/ Windows Sample: https://github.com/microsoft/DirectStorage Collection of Links: https://devblogs.microsoft.com/directx/directstorage-api-downloads/

With respect to use cases, I believe numerous xbox games have this feature, but I don't know any uses on windows yet beyond the above linked sample.

[K0bin]

The released version of DirectStorage doesn't interact with the GPU at all besides standard D3D12 calls. GPU decompression and direct SSD to GPU transfers are planned but not yet released.

[APeasantFarmer]

Ah, you are correct.

This release of DirectStorage provides developers everything they need to move to a new model of IO for their games, and we’re working on even more ways to offload work from the CPU. GPU decompression is next on our roadmap, a feature that will give developers more control over resources and how hardware is leveraged.

[therpgmaster]

Using DirectStorage to load data into device memory is showing significant speed improvements (up to 3x faster). It would be great to have a system like that usable with Vulkan. I don't know exactly what feature of DirectStorage has the most impact on performance, but it's probably GPU-related, likely in scope for Vulkan.

[janekb04]

Hi @janekb04, thanks for your interest! We are aware of the functionality but are not currently doing any active development. As the Microsoft API is not public, we can't really comment on what we would or would not add to Vulkan either.

Any information you can provide about use cases or that would help us prioritise this feature request would be most welcome.

@Tobski Some time has passed and Microsoft has released the API. Regarding use cases, MS’s own Developer Guidance describes what I had in mind when opening this issue. To summarize, I would imagine the usage to look something like this:

• create a special kind of queue for storage operations (VkStorageQueue perhaps?)
• create a special kind of command buffer to manage a batch of storage operations (VkStorageCommandBuffer perhaps?)
• record commands to the buffer (VkBeginStorageCommandBuffer perhaps?)
  • a command like VkStorageCmdRead
  • a command like VkStorageCmdDecompress which would leverage hardware decompression
• end recording the command buffer and then submit it analogously to how regular command buffers are used

I imagine that to keep up with the philosophy of Vulkan, contrary to DStorage, the user would be responsible for creating any staging buffers and submission threads, rather then the API itself.

Has the idea of introducing such an asynchronous loading API to Vulkan been considered since the issue had been opened or since the DirectStorage API was released?

[zeno-endemann-kdab]

From what I can tell from the linked MS Developer Guidance, I agree with K0bin, there currently seems nothing interacting with the GPU directly in there, so you can implement something like this in your own engine or in a helper library on top, no need for a Vulkan API for that.

What I'd be much more interested in would be an extension that would provide a transfer queue command that, instead of taking a staging buffer as source, would take a file descriptor, to let the OS transfer file contents directly to the GPU thus skipping the whole staging buffer copy.

[janekb04]

• DirectStorage 1.1 is about to be released with GPU decompression support.
• NVIDIA created an extension VK_NV_memory_decompression. It adds a single command vkCmdDecompressMemoryNV, which seems to work exactly like how I suggested above.
• Are there any plans to make this a KHR extension or an optional core functionality? (similarly to raytracing or mesh shaders)

[Sirtsu55]

I am also interested in this, has there been any work done towards this to make it a KHR extension? The NVIDIA's API leverages GPU decompression and is a leap forward in performance from CPU decompression. Because Vulkan is a Graphics API, it makes little sense to tackle OS-specific file I/O, but tackling decompression is in the realms of Vulkan. Perhaps maybe a cross-vendor API draft can be built upon NVIDIA's extension? It would be nice if this wasn't exclusive to NVIDIA GPUs. It is worth mentioning that the compression/decompression method is called GDeflate.

[skewty]

What about AMD support? This Reddit post claims AMD supports DS 1.1 in GCN gen 1 all the way back to the HD 7000 GPUs.

If I am understanding correctly, I believe that means AMD should have direct storage -> GPU (bypassing CPU) transfers in Vulkan supported in GPUs back to HD 7000 (circa 2012). Is this correct / confirmed?