Pipeline caches on DX12

[kvark]

We have the API for pipeline caches, but the backends don't yet fully implement it. See create_graphics_pipeline signature and other use of PipelineCache.

D3D12 could store the pipeline blobs in the cache: https://docs.microsoft.com/en-us/windows/desktop/api/d3d12/ns-d3d12-d3d12_cached_pipeline_state

We'd need to be able to save/load that cache from disk, as well as properly select one (of those saved in the PipelineCache implementation) when a new pipeline is created:

This structure is intended to be filled with the data retrieved from ID3D12PipelineState::GetCachedBlob. This cached PSO contains data specific to the hardware, driver, and machine that it was retrieved from. Compilation using this data should be faster than compilation without. The rest of the data in the PSO needs to still be valid, and needs to match the cached PSO, otherwise E_INVALIDARG might be returned.

The logic of the backend, when creating a pipeline with the cache provided, would need to be the following:

1. Lookup the given pipeline state in a HashMap<PipelineState, D3D12_CACHED_PIPELINE_STATE> owned by the pipeline cache.
2. If there is a cached state, try to create a pipeline with it.
3. If succeeded, that's good! No need to do anything else.
4. If failed with D3D12_ERROR_DRIVER_VERSION_MISMATCH or D3D12_ERROR_ADAPTER_NOT_FOUND, issue a warn! message accordingly and proceed again but without using the D3D12_CACHED_PIPELINE_STATE.
5. Obtain the new blob and store it in the pipeline cache. It may be overwriting the old value if it failed to work (due to driver/gpu/os incompatibility), which is fine.

Note: pipeline caches are internally synchronized, so any mutation of them should be done behind a sync structure, such as RwLock or Mutex. Alternatively, if you feel courageous, may try Kudzu like explained in #2860

Marking as high priority because it is requested by the Szeged team in order to speed up WebRender startup when running on DX12. cc @zakorgy

[Cartogy]

Howdy, I would like to work on this if it is not being worked on currently.

[kvark]

It's not being worked on, but still very much needed. Your help is much welcome!

[Cartogy]

Regarding the data parameter in the create_pipeline_cache.

As the cache needs to be loaded from disk, I was thinking of passing the file path via the data parameter. This provides two scenarios that I could think of:

1. When creating a cache, the PipelineStatemust be passed as data in order to create the blob and provide the key in the HashMap<PipelineState, D3D12_CACHED_PIPELINE_STATE>. Meaning the data parameter would provide two different types of information: File path or Pipeline State. (Not ideal)

2. Create an empty PipelineCache when data is None. Pass this empty PipelineCache to create_graphics_pipeline and fill the PipelineCache once it has created the PipelineState in the function.

I am in favor of option 2, but that would mean that a PipelineCache can not be created from an already existing pipeline. It can only be created during the create_graphics_pipline function, as the data parameter would only store the file path to load from disk.

Is option 2 one way of doing it and having the data parameter be used as a file path or is there another option that I am completely unaware of?

A thought just occurred to me: will having data be used for the file path break code compatibility across backends?

(I am fairly new to Pipeline Caches, so there could be some things I may be missing. If there are any documents or articles that you recommend I read, please feel free to share)

[kvark]

@Cartogy we don't want to change the API for this. The current API follows Vulkan (you can also check the VkPipelineCache documentation there), and we need to keep it this way. Maybe this will help?

[hgallagher1993]

Hi, just to let you know I've started looking into this since it seems to be free

[hgallagher1993]

A few things came up unexpectedly in the last few weeks which meant I had basically no free time but I'm back working on this now

[hgallagher1993]

Ok I've been reading the D3D12 documentation and I think I'm pretty clear on that but I've a question on the gfx-rs API. In the dx12 device.rs each use of _cache has the type () because the backend is implemented with type PipelineCache = (); I'm wondering what I should change the PipelineCache type to? I tried using command::PipelineCache but it won't compile because *mut ID3D12PipelineState cannot be sent between threads safely

So should I continue trying to use command::PipelineCache and just figure out how to make that work or would it be better to create my own PipelineCache in a file called pipeline_cache.rs that just contains HashMap<PipelineState, D3D12_CACHED_PIPELINE_STATE> basically like how it was done in #3719?

[kvark]

@hgallagher1993

I'm wondering what I should change the PipelineCache type to?

You'd declare a struct with stuff inside. There is no ready to go implementation for you, or otherwise this bug would be closed long ago. The struct would need to contain some sort of a hashmap from D3D12 pipeline states to D3D12 blobs.

I tried using command::PipelineCache

This is an entirely different thing, unrelated to the problem at hand. It should be renamed to avoid confusion.

In general, looking at #3719 should give you a rough idea :)

[hgallagher1993]

Perfect that clears things up, thanks!

[hgallagher1993]

Just a small update since it's been a while, I have the basic functionality of saving / loading the cache to and from disk done I think but I'm getting an invalid argument error when I actually try to use the cached blob so I'm pretty much just debugging that now

[hgallagher1993]

There's been a bit of a disaster. So last Saturday I was on my laptop then all of a sudden it turned off.....and hasn't turned on since, won't charge either just dead like I have to write this from my work laptop. I actually had this task pretty much finished I think but of course I didn't back it up by pushing any changes to my fork so they're gone basically.

If this is still free by the time I get set up with a new laptop I'll actually finish it then, but that could be weeks.

[kvark]

That's unfortunate! Hope you can recover anything.