Open cwfitzgerald opened 1 year ago
I like the sound of this! Especially the flexibility of the ownership model for imported resources.
Second we change the as_hal interop functions to be as_inner_api and take
A: Api
impl CommandEncoder { fn as_inner_api<A: Api>(&self, f: impl FnOnce(&mut A::CommandBuffer) -> R) -> R; }
This would need to return Option<R>
or panic if you pass the wrong associated type? Multiple backends in wgpu-hal being the main reason.
The ownership of all wgpu-imported resources will depend on the presence of a DropGuard. This drop guard is a
Box<dyn FnOnce(A::Raw*)>
which will be called when the resource is destroyed.
Do we need describe any threading guarantees about the drop guard? Also when are objects released? Currently the drop guard is Send + Sync. Assuming all the api types are copy or stateless, Send + Sync is probably fine and you'd just send the type back through a channel.
Creation
For each importable object, we will add a associated type that gives all the information wgpu-hal needs to successfully import a type.
trait Api { type InstanceImportDescriptor; type AdapterImportDescriptor; type DeviceImportDescriptor; type TextureImportDescriptor; type BufferImportDescriptor; // Note _not_ command encoder. type CommandBufferImportDescriptor; }
A few of these associated types could be quite API dependent? I'm not sure how you'd import a command buffer from GL given that doesn't conceptually exist.
For Instance, Adapter and Device, do we need to provide a way for external apis to query what features/extensions wgpu expects? Vulkan is the biggest example and pain of the current process.
On the wgpu level, it will look like:
impl Device { unsafe fn create_texture_from_inner<A: Api>( &self, raw: A::Texture, desc: &TextureDescriptor, import_desc: A::TextureImportDescriptor, drop_guard: DropGuard, ); }
May need to handle panics or return an error from wrong API type?
Resource States
I imagine most of this is going to be Vulkan and DX12 related?
Other things:
Some APIs have primitives for fences and semaphores. How should wgpu allow having these be signalled when you submit commands from wgpu. There might also be an argument to allow exporting a fence/sync object from wgpu (I think EGLSync might be an example of this? Need to check)
Some use cases like using wgpu in a Wayland compositor will ideally have wgpu signal a fence I give to kernel modesetting so that when wgpu finishes rendering an atomic flip is committed. So this is probably an important thing to consider,
I'm strongly interested in this, I've already implemented a bunch of interop functions and was thinking about making it a separate crate like wgpu-interop
or something, but it would be even better to have it directly in wgpu
If it's of any help, my implementations are here: https://github.com/gyroflow/gyroflow/tree/master/src/core/gpu in wgpu_interop*.rs, and a list of interop possibilities is here
For Instance, Adapter and Device, do we need to provide a way for external apis to query what features/extensions wgpu expects? Vulkan is the biggest example and pain of the current process.
Do you mean like https://docs.rs/wgpu-hal/0.17.0/wgpu_hal/vulkan/struct.Instance.html#method.required_extensions and https://docs.rs/wgpu-hal/0.17.0/wgpu_hal/vulkan/struct.Adapter.html#method.required_device_extensions? Or something else?
It should be possible for an application creating a Vulkan instance to call Instance::from_raw
and then call required_extensions
, I think. I don't see an API to create an adapter from raw Vulkan types, so it would have to enumerate adapters from the instance to call required_device_extensions
.
Conversely, to make use of as_hal
with Vulkan when the adapter is created by wgpu, it seems like there would need to be a way to tell wgpu/wgpu_hal what additional Vulkan device/instance extensions the applications wants? Though that isn't technically necessary if the application can use *from_raw
with to create the instance and device from Vulkan types.
For Instance, Adapter and Device, do we need to provide a way for external apis to query what features/extensions wgpu expects? Vulkan is the biggest example and pain of the current process.
Do you mean like https://docs.rs/wgpu-hal/0.17.0/wgpu_hal/vulkan/struct.Instance.html#method.required_extensions and https://docs.rs/wgpu-hal/0.17.0/wgpu_hal/vulkan/struct.Adapter.html#method.required_device_extensions? Or something else?
It should be possible for an application creating a Vulkan instance to call
Instance::from_raw
and then callrequired_extensions
, I think. I don't see an API to create an adapter from raw Vulkan types, so it would have to enumerate adapters from the instance to callrequired_device_extensions
.
Yes, but there are finer details like ensuring device features that are given are enabled and a way to negotiate optional ones.
Conversely, to make use of
as_hal
with Vulkan when the adapter is created by wgpu, it seems like there would need to be a way to tell wgpu/wgpu_hal what additional Vulkan device/instance extensions the applications wants? Though that isn't technically necessary if the application can use*from_raw
with to create the instance and device from Vulkan types.
This wouldn't work since some extensions require chaining something to VkDeviceCreateInfo.
For the GL(ES) backend, what would be be the "Raw" Instance, Adapter and Queue?
For EGL there is a context in the Instance, Adapter and Queue.
Also I'd probably ask the same regarding the pipelines and commandbuffer/encoder.
For dx11, these would probably all be unit, but probably not of use right now (so unit ()
).
For Metal everything seems to map well except for Instance not really existing, and Device
and Adapter
are the same thing pretty much.
DX12, I think d3d12::DxgiFactory in an instance and everything else maps over.
A future pull request over #4573 might mean a RawTexture
on Vulkan is in fact an array of textures? Not sure where this will go.
A future pull request over #4573 might mean a
RawTexture
on Vulkan is in fact an array of textures? Not sure where this will go.
Texture arrays and multi-planar textures are a different thing, and that's not specific to Vulkan. Multi-planar textures (used in video decoders/encoders) are a single texture, but they have a special ability to create a texture view to each plane.
In general, video APIs which output/input these hardware video textures are: DXVA2/D3D11VA/MF (windows), VideoToolbox (apple), NVIDIA Video API (windows, linux), MediaCodec (android), AMD AMF (windows, linux), Intel Quick Sync (windows, linux), VA-API (linux, windows), VDPAU (linux), Vulkan video extension
Hi, I've seen there's some work on this and quite a few issues closed and pointing this instead.
I had a look as saw we have Texture::as_hal
, I tried to use it like so but nothing in Dx12::Texture
is public.
let d3d12_resource = unsafe {
texture.as_hal::<Dx12, _, _>(|hal_texture| {
if let Some(hal_texture) = hal_texture {
Some(hal_texture.resource.as_ptr())
} else {
None
}
})
};
126 | Some(hal_texture.resource.as_ptr())
| ^^^^^^^^ private field
Was hoping to cast it to a IDXGIResource
so that I can grab the handle and share it to other processes:
if let Some(d3d12_resource) = d3d12_resource {
use windows::core::ComInterface;
let dxgi_resource: IDXGIResource = d3d12_resource.cast().expect("Should be able to cast.");
let shared_handle = unsafe { dxgi_resource.GetSharedHandle() }.unwrap();
println!("Shared handle: {:?}", shared_handle);
} else {
println!("Failed to get D3D12 resource from texture");
}
Should those fields be public in Dx12::Texture
? I'm failing to see what I could actually do with the texture I get from as_hal
, I've seen examples around of creating texture from raw but not seeing the reverse.
Cheers
Was hoping to cast it to a IDXGIResource so that I can grab the handle and share it to other processes
I'd like to be able to grab handles for both vulkan and dx12 (for buffers/textures). This is also quite hard for vulkan (basically need to duplicate the entire create buffer code), it would be nice to have this as a wgpu feature. It seems that this is supported in dx12 always and vulkan after 1.1 (or with VK_KHR_external_memory
).
It is decently common for people to want to integrate wgpu into a larger application that is using a graphics api or to use a library built around a graphics api to integrate with wgpu. There are many different ways and with a wide variety of resources that you could need to do interop with, so this proposal will be a set of smaller api changes that combine to have a unified picture to underlying apis.
This API is sure to evolve as it is refined, this is just a first attempt to unify it
wgpu layer
We currently only support getting the underlying types for a wgpu type if we're running on wgpu-core/hal. We should also allow this for access to the underlying WebGPU resource. To facilitate this, we should have a trait that mirrors wgpu_hal::Api but only for associated types. This is only there for allowing generic functions, so no real trait bounds need to exist.
Second we change the as_hal interop functions to be as_inner_api and take
A: Api
Ownership
The ownership of wgpu-created resources always lies in wgpu, all client code must keep the wgpu object alive while they are using the object.
The ownership of all wgpu-imported resources will depend on the presence of a DropGuard. This drop guard is a
Box<dyn FnOnce(A::Raw*)>
which will be called when the resource is destroyed.Creation
For each importable object, we will add a associated type that gives all the information wgpu-hal needs to successfully import a type.
On the wgpu level, it will look like:
This should work for all the importable objects.
Resource States
Add the following to the API trait
When using buffers and textures with external code, or external command buffers, you must follow the following flow (as seen by the queue command stream)
The release/acquire terminology could probably be improved.
Step one and three can be accomplished with the following api: