Is it possible to convert textures & buffers to/from backend handle types?

[Gonkalbell]

If I'm only concerned about a single backend, and I am using and API that directly returns/uses raw handles to OpenGL/Vulkan objects, is there a way to convert those handles to/from their corresponding wgpu object? I don't seem to see any way to currently do this in wgpu-rs, wgpu-native, or gfx-hal, but I could have missed something. If it's not a feature yet, is there any plan to add it?

[kvark]

An ability to do this would first need to be added to gfx-rs backends, individually. So it's going to be backend-specific public APIs. Then wgpu-core would need to have some way of creating resources from external handles. There are concerns to think about:

1. who owns the resource? if the ownership transfer goes to wgpu, then we are good
2. how should resource be destroyed, given that wgpu doesn't know how it was created?
3. what can this resource do? often, external resources are much more limited in what they can do

[schulzch]

I guess accessing texture handles is currently impossible?

Use case: Most VR/AR/MR devices require access to a couple of "framebuffer-ish" texture handles to then do the compositing, e.g.:

/**
 * Converts a Direct3D 12 texture to #varjo_Texture.
 */
VARJO_API struct varjo_Texture varjo_FromD3D12Texture(struct ID3D12Resource* texture);

(see also OpenXR, e.g., here for a more complete example on swap chain construction)

I spend a few minutes digging into gfx-hal/dx12/resource.rs. Am I right that the backend has resources publicly accessible, but wgpu::Backend is a dead end regarding access to the backend?

[kvark]

Conceptually, wgpu abstracts away the native API. That's the only way we can make it safe and sound. Clearly, the VR/XR requires some punching through this abstraction, and it needs to touch gfx, wgpu, and wgpu-rs - all of them.

[schulzch]

Just to mention another use case: We use Spout to share OpenGL textures (from our particle visualization framework MegaMol) with DirectX (to Unity). This way our framework runs out of process (e.g., on a GPU cluster, not crashing Unity, ...), while simplifying XR prototyping.

[schulzch]

We sticked with gfx-pre-ll for now and did OpenGL texture ID conversion like this:

pub fn texture_from_id_and_size<T>(
    texture_id: gfx_gl::types::GLuint,
    width: u32,
    height: u32,
) -> gfx_core::handle::Texture<gfx_device_gl::Resources, <T as gfx::format::Formatted>::Surface>
where
    T: gfx::format::TextureFormat + gfx::format::RenderFormat,
{
    use gfx_core::handle::Producer;
    let mut temp: gfx_core::handle::Manager<gfx_device_gl::Resources> =
        gfx_core::handle::Manager::new();
    let raw_texture = temp.make_texture(
        gfx_device_gl::NewTexture::Texture(texture_id),
        gfx_core::texture::Info {
            levels: 1,
            kind: gfx_core::texture::Kind::D2(
                width as u16,
                height as u16,
                gfx_core::texture::AaMode::Single,
            ),
            format: gfx_core::format::SurfaceType::R8_G8_B8_A8,
            bind: gfx_core::memory::Bind::RENDER_TARGET | gfx_core::memory::Bind::TRANSFER_SRC,
            usage: gfx_core::memory::Usage::Data,
        },
    );
    use crate::gfx::memory::Typed;
    gfx::handle::Texture::new(raw_texture)
}

The function above is used here: visual-system-simulator/.../varjo.rs. Maybe it helps :)

[cwfitzgerald]

Duplicate of #965.