proposal: swapchain & presentation rework

[rajveermalviya]

Context

Currently the WGPUSwapChain API diverts from the js spec; this proposal tries to match the js spec while also maintaining the functionality that is required in the native context.

Keep in mind this proposal is mostly what wgpu-rs already exposes so it tries to match it.

Update present mode enum to add automatic modes

// matches wgpu
// https://docs.rs/wgpu-types/latest/wgpu_types/enum.PresentMode.html
typedef enum WGPUPresentMode {
    WGPUPresentMode_Fifo = 0x00000000, // default
    // VK_PRESENT_MODE_FIFO_RELAXED_KHR
    WGPUPresentMode_FifoRelaxed = 0x00000001, // new
    WGPUPresentMode_Immediate = 0x00000002,
    WGPUPresentMode_Mailbox = 0x00000003,
    WGPUPresentMode_Force32 = 0x7FFFFFFF
} WGPUPresentMode;

Remove WGPUSwapChain (and merge it's interface into Surface api)

In the js spec this was done a while ago. Fixes: https://github.com/webgpu-native/webgpu-headers/issues/88

// matches wgpu
// https://docs.rs/wgpu-types/latest/wgpu_types/enum.CompositeAlphaMode.html
typedef enum WGPUCompositeAlphaMode {
    // Chooses between Opaque & Inherit whichever is available.
    WGPUCompositeAlphaMode_Auto = 0x00000000,
    WGPUCompositeAlphaMode_Opaque = 0x00000001,
    WGPUCompositeAlphaMode_PreMultiplied = 0x00000002,
    WGPUCompositeAlphaMode_UnPreMultiplied = 0x00000003,
    WGPUCompositeAlphaMode_Inherit = 0x00000004,
    WGPUCompositeAlphaMode_Force32 = 0x7FFFFFFF
} WGPUCompositeAlphaMode;

// Replaces WGPUSwapChainDescriptor
typedef struct WGPUSurfaceConfiguration {
    WGPUChainedStruct const * nextInChain;
    WGPUDevice device;
    WGPUTextureFormat format;
    WGPUTextureUsage usage;
    size_t viewFormatCount;
    WGPUTextureFormat const * viewFormats;
    WGPUCompositeAlphaMode alphaMode;

    uint32_t width;
    uint32_t height;
    WGPUPresentMode presentMode;
} WGPUSurfaceConfiguration;

// Replaces wgpuDeviceCreateSwapChain
void wgpuSurfaceConfigure(WGPUSurface surface, WGPUSurfaceConfiguration const * config);
// This function probably should be skipped but we need to match js spec.
void wgpuSurfaceUnconfigure(WGPUSurface surface);

Provide current Texture instead of TextureView

Fixes: https://github.com/webgpu-native/webgpu-headers/issues/89

typedef enum WGPUSurfaceGetCurrentTextureError {
    WGPUSurfaceGetCurrentTextureError_NoError = 0x00000000,
    WGPUSurfaceGetCurrentTextureError_Timeout = 0x00000001,
    WGPUSurfaceGetCurrentTextureError_Outdated = 0x00000002,
    WGPUSurfaceGetCurrentTextureError_Lost = 0x00000003,
    WGPUSurfaceGetCurrentTextureError_OutOfMemory = 0x00000004,
    WGPUSurfaceGetCurrentTextureError_DeviceLost = 0x00000005,
    WGPUSurfaceGetCurrentTextureError_Force32 = 0x7FFFFFFF
} WGPUSurfaceGetCurrentTextureError;

typedef struct WGPUSurfaceTexture {
    WGPUTexture texture;
    // If suboptimal is true presentation will still succeed,
    // but the internal swapchain is no longer configured optimally
    // for the surface it targets. Applications should recreate
    // their swapchain at the next convenient opportunity.
    bool suboptimal;
    WGPUSurfaceGetCurrentTextureError error;
} WGPUSurfaceTexture;

// Replaces wgpuSwapChainGetCurrentTextureView
void wgpuSurfaceGetCurrentTexture(WGPUSurface surface, WGPUSurfaceTexture * surfaceTexture);

Provide a way to query information about the surface

typedef struct WGPUSurfaceCapabilities {
    WGPUChainedStructOut * nextInChain;
    size_t formatCount;
    WGPUTextureFormat * formats;
    size_t presentModeCount;
    WGPUPresentMode * presentModes;
    size_t alphaModeCount;
    WGPUCompositeAlphaMode * alphaModes;
} WGPUSurfaceCapabilities;

void wgpuSurfaceGetCapabilities(WGPUSurface surface, WGPUAdapter adapter, WGPUSurfaceCapabilities * capabilities);
void wgpuSurfaceCapabilitiesFreeMembers(WGPUSurfaceCapabilities capabilities);

Note: wgpuSurfaceGetCapabilities could replace current wgpuSurfaceGetPreferredFormat by guaranteeing that first format is the preferred one. But removing wgpuSurfaceGetPreferredFormat will divert from js spec.

Move Present function to Surface

// DevicePushErrorScope/DevicePopErrorScope can be used to catch errors
void wgpuSurfacePresent(WGPUSurface surface);

open discussion: presentation extensions

frame pacing & damage support ref: https://github.com/gfx-rs/wgpu/issues/2869

[Kangz]

This seems like a pretty good direction, anything not commented on looks good!

Update present mode enum to add automatic modes

Why is FifoRelaxed in AutoVsync? It can result in visible tearing in some cases, is that ok? Can we guarantee Fifo is always supported?

Remove WGPUSwapChain (and merge it's interface into Surface api)

Why do we need all these various alpha modes? Are any guaranteed to be present? +1 for ColorSpaces, they could be used in other places like helper functions to implement importExternalTexture and copyEI2T. In general for higher level language bindings it would be nice if most things could have default values. (except present mode, format and width/height?). Keeping Configure still makes sense as it removes a reference to the WGPUDevice and lets the CPU allocation be freed in some cases. (similar reason to why we have this in JS).

Provide current Texture instead of TextureView

Why do we have an errorMsg? Can't this use the regular facilities for errors on the WGPUDevice? Also the FreeMembers would depend on the resolution to that.

Provide a way to query information about the surface

Why not use a ChainedStructOut and a FreeMembers method? It seems likely that we are going to add things to this structure over time. IMHO we could have a duplicate method for GetPreferredFormat just to match JS.

[rajveermalviya]

Why is FifoRelaxed in AutoVsync? It can result in visible tearing in some cases, is that ok?

Up for debate

Can we guarantee Fifo is always supported?

Vulkan requires it

Why do we need all these various alpha modes?

It matches VkCompositeAlphaFlagBitsKHR.

Are any guaranteed to be present?

No, hence Auto; it will choose between Opaque & Inherit, whichever is available. (adding this info in proposal) Vulkan on Android only advertises Inherit

Why do we have an errorMsg? Can't this use the regular facilities for errors on the WGPUDevice? Also the FreeMembers would depend on the resolution to that.

Re:Question: What do you think about the alternative API proposed in that section, it's much simpler and removes the struct?

Why not use a ChainedStructOut and a FreeMembers method?

Yeah, I wanted to keep the API simpler (one shot). Guess we should use FreeMembers there to keep things consistent. (updating the proposal).

[Kangz]

Re:Question: What do you think about the alternative API proposed in that section, it's much simpler and removes the struct?

Maybe it could be a return enum with also SurfaceLost and DeviceLost, but that would work!

[kainino0x]

Why do we have an errorMsg? Can't this use the regular facilities for errors on the WGPUDevice? Also the FreeMembers would depend on the resolution to that.

Re:Question: What do you think about the alternative API proposed in that section, it's much simpler and removes the struct?

I think we liked the pattern we described here for GetMappedRange: https://github.com/webgpu-native/webgpu-headers/issues/143#issuecomment-1612208345 So maybe same for GetCurrentTexture?

[kainino0x]

There was a lot of discussion on this today so I'll paste the raw meeting notes (not totally complete but should have captured the important stuff):

• Discussion of present modes
  • Depends on whether you care about tearing, or about backpressure.
  • FifoRelaxed allows tearing but provides backpressure
• Discussion of extra return stuff from GetCurrentTexture, like “suboptimal”
  • CF: wgpu always suboptimal on android because of some screen rotation stuff
  • in the JS API getCurrentTexture does have one [synchronous] error case (surface not configured)
  • CF: Error handling from GetCurrentTexture needs to be both synchronous and required
  • Failure cases of GetCurrentTexture?
    • unconfigured
    • surface lost
    • window minimized? this causes a resize to 0 also, but it’s racy
  • RM: Alternative API 2 has an enum instead of a string
  • RM/CF: Comfortable with an error enum instead of a string
  • CF: have GetCurrentTexture return a struct (by value) with: texture, error enum, suboptimal bool
  • RM: so we don’t need FreeMembers for this.
  • Also don’t need chained struct?
    • (agreement)
  • [x] do that
• GetCapabilities
  • CF: Why takes a pointer instead of returning struct?
  • Lets the user set up the chained struct so we don’t return anything they don’t understand
  • looks good
• Color spaces
  • CF: Windows particular about what color spaces you can use in different ways. Like rec2100 requires 10 or 11 bit unorm.
  • discussion of color spaces and hdr
    • https://www.w3.org/TR/css-color-4/
    • https://drafts.csswg.org/css-color-hdr/ (DRAFT)
    • see also https://github.com/gpuweb/gpuweb/issues/4239
  • CF: Need to either not add colorspaces or add a way to query the requirements. Should actually go find out what’s allowed where.
  • KN: Remove colorspaces completely for now, and plan to add them before V1?
    • Emscripten might just keep it in so people can use it in wasm
  • [x] do that (now filed as #200)
• Alpha
  • (discussion)
    • see also https://github.com/gpuweb/gpuweb/issues/1988
  • [ ] needs investigation
• Present modes
  • CF: FIFO is always supported so if you don’t want tearing, you can always use it. It’s a good default
    • I thought there was an issue with it on wayland but RM said it’s required on Vulkan?
    • RM: FIFO and mailbox …
  • [ ] CF: Should change the name of AutoVsync. Unclear whether it means tearing or backpressure.
  • default should be fifo
  • [x] change enum order so fifo is value 0

[rajveermalviya]

(updated the proposal)

[kainino0x]

Looks good so far! Filed #200 for color spaces.

I think the big thing left to discuss is alpha modes. We might want to split that one to a new issue as well, if we don't find the answers yet.

[kainino0x]

• Auto present modes
  • CF: No tearing would be Mailbox→Fifo
  • CF: Backpressure would be FifoRelaxed→Fifo
  • CF: Should there be auto modes at all? Handy in wgpu but Fifo is a good default and we could say you need to understand what you want to choose something else. Basically for backward compatibility
  • Remove them
  • CF: FifoRelaxed is only on AMD, could remove?
  • KN: Seems to be NVIDIA too, Windows and Linux
    • http://vulkan.gpuinfo.org/listsurfacepresentmodes.php
  • keep it
  • and keep the order in the proposal now
• Alpha modes
  • KN: Some platforms “opaque” mode requires the alpha to be 1, or it causes undefined compositing results
  • CF: Sounds like two opaque modes - one for “don’t use alpha channel” and one for “assume alpha channel is 1”. One would be optional
  • KN: Could support both with a copy+clear
  • CF: Prefer not to, just expose the limitation to the user
  • KN: Some portability hazard to the assume-alpha one
  • Auto would pick the assume-alpha one
  • (What is Inherit?)
    • RM: Android code: \ https://cs.android.com/android/platform/superproject/main/+/main:frameworks/native/vulkan/libvulkan/swapchain.cpp;l=740;drc=f499b5a6d57478ad842b55df85a63da0529e72f6
    • “On Android, window composition is a WindowManager property”
    • Vulkan: “VK_COMPOSITE_ALPHA_INHERIT_BIT_KHR: The way in which the presentation engine treats the alpha component in the images is unknown to the Vulkan API. Instead, the application is responsible for setting the composite alpha blending mode using native window system commands. If the application does not set the blending mode using native window system commands, then a platform-specific default will be used.”
  • KN: Do we need to list both opaque modes? Each platform only has one
  • CF: Could be useful to at least know which one you’re getting
  • KN: Think it’s per platform, could just document which ones are which and let users detect the platform themselves
  • Do that
• Premultipled vs postmultiplied?
  • KN: Both have very small support on native Vulkan http://vulkan.gpuinfo.org/listsurfacecompositealphamodes.php
  • Guessing usually you use Inherit for this?
  • Should we have postmultiplied/unpremultiplied?
    • CF: Apple has it, in fact it’s the only supported transparent mode (opaque NO). MoltenVK maps that to POST_MULTIPLIED
    • Keep it
  • naming: postmultiplied vs unpremultiplied vs straight-alpha
  • (discussion of overlays and underlays)
  • order and renames: Auto = 0 (Opaque→Inherit), Opaque, Premultiplied, Unpremultiplied, Inherit

[rajveermalviya]

Updated the proposal and #203 incorporates changes in the header.