Sharing textures doesn't work on macOS/CGL

[kvark]

Here is what gfx-backend-gl tries to do:

1. create one GL context ("root") without a surface, most operations are done on that context by rendering into textures.
2. create an additional GL context for the window/surface. When it's time to present, we want to blit from a renderbuffer of the root context to this context. Then we present the surface.

This is needed in order to detach the "Instance" we have from the "Surface". In order to work properly, it requires that a certain renderbuffer is shared between the GL contexts (rendered into by "root", read by "surface".

We were struggling to get this to work with @zicklag, and we found out that CGL context is created without any sharing. I experimented with a surfman API that would allow sharing contexts in this branch and confirmed that this would work.

@pcwalton is this a bug in the code, feature not yet implemented (sharing), or a limitation by design?

Note: it is very much desired to avoid the blit altogether, and just be able to render to a surface from the "root" context. But apparently this isn't supported by surfman, at least now. We are looking forward to see if this becomes an option.

[zicklag]

@kvark Did the changes in your surfman branch fix the issue with the surfman backend in GFX?

[kvark]

Yes, see https://github.com/gfx-rs/gfx/pull/3216#issuecomment-625322960 Here are the corresponding backend changes. It creates another context for the Instance, which is shared with both adapter contexts and surfaces.

[pcwalton]

My goal was to avoid any GL context sharing to avoid locking and driver bugs. (Servo used to use sharing and it had a lot of problems.) It's an easy trap for people who aren't as experienced with graphics APIs to use GL context sharing and make a mess.

But for advanced use I could see how sharing could be useful, as long as the documentation warns against it…

[zicklag]

I don't know how this works much, but would texture sharing be a safe alternative that would work for our GFX use-case? Could we share the rendered texture instead of the GL Context?

[kvark]

If there is a way for us to get that use case I described without sharing contexts, I'd love to try it out. Otherwise, we'll proceed with a PR adding the sharing capability to surfman.

[zicklag]

I can't tell what's going on, but maybe that is similar to what's going on in the threads example somewhere around here:

https://github.com/pcwalton/surfman/blob/ae14cc9afd140bc73e89f3fe8c080f729f09ccd2/surfman/examples/threads.rs#L218-L225

It looks like that is sending the surface around over a channel, maybe not the texture. I don't know if that would help.

[pcwalton]

You should be able to detach surfaces from contexts and send them over channels; that's how the threads example works. I'm not sure whether this works for your use case?

[kvark]

@zicklag That function allows a different context to read the texture of a surface. This is indeed a limited form of surface/texture sharing that doesn't involve the whole context sharing. However, the key limitation here is read, so we can't share the final surface texture with it (since we want to write to it). So, if I understand correctly, it's not useful to our case.

[zicklag]

Can we create a context and a surface associated to that context, render to it, then send that surface to the thread that owns the window handle's surface, and then create a texture ( read only ) from the rendered surface that we can copy to the window's surface?

[pcwalton]

@zicklag Yes, that should work.

BTW, you may find @asajeffrey's surfman-chains useful, if not to use, than at least to look at as a reference: https://github.com/asajeffrey/surfman-chains It builds swap chains on top of surfman. Servo uses it for compositing WebGL/WebXR content.

[kvark]

@zicklag I don't see how this would solve our case. Would that surface be associated with the swapchain? There is no way to get that without sharing the resources with whatever else there is on the device.

[zicklag]

@pcwalton surfman-chains looks promising for sure.

@kvark I don't really know. I never fully grasped how the Vulkan-ish GFX API was going to use the surfaces and such.

It feels like we're just going to have to put the context that we write to in an RwLock to enforce single-thread-at-a-time access to whatever is doing the rendering and avoid any sharing. Then, using the surfman-chains project we can create a swap chain. According to the surfman-chains doc:

Each swap chain has a producer context, responsible for creating and destroying surfaces, and a number of consumer contexts, (usually just one) which take surfaces from the swap chain, and return them for recycling.

So the producer context would be the context that we render to. That context would have to be behind an RwLock. The Window would be the consumer context. I can't remember how the Swapchain relates to the window surface in GFX...but...if I remember right the swapchain has the window surface associated to it and it would be responsible for grabbing the next frame in the swap chain and displaying it on the window surface.

I might be missing something. Does that make any sense?

[kvark]

Nope, sorry, I don't think it's going to work like that.

When the GL backend is given a stream of commands (i.e. a list of command buffers at submission), it doesn't have any association of them to any surface. They might be rendering to some swapchain, or multiple, or not any at all. This is where we are producing the contents of the swapchain images, and we can't use their personal contexts for that. We can only render to some buffer that can be blit onto the surface, but that requires that the surface sees that buffer, and this is exactly the context sharing we are missing.

[pcwalton]

Reading your initial comment, I think I understand the problem: you want to share a plain old GL renderbuffer over to another GL context. The workflow surfman was designed for is this: you don't render to a plain GL renderbuffer object, but rather to a surface created specially through surfman, which you can then share over to another thread. Is there a reason why you need to render to a GL renderbuffer as opposed to a surfman surface in the first place?

(I'm not saying no to sharing, I just want to understand why it's necessary.)

[pcwalton]

Note that this is similar to the way that DXGI requires that you pass a special SHARED flag when creating a surface to be shared between threads/processes, rather than just sharing an ordinary D3D texture, and in fact DXGI shared surfaces are the implementation surfman uses on Windows.

[kvark]

Is there a reason why you need to render to a GL renderbuffer as opposed to a surfman surface in the first place?

Imagine that there is a Device, and all the resources the user creates belong to this device. It includes shaders, buffers, textures, and other things. Let's say that command buffers are also created from the device and submitted for execution on it, for simplicity. Some of the work that the user does may need to render to some surface. How do you suggest us to render to this surface? The work of rendering to it depends on many resources the user has created on the device. The device is created without any knowledge of surfaces (and thus, any resources on it). So how does rendering to a surface get access to the resources used in that rendering? I only see sharing as an option.

Now, maybe there isn't need for a blit there under certain conditions. Maybe we can just identify that render pass that targets the surface, and switch to the surface context for it. This isn't really relevant to the subject here (since the sharing is still needed), and it would require the following conditions:

• that render pass doesn't render to multiple surfaces at once
• the context of a surface can modify resources of the device context (e.g. with load/store operations, or by have a depth-stencil attachment from it), OR
• some guarantee that this particular pass only reads the resources

[zicklag]

What if we put all the resources in one thread, and "remote controlled" that thread with messages sent over a channel for every operation we needed to make to the graphics resources.

Do we still need shared contexts if everything happens in one thread?

[kvark]

This isn't about threading at all. In fact, we are pretty much doing what you described today: the command buffers are recorded without touching any GL contexts, and we actually issue command on queue submit(), which typically happens on one thread. The problem is that no single context has access to all resources as well as the target surface(s), if we don't have sharing.

[zicklag]

So it is sounding like we need sharing, and surfman may or may not grant us a lot of extra advantage over gluten except for its lack of a dependency on winit?

[kvark]

Yep. if glutin were able to work with RWH instead of being locked to winit, we'd just use it. Anyhow, the glutin discussion is largely off-topic in this issue.

[zicklag]

OK, so that makes this still relevant to the work we are doing for GFX and it means that we will also need PRs for the MacOS, X11 (, XCB? ), and Wayland to enable sharing in surfman.

I'm not sure that I can do that one, or that I'll find the time very soon. I haven't looked at @kvark 's experiment branch yet, but if it is simple enough then I might be able create PRs for X11 when I find the time. If sharing support lands in surfman then I think our surfman backend for gfx doesn't really need changes other than to enable the sharing.

Edit: Oh, sorry this probably should have been on the other thread.

[kvark]

It seems to me that @pcwalton isn't opposed to sharing, so the next step for us could be turning this work into a PR that can be discussed here.

[zicklag]

OK, that doesn't look super hard. Whenever I next have time I'll probably check it out, if nobody beats me to it.

[asajeffrey]

Looking over https://github.com/servo/surfman/pull/185 it looks like a good implementation of context sharing, the question is whether we need it? In the other cases where we need to step outside the surfman/surfman-chains model, I've done it by using (e.g.) NativeContext and create_context_from_native_context (https://github.com/servo/surfman/blob/master/surfman/src/device.rs#L58-L60) Could we do that in this case too?

[kvark]

Not sure exactly what you are suggesting, @asajeffrey . Isn't the whole point of surfman to abstract away the context? Why would we want to use NativeContext with surfman? We might as well just reimplement surfman-ish thing then... unless I'm totally missing your point.

[asajeffrey]

I guess I am equally confused :/ The Native* APIs are the "escape hatch" for "surfman's model mostly fits my application, but I need to interoperate with something else", e.g. for gstreamer we need to use its GL APIs, which we do via Native*. I'm wondering whether we could use NativeContext to create a GL context with sharing, then construct a surfman Context from it?

Even better would be to work out if there's a way to use surfman in gfx without texture sharing, but this might be tricky as surfman/surfman-chains is based on each Surface having a producer context and a number of consumer contexts which only get read access. It sounds like this is a different model than gfx uses?

[asajeffrey]

Chat at https://chat.mozilla.org/#/room/#gfx:mozilla.org/$wg5m0QokgZulwv-lx8eAKSWSC-yjbcXJzrR0o17ytR4

tl;dr: we might be able to lift the restriction that surfman Surfaces have an associated producer Context that has draw access. This restriction was needed for GLX, but that's not a supported platform any more.