Rendering issues when having multiple WPF Windows with seperate GLWpfControls

[dedmen]

Ref: https://github.com/mono/SkiaSharp/issues/745#issuecomment-805199563 Copy of my comments from there and what i tried.

But it does have some problems, it doesn't really seem to like rendering two dock windows (AvalonDock) it gets confused. Some elements suddenly stop showing up, swapping back and forth when zooming. When I hide my grid element (basically just rendering a bunch of lines) it partially comes back, but other things bug out.

Definitely related to GL, if I swap back to SKElement it renders normally again. I'm using one SKGLWpfControl per window, and it messes up pretty badly. If I use SKElement for both it works fine. Hacky workaround, use SKGLWpfControl for the first Map window, and SKElement for all further created Map windows. https://github.com/WolfCorps/TacControl/blob/bba4d364548fc18a29d29b2138a8e44de619f7d2/src/DesktopClient/Misc/MapControl.cs#L547-L559 Works nicely too, but of course is missing the performance improvement on the second window.

https://github.com/WolfCorps/TacControl/commit/a90164a9e6c1dc91091d55982270a42f4d46d338 Here is my code, didn't make any changes to SKGLWpfControl itself. I assume the issue is inside OpenTK.GLWpfControl

Edit: I thought this might be the problem. https://github.com/opentk/GLWpfControl/blob/0e657c1033944fbc6b6e7fc53694a7aae139dadc/src/GLWpfControl/DXGLContext.cs#L115 It shares the same context, for both windows, which it seems like it shouldn't. But even using mainSettings.ContextToUse to select seperate context for both windows, doesn't help.

Changing the code to manually get the correct window handle via Application.Current.Windows, also not better. Two contexts in two seperate windows will confuse eachother. Out of ideas at this point, but I also don't really know GL nor any of this code.

I don't have time currently to make a simpler repro, without maybe SkiaSharp/Mapsui/AvalonDock.

[najak3d]

@NogginBops - This one is huge for us. We've got an WPF application on the PC on .NET 4.8, where we just can't seem to get more than one OpenGL Context to work, without severe/random crosstalk. I've spent all night experimenting trying to get it to work, and eliminate the crosstalk (from user standpoint, mucking with using the GLWpfControl "as is".

I tried creating a normal GLControl, MakeCurrent(), and creating a GL Context, and from that an SKSurface.... but the Context of that Surface has severe crosstalk with the GLWpfControl.

What we need most here is to be able to create "Layers", such that we can use Skia to create more-than-one SKCanvas bound to the GPU -- and then use "canvas.DrawCanvas(otherCanvas, 0, 0)" to the main canvas that belongs to the GLWpfControl.

It doesn't work out well for us to have all Skia draw calls (and shaders) operate directly upon the GLWpfControl surface.

As of now, the only way we can employ this layered-approach, is if the other Surfaces are bound to CPU-based memory, not GPU/OpenGL. This is much slower performance.

Is there anything we can do to help prioritize this work? We're stuck and spinning our wheels now.

[NogginBops]

@najak3d if you give me a reproducible project I can take a look at what is going on.

[najak3d]

Working on it now. Starting with dedmen's zip file for "SKGLWpfControlExample.zip" -- and adding in a 2nd surface on the GPU (attempting to), and then using SkiaSharp to draw to the 2nd surface, and then DrawSurface to the main GLWpfControl canvas/surface. See if this raw project behaves same as our business project. Aiming to be done within the hour.

[najak3d]

Here it is. The test edits are all in "Example/MainWindow.xaml.cs" the "DrawOtherSurface()" method.

Here I oscillate every 3 seconds between 3 modes:

1. DirectDraw - works fine -- draws to the Main GLWpfControl's surface, not the "otherSurface".
2. CopyFromOther - renders an oscillating image to _otherSurface, then does "DrawSurface()" to the MainControl's surface... this just renders a FROZEN/STATIC image, and does not oscillate. So my draw to "otherSurface" only works first time.
3. NoCopy - same as # 2, but I do NOT do the 'DrawSurface' call... and this exhibits CrossTalk! - here drawing to _otherSurface causes changes to the MainControl's Surface!

==== All said - I'm doing my draws and contexts were created on the SAME thread. As I was creating this example for you, I noticed some summaries that indicated I might need to do my rendering on different threads, ot have different contexts... so perhaps this was my whole problem? (going to test this next)

SKGLWpfControlExample-TwoContexts.zip

[najak3d]

OK - I tried creating and Drawing to the _OtherSurface on a new Thread -- and this still failed, but did eliminate the cross talk.

With this separate thread:

1. I needed to Create GLControl() - to create a new Context.
2. But when I try to DrawSurface to the MainControl's surface - it has NO EFFECT.

Here is the one-file I changed with these edits.

MainWindow.xaml.zip

In "DrawOtherSurface()" - I forced "NoCopy" mode, so it simply renders to the _OtherSurface (offscreen).

Then inside "sk_PaintSurface" I added this call: canvas.DrawSurface(_otherSurface, 0, 0);

Which had no effect.

[NogginBops]

@najak3d I'm not 100% sure how the opengl concepts such as context and framebuffers map to Skia, so I might not be 100% correct here. But I think I've figured out parts of this.

GLWpfControl works by utilizing DX interop to be able to transfer the OpenGL drawing results to DirectX (which wpf uses). To do this we have a single OpenGL context running that renders all of the contents, to change where we are rendering each GLWpfControl has it's own framebuffer (that is linked to a DirectX framebuffer) that gets set as the current framebuffer before each paint event.

I found an issue in your code where you created a new GRContext while you should be using the same GRContext for basically all operations. Fixing that made the CopyFromOther case work as expected (although possibly quite slow).

I'm not sure how to see what is crosstalk and what is not as I'm not sure what is supposed to be drawn here to be honest. Is the issue that stuff you are drawing to the SKElement is showing up in the SKGLWpfControl?

[najak3d]

I found an issue in your code where you created a new GRContext while you should be using the same GRContext for basically all operations. Fixing that made the CopyFromOther case work as expected (although possibly quite slow).

Your fix worked! (i.e. changing to "_context = sk.GRControl;" instead of creating a new context).

I got the strong notion from other posts that we needed a separate GL Context to prevent crosstalk, so wasn't questioning that decision, ever. But for this, that was the culprit!

One major hurdle knocked down -- and performance is EXCELLENT. I'm blitting a 2000x1000 surface to another, 100x per frame (put in a for loop) - and it still runs at 60 FPS, the throttle). Looks like true GPU performance here, no bottleneck.

The other hurdle, that we can tolerate is the ability to have More-than-One of these SKGLWpfControls in the app at one time. This continues to have crosstalk, out of box (per the issue stated by @dedmen above.

=== And to clarify, there is ZERO crosstalk between SKElement and the SKGLWpfControl.... none at all. This is how we can get around our 2nd issue -- we only have ONE critical need for a surface with Custom Skia Shaders -- and so we will simply make all of the rest of our popup forms be SKElements.

Our entire UI is similar to Avalonia UI -- our app's windows are each comprised of one-big-SKGLWpfControl, and our UI controls are all rendered using graphics calls to this Image. Each of our popup windows creates it's own new SKGLWpfControl.... and due to the crosstalk, we are currently forced to use SKElement for these popups, and render our UI on the CPU, not GPU. We can make this work, though.

[NogginBops]

Hm, it should definitely be possible to show two GLWpfControls at the same time, if you want me to look into that too I can do it if you are able to produce a repro. 🙂

[najak3d]

AFAIK, the whole point of this open issue from @dedmen , is exactly that... having two SKGLWpfControls has major cross talk. I get around it by only having ONE, and using SKElement for the rest of my forms/surfaces.

Tomorrow, I will produce for a small sample, that demonstrates the issue. (Two on one Window, and also, having another one in a popup-window.)

[najak3d]

Here I've modified the previous project by changing the Right-Side SKElement into a SKGLWpfControl. Now there is rampant crosstalk.

GOOD: This first image shows what it SHOULD look like, if there wasn't crosstalk (this image is a composite of two runs -- one where I'm rending only to LeftSide, and 2nd only to RightSide -- so that there is no crosstalk):

CROSS-TALK!: But when I render to Both sides (two SKGLWpfControls) - I instead looks like this - demonstrating rampant CrossTalk:

To toggle between the three modes, just swap which line is uncommented at the top of "MainWindow.xaml.cs":

        // NOGGINS - uncomment just one of the lines below.
        //private const DrawModes Mode = DrawModes.LeftSide; // GOOD - Draws Flashing Colors image
        //private const DrawModes Mode = DrawModes.RightSide; // GOOD - Draws Static image with Frame Count text
        private const DrawModes Mode = DrawModes.BothSides; // BAD - CROSS TALK!!

Thank you for your kind time and attention! I feel as though I'm being blessed by the gods.

SKGLWpfControl-CROSSTALK.zip

[TomArrow]

I'm not sure if mine is the same problem but I have two windows with separate GLWpfControls and if I create both windows, everything becomes extremely sluggish. The WPF GUI stops reacting properly and when I press any button (like an arrow key) I get a stack overflow exception (no stack trace available, all external code) and the program crashes.

In both windows the control has the same x:Name, is that a problem? It's a different window/class though, so they shouldn't conflict. Is there anything special I need to do to make this work? I just have the control in the .xaml and then

var settings = new GLWpfControlSettings
            {
                MajorVersion = 2,
                MinorVersion = 1,
                RenderContinuously = false,

            };
            OpenTkControl.Loaded += OpenTkControl_Loaded;
            OpenTkControl.Start(settings);

in the window constructors after InitializeComponent()

I also have some framerate limiting in the OnRender like this:

GL.MatrixMode(MatrixMode.Projection);
            GL.LoadIdentity();

            double timeSinceLast = (DateTime.Now - lastUpdate).TotalMilliseconds;
            //if (timeSinceLast < minTimeDelta) System.Threading.Thread.Sleep((int)(minTimeDelta- timeSinceLast));
            if (timeSinceLast > minTimeDelta) ;
            else return;
            GL.ClearColor(Color4.White);
// rest of drawing code

            lastUpdate = DateTime.Now;

Edit: The sluggish performance was unrelated I guess, I had an invalidatevisual after if (timeSinceLast > minTimeDelta) instead of a no op. But the crash still happens regardless even if the performance seems ok. I press a key like an arrow key, and there's an immediate Stack overflow crash

Edit: Nvm this is "solved" with one of the fixes from another issue. Forgot to check.