search

AvaloniaUI / Avalonia

Develop Desktop, Embedded, Mobile and WebAssembly apps with C# and XAML. The most popular .NET UI client technology
https://avaloniaui.net
MIT License
25.81k stars 2.23k forks source link

Native Embeds don't render in control on Linux #6124

Open RatikKapoor opened 3 years ago

RatikKapoor commented 3 years ago

Describe the bug Following the NativeEmbed sample (https://github.com/AvaloniaUI/Avalonia/blob/master/samples/interop/NativeEmbedSample/EmbedSample.cs) results in mplayer working fine, but the file open dialog does not render inside the right pane as expected.

To Reproduce Steps to reproduce the behavior:

  1. Run the NativeEmbedSample on Linux (in this repo)

This same issue can be reproduced by using a .NET Core 5.0 app (the demo is using netcoreapp2.0)

Expected behavior The native control should be embedded in the Avalonia window control, rather than showing a clear background

Screenshots image

Desktop (please complete the following information):

Additional context I am trying to embed a GTK widget into Avalonia to make my Linux project feel seamless. My goal is to embed a webkit2 GTK widget so that I can have a built-in browser control in Avalonia on Linux that can use the system's webkit runtime. My testing code so far with webkit is:

public static IPlatformHandle MakeWeb(IntPtr parent)
        {
            if (s_gtkTask == null)
                s_gtkTask = StartGtk();
            if (!s_gtkTask.Result)
                return null;
            return RunOnGlibThread(() =>
            {
                var window = gtk_window_new(0);

                // gtk_window_set_default_size(window, 500, 500);
                var widget = Webkit.webkit_web_view_new();

                gtk_container_add(window, widget);

                gtk_window_set_title(window, new Utf8Buffer("hello there"));

                // gtk_window_set_transient_for(window, parent);
                Console.WriteLine(parent.ToInt64());
                // gtk_window_set_decorated(window, 0);
                gtk_widget_show_all(window);

                gtk_window_set_keep_above(window, 1);
                Webkit.webkit_web_view_load_uri(widget, new Utf8Buffer("https://google.com"));

                gtk_widget_realize(window);
                var xid = gdk_x11_window_get_xid(gtk_widget_get_window(window));

                return new FileChooser(window, xid);
            }).Result;
        }

This makes a new GTK window appear with the webview in it. Avalonia shows the same behavior as the screenshot however.

I have also tried to embed a simple GTK button, however I have not had much luck. This is about as far as I have gotten:

public static IPlatformHandle CreateTest(IntPtr parentXid)
        {
            if (s_gtkTask == null)
                s_gtkTask = StartGtk();
            if (!s_gtkTask.Result)
                return null;
            return RunOnGlibThread(() =>
            {
                using (var title = new Utf8Buffer("Embedded"))
                {
                    var window = gtk_window_new(0);
                    gtk_window_set_default_size(window, 500, 500);
                    // var widget = gtk_file_chooser_dialog_new(title, IntPtr.Zero, GtkFileChooserAction.SelectFolder,
                    //     IntPtr.Zero);
                    var widget = gtk_button_new_with_label(new Utf8Buffer("hello"));
                    // gtk_container_add(IntPtr.Zero, widget);
                    gtk_widget_realize(widget);
                    // var xid = gtk_widget_get_window(widget);
                    // gtk_window_present(widget);
                    gtk_widget_realize(widget);
                    return new FileChooser(widget, widget);
                }
            }).Result;
        }

These are both based off of https://github.com/AvaloniaUI/Avalonia/blob/74951318b17e2bda38ff101ac514bdb2375622ac/samples/interop/NativeEmbedSample/GtkHelper.cs#L38

Any help here would be greatly appreciated! I think being able to embed a browser into Avalonia would make it so much more powerful for things like authentication and whatnot!

Thank you!

kekekeks commented 3 years ago

First of all, note that we currently don't support XEmbed protocol, so keyboard focus won't properly work. Until it's implemented, it won't be possible to properly embed webkit-gtk anyway. Sorry about that. You can try calling XSetInputFocus manually though.

Why GTK widget is not visible is an interesting question. Are you, by chance, running a Wayland session?

RatikKapoor commented 3 years ago

Thanks for the quick reply @kekekeks! That makes sense for XEmbed. I hope that's something we can figure out, being able to embed a browser control cross-platform would certainly be a very powerful addition to Avalonia's controls.

I'm running an X11 session: loginctl show-session $(awk '/tty/ {print $1}' <(loginctl)) -p Type | awk -F= '{print $2}' gives me X11 as a result!

vpopescu commented 1 year ago

I know this is 2-year old issue, but since it's still open, I thought i'd ask, since it's '23 and I'm seeing the same thing. Is there a working sample showing how to embed a GTK control?

Ubuntu 22.04 / Wayland

jp2masa commented 1 year ago

There's a native embed sample in the control catalog for GTK:

vpopescu commented 1 year ago

I haven't been able to get the Native Embed sample to work in WSL. All the other samples work fine until I click on Native Embed. Once I click on Native Embed it displays an empty panel, and the other samples start rendering to empty panel as well.

EDIT: n/m, i eventually got it to crash and realized mplayer wasn't installed. After installing mplayer though, I see the same behavior as the OP:

image

Note how the bottom panel is not rendered, and it shows the console window from behind it.

vpopescu commented 1 year ago

I fount this other issue:

https://github.com/AvaloniaUI/Avalonia/issues/10354

And the answer by @kekekeks points in the right direction. After adding the XReparentWindow, i was able to display a NativeHostControl that contained an Avalonia button. Still having trouble display a native GTKWidget, I am unsure how to properly create it and make it a child of an Avalonia window.

But at least, some progress. And I do agree that Avalonia needs to make this simpler.

Another related issue: #7729

This asks to expose internals, and justifies why it's not done. However, I think having a bridge between Avalonia and native would be useful, otherwise we can run into synchronization issues.

vpopescu commented 1 year ago

Just to post an update, since I don't know if this is expected to work, but I can't embed gtkwebkit (2 years after the OP). It just managed to sigsegv gtkwebkit. Can't yet tell why, it seems to happen after the CreateNativeControlCore returns a valid XID.

Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f830202c220 - [sessionID=2] WebProcess::initializeWebProcess: Presenting processPID=8484
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f83020e8100 - [webPageID=10] WebPage::constructor:
Jun 23 16:51:49 mylinux-v5 dotnet[8484]: 0x7ee74200c700 - [PID=0, throttler=0x7ee74207c630] ProcessThrottler::Activity::invalidate: Ending background activity / 'WebProcess initialization'
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f8302028340 - [pageID=10, frameID=3, isMainFrame=1] FrameLoader::setPolicyDocumentLoader: Setting policy document loader to 0x7f82b2019000 (was (nil))
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f82b2019000 - [pageID=10, frameID=3, isMainFrame=1] DocumentLoader::DocumentLoader::attachToFrame: m_frame=0x7f83020a0d00
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f8302028340 - [pageID=10, frameID=3, isMainFrame=1] FrameLoader::setProvisionalDocumentLoader: Setting provisional document loader to 0x7f82b2019000 (was (nil))
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f8302028340 - [pageID=10, frameID=3, isMainFrame=1] FrameLoader::setDocumentLoader: Setting document loader to 0x7f82b2019000 (was (nil))
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f8302028340 - [pageID=10, frameID=3, isMainFrame=1] FrameLoader::transitionToCommitted: Clearing provisional document loader (m_provisionalDocumentLoader=0x7f82b2019000)
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f8302028340 - [pageID=10, frameID=3, isMainFrame=1] FrameLoader::setProvisionalDocumentLoader: Setting provisional document loader to (nil) (was 0x7f82b2019000)
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f82b2019000 - [pageID=10, frameID=3, isMainFrame=1] DocumentLoader::startLoadingMainResource: Returning empty document
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f83020e8100 - [webPageID=10] WebPage::unfreezeLayerTree: Removing a reason to freeze layer tree (reason=1, new=0, old=0)
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f83020e6a30 - [webFrame=0x7f8302004360, webFrameID=3, webPage=0x7f83020e8100, webPageID=10] WebFrameLoaderClient::completePageTransitionIfNeeded: dispatching didCompletePageTransition
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: 0x7f8302028340 - [pageID=10, frameID=3, isMainFrame=1] FrameLoader::setState: main frame load completed
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]: Memory usage info dump at MainFrameLoadCompleted:
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]:   javascript_gc_heap_capacity_mb: 0
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]:   page_count: 1
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]:   document_count: 1
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]:   javascript_gc_heap_extra_memory_size_mb: 0
Jun 23 16:51:49 mylinux-v5 WebKitWebProcess[8521]:   backforward_cache_page_count: 0
Jun 23 16:51:49 mylinux-v5 dotnet[8484]: 0x7ee74207c480 - [PID=8521] WebProcessProxy::didClose (web process crash)
Jun 23 16:51:49 mylinux-v5 dotnet[8484]: 0x7ee74207c480 - [PID=8521] WebProcessProxy::shutDown:
Jun 23 16:51:49 mylinux-v5 dotnet[8484]: 0x7ee74207c480 - [PID=8521] WebProcessProxy::processWillShutDown:
Jun 23 16:51:49 mylinux-v5 dotnet[8484]: 0x7ee74207c480 - [PID=8521] WebProcessProxy::processTerminated:
Jun 23 16:51:49 mylinux-v5 WebKitNetworkProcess[8514]: 0x7fec400cc0e0 - [webProcessIdentifier=11] NetworkConnectionToWebProcess::didClose: WebProcess closed its connection. Aborting related loaders.
Jun 23 16:51:49 mylinux-v5 WebKitNetworkProcess[8514]: 0x7fec400cc0e0 - [webProcessIdentifier=11] NetworkConnectionToWebProcess::unregisterSharedWorkerConnection:
Jun 23 16:51:49 mylinux-v5 WebKitNetworkProcess[8514]: WebSharedWorkerServer::removeConnection(0x7fec40014840): processIdentifier=11
Jun 23 16:51:49 mylinux-v5 WebKitNetworkProcess[8514]: 0x7fec40014840 - [webProcessIdentifier=11] WebSharedWorkerServerConnection::~WebSharedWorkerServerConnection:
Jun 23 16:51:49 mylinux-v5 WebKitNetworkProcess[8514]: 0x7fec400cc0e0 - [webProcessIdentifier=11] NetworkConnectionToWebProcess::unregisterSharedWorkerConnection:
Jun 23 16:51:49 mylinux-v5 kernel: WebKitWebProces[10115]: segfault at 0 ip 00007f83233a0133 sp 00007fffbd33aba0 error 4 in libwebkit2gtk-4.0.so.37.57.9[7f8322bfa000+2688000]
Jun 23 16:51:49 mylinux-v5 kernel: Code: 0f ef c9 f3 0f 2a 4c 24 24 89 44 24 20 f3 0f 59 c8 f3 0f 2c c1 89 44 24 24 e8 99 2c fc ff f3 0f 59 85 88 00 00 00 48 8b 7d 60 <48> 8b 07 f3 0f 11 44 24 0c ff 50 30 48 8b 55 10 f3 0f 10 44 24 0c
Jun 23 16:51:49 mylinux-v5 kernel: potentially unexpected fatal signal 11.
Jun 23 16:51:49 mylinux-v5 kernel: CPU: 4 PID: 10115 Comm: WebKitWebProces Not tainted 5.15.90.1-microsoft-standard-WSL2 #1
Jun 23 16:51:49 mylinux-v5 kernel: RIP: 0033:0x7f83233a0133
Jun 23 16:51:49 mylinux-v5 kernel: Code: 0f ef c9 f3 0f 2a 4c 24 24 89 44 24 20 f3 0f 59 c8 f3 0f 2c c1 89 44 24 24 e8 99 2c fc ff f3 0f 59 85 88 00 00 00 48 8b 7d 60 <48> 8b 07 f3 0f 11 44 24 0c ff 50 30 48 8b 55 10 f3 0f 10 44 24 0c
Jun 23 16:51:49 mylinux-v5 kernel: RSP: 002b:00007fffbd33aba0 EFLAGS: 00010206
Jun 23 16:51:49 mylinux-v5 kernel: RAX: 00007f83020e8640 RBX: 00007f8302104110 RCX: 0000000000000006
Jun 23 16:51:49 mylinux-v5 kernel: RDX: 0000000000000017 RSI: 00007fffbd33abc0 RDI: 0000000000000000
Jun 23 16:51:49 mylinux-v5 kernel: RBP: 00007f8302163b80 R08: 000055fd410728c0 R09: 0000000000000007
Jun 23 16:51:49 mylinux-v5 kernel: R10: 0000000000000001 R11: ba06264942697264 R12: 00007fffbd33abc0
Jun 23 16:51:49 mylinux-v5 kernel: R13: 00007f8302163bf0 R14: 00007f8302163c50 R15: 00007fffbd33b240
Jun 23 16:51:49 mylinux-v5 kernel: FS:  00007f831b864ac0 GS:  0000000000000000

my stripped down code in CreateNativeControlCore is something like follows. Either I have a sequence error, or there is something about not running on the right thread.

var xid = GtkInteropHelper
        .RunOnGlibThread(() =>
        {
            _win = gtk_window_new(GTK.WindowType.Toplevel);
            _view = new WebKit.WebView();

            gtk_container_add(_win, _view.Handle);

            var display = gdk_x11_display_get_xdisplay(gtk_widget_get_display(_win));
            XSync(display, false);

            Thread.Sleep(10);
            gtk_widget_realize(_win);

            var xwin = gdk_x11_window_get_xid(gtk_widget_get_window(_win));
            _ = XReparentWindow(
                display,
                xwin,
                ParentWindow.PlatformImpl.Handle.Handle,
                0,
                0
            );

            XSync(display, false);
            Thread.Sleep(10);

            gtk_widget_show_all(_win);
            gtk_widget_grab_focus(_view.Handle);
           webkit_web_view_load_uri(_view.Handle, Uri);
            return xwin;
        })
        .Result;
return xid;

And yes, I can also reproduce it on "real" Ubuntu on baremetal.

vpopescu commented 1 year ago

So, just to close on this, I'm not yet sure who is at fault, but it actually works if I compile my own libwebkit2gtk.xx.so with default build switches. It just doesn't work with the in-box one on Ubuntu22.04.

npolyak commented 10 months ago

any ideas when it will be addressed?

DoubleDBE commented 3 months ago

@RatikKapoor Were you ever able to solve this? I'm having the exact same issue.

Trying to use WebKit as an embedded control, shows nothing...

If this could get fixed, that'd be great.

cangunaydin commented 2 months ago

i have the sample problem. i have opened a discussion about it, but couldn't move forward. https://github.com/AvaloniaUI/Avalonia/discussions/16578 any idea how to fix this issue?