search

microsoft / WindowsAppSDK

The Windows App SDK empowers all Windows desktop apps with modern Windows UI, APIs, and platform features, including back-compat support, shipped via NuGet.
https://docs.microsoft.com/windows/apps/windows-app-sdk/
MIT License
3.84k stars 320 forks source link

WebView2 blocks mouse input to the application after typing too fast #3266

Closed ghord closed 1 year ago

ghord commented 1 year ago

Describe the bug

WebView2 freezes mouse input when typing too fast.

Steps to reproduce the bug

  1. Create new application with WebView2 control
  2. Set source to any page with input (google/bing)
  3. Type in input very fast
  4. Mouse input to whole app freezes (notice unchanging cursors). Only keyboard input continues to work.

Expected behavior

WebView2 continues to work normally

Screenshots

Animation

NuGet package version

Windows App SDK 1.2.1: 1.2.221116.1

Packaging type

Unpackaged

Windows version

Windows 11 version 22H2 (22621, 2022 Update)

IDE

Visual Studio 2022

Additional context

This does reproduce on both packaged and unpackaged apps.

This does not reproduce on versions 1.1.x.

This issue occurs in every 1.2.x version I've tried.

applefanbois commented 1 year ago

We have this exact problem

applefanbois commented 1 year ago

The problem is still there in Microsoft.WindowsAppSDK 1.2.221209.1

Eilon commented 1 year ago

To repro this in a trivial WinUI3 app:

  1. Create a new C# WinUI3 project in VS
  2. Change MainWindow.xaml to have just a WebView2: 
    <WebView2 x:Name="wv2" ></WebView2>

  3. Change MainWindow.xaml.cs to have this code:

        public MainWindow()
    {
        this.InitializeComponent();

        wv2.CoreWebView2Initialized += Wv2_CoreWebView2Initialized;

        wv2.EnsureCoreWebView2Async();
    }

    private void Wv2_CoreWebView2Initialized(WebView2 sender, CoreWebView2InitializedEventArgs args)
    {
        wv2.NavigateToString(@"
<html>
<body>
<h1>Hello, world!</h1>

<div class=""container"">
    <div class=""row"">
        <div class=""col"">
            <input class=""form-control"" type=""text"" />
        </div>
    </div>
    <div class=""row"">
        <div class=""col"">
            <input class=""form-control"" type=""text"" />
        </div>
    </div>
</div>
</body>
</html>
");
    }
  4. Run the app and start typing in the two textboxes while using the mouse to click back and forth between them

Eventually you'll still be able to type, but clicking with the mouse won't work.

Can't really tell from the screenshot, but I can assure you that keyboard/typing works, yet I can't click on the textboxes (it ignores the clicks):

image
baaaaif commented 1 year ago

we're affected too --> upvote...

applefanbois commented 1 year ago

Do they even test it more than 2 seconds before releasing it?

To be fair, crApple is not better, last year the macOS WKWebview was crashing all the times and it took 8 months before they fix it. This year Apple WKWebview on macOS Intel will never trigger the onKey events, so every website on the planet with a text input will not work.

It just proves that big corps don't test before releasing and not only the programmers suffers, but in the end their customers suffer from their lack of respect. Just 5 minutes of using their own tools would show them that it does not work at all.

AdriaanLarcai commented 1 year ago

We are also affected by this problem and our implementation of the Monaco editor. We cannot go live with this issue. Please resolve this ASAP.

lgBlog commented 1 year ago

Is there any progress on the problem?

Xyncgas commented 1 year ago

This would've made bill gates poor back in the days

applefanbois commented 1 year ago

Bill Gates best contribution to the world is not Windows (a cheap copy of CPM and YES windows is CPM not Unix based), but the BASIC rom chip (he was the one who programmed it) that was included in every C64, ZX Spectrum, Tandy COCO, and Atari 8bit computers. Bill was a true programmer, not some XML pretentious scripters. BASIC was the revolution that made the 80's kids and normies do stuff and become programmers.

Xyncgas commented 1 year ago

Bill was a true programmer

Sorry I didn't know better, thanks for the information given

codendone commented 1 year ago

Tracking with internal bug.

Xyncgas commented 1 year ago

I suspect it's because they are using Task based mechanism to handle user input then send it to the browser like a proxy, and they use some kinds of mutex or lock that doesn't get released properly. If that's happening I would like to input directly to the browser without this extra layer and if we are using native control with webview then we cover the browser layer with the native layer but no proxy-ing (unless it's needed?)

applefanbois commented 1 year ago

The design of the MS framework is baffling and insane. Everything is async so everything becomes 400x slower than anything on Android and Apple because of the useless context switching (only I should decide what is async). And the biggest sin of the computer world : MS are a huge fan of thread re-entrance where our code running on the main thread will be pause execution mid-way (by MS) and another main thread task from MS will trigger again the same code and complete and then the code that was running will resume.

Instead of putting the next task on the main thread queue, they pause our main. This is criminally insane and dangerous. Another problem is that every MS function can throw and it is never mentioned in the documentation. Worst most of the time you will get unhandled exception inside a try catch block.

Not only it is painful to me an my customers because it require 100x more work to make the app just not crash every 2 minutes, but everything made by MS will be unstable, including WebView2 that crash every seconds.

There are so many wait, dispatch and useless validation just so the app won't crash every second. Anytime you call a MS function, you risk crashing the app. But getting so many unhandled exception inside try catch block is maddening.

Stop throwing MS, just make your code work. There is even thread re-entrance inside the MS library. Found that if you call the HttpClient thousands of time over 1 hour, eventually, deep inside MS code, there will be thread-re-entrance where the same line of code is executed twice and that will crash the app.

Because it is never mentioned anywhere on the internet that anything you do can be interrupted by MS and then called multiple times while the other did not even complete. You have to be paranoid and add a if (isFinishedExecuting) everywhere absolutely everywhere at the beginning of every functions. And if you do log the name of every function, you will see that everything is out of order.

I draw the buttons myself with Win2D, everything is async and waiting. Because of that drawing a button on windows is 400x slower than on a Mac. The app on iOS and macOS is rendering at 4x the resolution (oversampling) and it runs at 60fps. On windows, I had to render 1:1 and disabled animation. Everything is stuttering on MS, nothing is smooth. My pc has a 2080ti and the apple product have cheap onboard gpu. Microsoft design has a cost in speed and in stability and we are the one paying for it.

Xyncgas commented 1 year ago

How come cloud gaming doesn't have this problem

KieranDevvs commented 1 year ago

Tracking with internal bug.

Internal tracking is great... but can the users please get an update?

alelom commented 1 year ago

Just reporting another instance of this bug that is completely deal-breaking for my application developed in MAUI Hybrid.

ctigrisht commented 1 year ago

We can't develop our app and was forced to put the windows version of the MAUI blazor project on hold, forced to use it on android device only.

Xyncgas commented 1 year ago

We can't develop our app and was forced to put the windows version of the MAUI blazor project on hold, forced to use it on android device only.

.NET 6 still works On another environment, copy your project and change it all from .NET 7 to .NET 6, then wait for the problem fixed. You can update client on windows once it's fixed

ctigrisht commented 1 year ago

You can update client on windows once it's fixed

It says "fixed internally", does that mean a public fix is coming soon? thanks?

KieranDevvs commented 1 year ago

We can't develop our app and was forced to put the windows version of the MAUI blazor project on hold, forced to use it on android device only.

.NET 6 still works On another environment, copy your project and change it all from .NET 7 to .NET 6, then wait for the problem fixed. You can update client on windows once it's fixed

Its not that easy, there are a lot of features and API's that were introduced in .NET 7 that people are using. It turns into a migration project.

Xyncgas commented 1 year ago

This bug actually seems very interesting, can you guys tell a story on how it was fixed, what went wrong was it hard

applefanbois commented 1 year ago

I bet that it was threading problem. Same code executing again while the previous one has not completed. They should use more TryEnqueue() and put the next job on the thread queue so it will be executed when the thread is free.

KieranDevvs commented 1 year ago

This bug actually seems very interesting, can you guys tell a story on how it was fixed, what went wrong was it hard

Lets just focus on actually getting the bug fix first.

@gabbybilka What's the point of having an open source community with issue tracking if you're just going to track it internally and never respond to the public?

applefanbois commented 1 year ago

Open Source does not always mean the Richard Stallman (St-iGNUsius) definition of open source. Anything can be open source with a good disassembler I guess.

Open Source is just a generic feel good name like democracy and peace, but in practice it means quiet the opposite.

Xyncgas commented 1 year ago

Open Source does not always mean the Richard Stallman (St-iGNUsius) definition of open source

Richard said was free is not open source, former is better

Anything can be open source with a good disassembler I guess

it's way easier to monitor and modify with 'source', server still is a black box, closer to hardware then you need special methods like mEMR to probe electron bits in ram

codendone commented 1 year ago

This bug actually seems very interesting, can you guys tell a story on how it was fixed, what went wrong was it hard

This was a multi-threaded multi-HWND issue. Mouse/pointer input is delivered to a background thread (to enable things like smooth touch panning of ScrollViewers) while keyboard input still goes to the UI thread. These need to be synchronized. WebView2 introduces an extra HWND for keyboard input, and this caused problems in the synchronization code.

Xyncgas commented 1 year ago

This was a multi-threaded multi-HWND issue. Mouse/pointer input is delivered to a background thread (to enable things like smooth touch panning of ScrollViewers) while keyboard input still goes to the UI thread. These need to be synchronized. WebView2 introduces an extra HWND for keyboard input, and this caused problems in the synchronization code.

That's amazing. Thanks for fixing this bug.

true-perfect-code commented 1 year ago

Then through which way the update comes to us developers (Windows, Visual Studio, NET MAUI...)?

Thanks

ManbirSinghRakhra commented 1 year ago

When will you release fix?

true-perfect-code commented 1 year ago

@codendone Any news about an update possibility?

applefanbois commented 1 year ago

no

codendone commented 1 year ago

I don't yet have a release timeline to share beyond we'd like it to be soon. The fix is currently going through extra testing.

true-perfect-code commented 1 year ago

Will the update be available to us when a new nuget version of Microsoft.Web.WebView2 is released here?

See https://www.nuget.org/packages/Microsoft.Web.WebView2

Ewerton commented 1 year ago

By the way, is there a way of force my .Net Maui App to use a specific version of Microsoft.Web.WebView2?

vallgrenerik commented 1 year ago

@Eilon Do you know how we (.NET MAUI Blazor developers) will get this fix? Is it by nuget, VS, new SDK? 😊 Thanks in advance and have a nice weekend!

codendone commented 1 year ago

The fix will be delivered as a Windows App SDK update.

vallgrenerik commented 1 year ago

The fix will be delivered as a Windows App SDK update.

Hi! I don't think I have installed any Windows sdk explicitly. I have just installed the Visual studio MAUI workload. Will I get it in a VS update then? 😊

Xyncgas commented 1 year ago

Hi! I don't think I have installed any Windows sdk explicitly. I have just installed the Visual studio MAUI workload. Will I get it in a VS update then? 😊

yes, Visual Studio is microsoft's powerful IDE for making all kinds of program. When you install a 'workload' you installed what you needed to develop for the application. A couple years / months ago there is Xamarin, their selling point was that you can write UI that run on Android + IOS, while still having access to native UI you are mostly writing 'common' UI features for both platform together. Xamarin achieved this by using native dependency for each platforms, MAUI too is using WindowsSDK when your app is going to be running on WIndows. When you are updating Visual Studio you are getting the newest version of the SDK for your development.

Meanwhile, these SDK can still be installed via nuget packages sometimes. People who doesn't use Visual Studio does a lot stuffs manually.

For example, in order to write C++ to compile to webassembly, you first have to find tools like emscripten, which turns out providing you with other tools to compile your programing languages to IL using LLVM and then compiles IL to webassembly. It's people's decision for how to do certain things and they provide the way to do it for you, you don't have to worry about it mostly because it works, just like when you are coding for Blazor.Webassembly .NET compiles your code to webassembly using emscripten too

true-perfect-code commented 1 year ago

If I understand you correctly @Xyncgas , we don't have to look for new Nuget versions or donwload SDK (see https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/downloads), we just wait for the next Visual Studio update.

I need to get another thing off at this point. The fact that this problem has dragged on for so long indicates that MAUI is used almost exclusively with XAML. And if someone then wants to display the web content in their app, then they will have the current problem through WebView2. And from the way it seems, it's really only a few programmers. On the other hand, if you use MAUI with Blazor, then there is no workaround and with such errors in the VebView component, the developed app is unusable because Blazor cannot work with MAUI without WebView. Two points worry me a lot here:

  1. the heavy dependency on WebView from MAUI Blazer or no alternative.
  2. the fact that few people realise the benefits of MAUI Blazor. Here I am thinking especially of XAML vs. HTML/CSS. I have been programming with WPF for more than 10 years and I used to work on one or another web project in between. I was very happy a few years ago when Blazor came out, because it has a lot of advantages compared to XAML. Unfortunately, I have the impression that most programmers don't see it that way. Why is that?

Thanks

Xyncgas commented 1 year ago

I don't like XAML, it's a static description, where as html you can change it dynamically and add event listeners to elements. The overhead of parsing plaintext html might seem stupid, it is actually very fast to parse string due to SIMD optimizations.

Everyone seriously wanting a MAUI.Blazor app can switch back to .NET 6 and this problem will not exist, publishers can choose the publish a webview that's evergreen or with the version locked.

the heavy dependency on WebView from MAUI Blazer or no alternative.

The dependency for MAUI.Blazor to have webview is so there is a browser to render web content, although it would be nice if we can draw ui using a unified rendering engine like a browser but without a browser and directly use the underlying available GUI framework for that platform

koenvd commented 1 year ago

Tried to get some feedback in the WinUI community call but had no luck @gabbybilka (I noticed as well the 1.3 milestone was added). But any update when a fix would become available? cc @codendone

gabbybilka commented 1 year ago

Fixed and available in 1.2.5: https://www.nuget.org/packages/Microsoft.WindowsAppSDK/1.2.230313.1 Will be available in 1.3.0 when it ships as well.

AdriaanLarcai commented 1 year ago

Fixed and available in 1.2.5: https://www.nuget.org/packages/Microsoft.WindowsAppSDK/1.2.230313.1 Will be available in 1.3.0 when it ships as well.

Thank you!

koenvd commented 1 year ago

Awesome! Thanks!!

KieranDevvs commented 1 year ago

@Eilon can we get Microsoft.WindowsAppSDK upgraded to this version (1.2.230313.1) on MAUI?

DierkDroth commented 1 year ago

Alright! At first glance a long standing issue which as road block for me is resolved now https://github.com/MicrosoftEdge/WebView2Feedback/issues/3167

Fingers crossed!

ManbirSinghRakhra commented 1 year ago

How do we update it in MAUI? I have seen the nuget package manager is saying this package is blocked by the project. I have tried to add it in the csproj manually.

Do we need to do any thing special for updating it inside Maui Blazor App?

rusfield commented 1 year ago

How do we update it in MAUI? I have seen the nuget package manager is saying this package is blocked by the project. I have tried to add it in the csproj manually.

Do we need to do any thing special for updating it inside Maui Blazor App?

In my MAUI Blazor csproj, I have added this package reference manually into an ItemGroup tag (where all my other package references are), and it seems to have done it for me (at least no freezes so far): `

` I was also struggling with this "blocked by project" thing and could not do it through the package manager. I think you can verify whether the new version gets installed by expanding your Blazor MAUI project > Dependencies > Packages > Microsoft.WindowsAppSDK (version), in Visual Studio after rebuilding, to verify that its updated.
Xyncgas commented 1 year ago

When can we get patched

true-perfect-code commented 1 year ago

I have updated my Visual Studio to version 17.5.3 and the error is fixed on Windows 10.

Unfortunately, it did not work on Windows Server 2019. I have written a new post about this here (but no feedback from WebView2 team yet). See https://github.com/MicrosoftEdge/WebView2Feedback/issues/3333