Automate all E2E tests on WebView

[SteveSandersonMS]

We now have test apps that can run the Blazor E2E tests from BasicTestApp within a WinForms or WPF BlazorWebView (see https://github.com/dotnet/aspnetcore/pull/31021). This represents the majority of Blazor's E2E test cases. Most already pass without trouble, some fail for expected reasons (they aren't applicable scenarios, or can't work until SWA is added), and a few fail unexpectedly and probably indicate a missing piece of implementation or an actual bug. This issue is to track getting them all to work or at least explaining why they aren't applicable.

Note that I've only checked the following within WinForms hosting, just because I got to it first. There's no reason to think that WPF or MAUI hosting would behave differently (except if MAUI is on Mono, in which case certain dependency edge cases could vary).

Test cases

• [x] BasicTestApp.AddRemoveChildComponents (Add/remove child components)

  • OK

• [x] BasicTestApp.AfterRenderInteropComponent (After-render interop component)

  • OK

• [x] BasicTestApp.AsyncEventHandlerComponent (Async event handlers)

  • OK

• [ ] BasicTestApp.AuthTest.AuthRouter (Auth cases)

  • FAIL: There's no AuthenticationStateProvider service
  • Need to determine whether this test should be applicable and if so how it should be made to work

• [ ] BasicTestApp.AuthTest.CascadingAuthenticationStateParent (Cascading authentication state)

  • FAIL: Can't load some type. Another auth case.

• [x] BasicTestApp.BindCasesComponent (bind cases)

  • Looks OK

• [x] BasicTestApp.CascadingValueTest.CascadingValueSupplier (Cascading values)

  • OK

• [x] BasicTestApp.ComponentRefComponent (Component ref component)

  • OK

• [x] BasicTestApp.ConcurrentRenderParent (Concurrent rendering)

  • OK

• [ ] BasicTestApp.ConfigurationComponent (Configuration)

  • FAIL: There's no DI service IWebAssemblyHostEnvironment.
  • Need to determine whether this test should be applicable and if so how it should be made to work

• [x] BasicTestApp.ContentEditable (Content-editable)

  • Doesn't really do anything; probably works.

• [x] BasicTestApp.CounterComponent (Counter)

  • OK

• [x] BasicTestApp.CounterComponentUsingChild (Counter using child component)

  • OK

• [x] BasicTestApp.CounterComponentWrapper (Counter wrapped in parent)

  • OK

• [ ] BasicTestApp.CulturePicker (Culture Picker)

  • FAIL: This needs to be wired up differently if we want it to actually change the test app culture.
  • Doesn't imply we have a bug though

• [x] BasicTestApp.DataDashComponent (data- attribute rendering)

  • OK

• [ ] BasicTestApp.DispatchingComponent (Dispatching to sync context)

  • FAIL Produces output in wrong order, with one item missing entirely
  • This one looks like a genuine bug in WinForms hosting. If we want the sync context semantics to be the same across all hosting models, looks like more work is needed.

• [x] BasicTestApp.DuplicateAttributesComponent (Duplicate attributes)

  • OK

• [x] BasicTestApp.DynamicComponentRendering (DynamicComponent rendering)

  • OK

• [x] BasicTestApp.ElementFocusComponent (Element focus component)

  • OK (fixed in https://github.com/dotnet/aspnetcore/pull/31050)

• [x] BasicTestApp.ElementRefComponent (Element ref component)

  • OK

• [x] BasicTestApp.ErrorComponent (Error throwing)

  • OK

• [x] BasicTestApp.EventBubblingComponent (Event bubbling)

  • OK

• [x] BasicTestApp.EventCallbackTest.EventCallbackCases (EventCallback)

  • OK

• [x] BasicTestApp.EventCustomArgsComponent (Event custom arguments)

  • OK

• [x] BasicTestApp.EventDisablingComponent (Event disabling)

  • OK

• [x] BasicTestApp.EventDuringBatchRendering (Event during batch rendering)

  • OK

• [x] BasicTestApp.EventPreventDefaultComponent (Event preventDefault)

  • OK

• [x] BasicTestApp.ExternalContentPackage (External content package)

  • OK - fixed in https://github.com/dotnet/aspnetcore/pull/31054

• [x] BasicTestApp.FocusEventComponent (Focus events)

  • OK

• [x] BasicTestApp.FormsTest.InputFocusComponent (Input Focus)

  • OK (fixed in https://github.com/dotnet/aspnetcore/pull/31049 and https://github.com/dotnet/aspnetcore/pull/31050)

• [x] BasicTestApp.FormsTest.NotifyPropertyChangedValidationComponent (INotifyPropertyChanged validation)

  • OK

• [x] BasicTestApp.FormsTest.SimpleValidationComponent (Simple validation)

  • OK

• [ ] BasicTestApp.FormsTest.SimpleValidationComponentUsingExperimentalValidator (Simple validation using experimental validator)

  • FAIL Could not load type 'BasicTestApp.FormsTest.SimpleValidationComponentUsingExperimentalValidator'
  • I have no idea what's wrong here. Needs investigation.

• [x] BasicTestApp.FormsTest.TypicalValidationComponent (Typical validation)

  • OK

• [ ] BasicTestApp.FormsTest.TypicalValidationComponentUsingExperimentalValidator (Typical validation using experimental validator)

  • FAIL Could not load type 'BasicTestApp.FormsTest.TypicalValidationComponentUsingExperimentalValidator'
  • I have no idea what's wrong here. Needs investigation.

• [x] BasicTestApp.FormsTest.InputFileComponent (Input file)

  • OK (fixed in dotnet/aspnetcore#31049)

• [ ] BasicTestApp.NavigateOnSubmit (Navigate to submit)

  • Inconclusive - I'm not certain what this test case is meant to do. It seems to do nothing.

• [x] BasicTestApp.GlobalizationBindCases (Globalization Bind Cases)

  • OK

• [ ] BasicTestApp.GracefulTermination (Graceful Termination)

  • Probably fine but it's not 100% clear whether these cases should behave differently in some way on WebView

• [x] BasicTestApp.HierarchicalImportsTest.Subdir.ComponentUsingImports (Imports statement)

  • OK

• [x] BasicTestApp.HtmlBlockChildContent (ChildContent HTML Block)

  • OK

• [x] BasicTestApp.HtmlEncodedChildContent (ChildContent HTML Encoded Block)

  • OK

• [x] BasicTestApp.HtmlMixedChildContent (ChildContent Mixed Block)

  • OK

• [ ] BasicTestApp.HttpClientTest.BinaryHttpRequestsComponent (Binary HttpClient tester)

  • Should be fine since HttpClient is independent of the code here. Validating it in more detail requires figuring out exactly what the automated tests would do (e.g., which URLs they request).

• [ ] BasicTestApp.HttpClientTest.CookieCounterComponent (HttpClient cookies)

  • Should be fine since HttpClient is independent of the code here. Validating it in more detail requires figuring out exactly what the automated tests would do (e.g., which URLs they request).

• [ ] BasicTestApp.HttpClientTest.HttpRequestsComponent (HttpClient tester)

  • Should be fine since HttpClient is independent of the code here. Validating it in more detail requires figuring out exactly what the automated tests would do (e.g., which URLs they request).

• [x] BasicTestApp.InputEventComponent (Input events)

  • OK

• [x] BasicTestApp.InteropComponent (Interop component)

  • OK - fixed in https://github.com/dotnet/aspnetcore/pull/31054

• [x] BasicTestApp.InteropOnInitializationComponent (Interop on initialization)

  • OK

• [x] BasicTestApp.JsonSerializationCases (JSON serialization)

  • OK

• [x] BasicTestApp.KeyCasesComponent (Key cases)

  • OK

• [x] BasicTestApp.KeyPressEventComponent (Key press event)

  • OK

• [x] BasicTestApp.LaggyTypingComponent (Laggy typing)

  • OK

• [x] BasicTestApp.LimitCounterComponent (Limit counter component)

  • OK I think; unclear how to repro the test case manually

• [x] BasicTestApp.LocalizedText (Localized Text)

  • OK

• [x] BasicTestApp.LogicalElementInsertionCases (Logical element insertion cases)

  • OK

• [ ] BasicTestApp.LoggingComponent (Logging)

  • FAIL - produces no log output, either to browser console or .NET host app console, at least on WinForms
  • We need to work out what's supposed to happen in the WebView case here and make sure it's right

• [x] BasicTestApp.LongRunningInterop (Long running interop)

  • OK

• [x] BasicTestApp.MarkupBlockComponent (Markup blocks)

  • OK

• [x] BasicTestApp.MouseEventComponent (Mouse events)

  • OK

• [x] BasicTestApp.MovingCheckboxesComponent (Moving checkboxes diff case)

  • OK

• [x] BasicTestApp.MultipleChildContent (Multiple child content)

  • OK

• [x] BasicTestApp.NavigationFailureComponent (Navigation failure)

  • OK... I think? It does produce an exception, which seems to be what the test case wants to do.

• [x] BasicTestApp.ParentChildComponent (Parent component with child)

  • OK

• [x] BasicTestApp.PropertiesChangedHandlerParent (Parent component that changes parameters on child)

  • OK

• [x] @GetTestServerProjectComponent("Components.TestServer.ProtectedBrowserStorageUsageComponent") (Protected browser storage usage)

  • Not applicable - ProtectedBrowserStorage doesn't make sense in WebView (same as in WebAssembly)

• [x] @GetTestServerProjectComponent("Components.TestServer.ProtectedBrowserStorageInjectionComponent") (Protected browser storage injection)

  • Not applicable - ProtectedBrowserStorage doesn't make sense in WebView (same as in WebAssembly)

• [x] BasicTestApp.RazorTemplates (Razor Templates)

  • OK, I think (unclear what the test case proves)

• [x] BasicTestApp.Reconnection.ReconnectionComponent (Reconnection server-side blazor)

  • Not applicable - reconnection doesn't make sense in WebView (it's only for Server)

• [x] BasicTestApp.RedTextComponent (Red text)

  • OK

• [x] BasicTestApp.ReliabilityComponent (Server reliability component)

  • Not applicable - these are only for Server

• [x] BasicTestApp.RenderFragmentToggler (Render fragment renderer)

  • OK

• [x] BasicTestApp.RenderAttributesBeforeConnectedCallback (Render attributes before ConnectedCallback)

  • Not applicable - these are only for Server

• [x] BasicTestApp.ReorderingFocusComponent (Reordering focus retention)

  • OK

• [x] BasicTestApp.RouterTest.NavigationManagerComponent (NavigationManager Test)

  • OK

• [ ] BasicTestApp.RouterTest.TestRouter (Router)

  • Inconclusive - There are a lot of subcases here and it will take more work to figure out what we expect in each case on WebView

• [ ] BasicTestApp.RouterTest.TestRouterWithOnNavigate (Router with OnNavigate)

  • Inconclusive - There are a lot of subcases here and it will take more work to figure out what we expect in each case on WebView

• [ ] BasicTestApp.RouterTest.TestRouterWithLazyAssembly (Router with dynamic assembly)

  • FAIL There's no DI service LazyAssemblyLoader
  • Lazy loading isn't really applicable on WebView, but we should consider whether we want it to work anyway so that the same components would work across WebAssembly and WebView

• [ ] BasicTestApp.RouterTest.TestRouterWithAdditionalAssembly (Router with additional assembly)

  • Inconclusive - There are a lot of subcases here and it will take more work to figure out what we expect in each case on WebView

• [x] BasicTestApp.SelectVariantsComponent (Select with component options)

  • OK

• [ ] BasicTestApp.SignalRClientComponent (SignalR client)

  • Inconclusive - Need a SignalR hub at the other end to exercise this test
  • It's not really specific to WebView anyway. As long as the SignalR client works on the underlying runtime, there's nothing else for us to test here

• [x] BasicTestApp.StringComparisonComponent (StringComparison)

  • OK, I think (unclear what the test expects)

• [x] BasicTestApp.SvgComponent (SVG)

  • OK

• [x] BasicTestApp.TextOnlyComponent (Plain text)

  • OK

• [x] BasicTestApp.ToggleEventComponent (Toggle Event)

  • OK

• [ ] BasicTestApp.TouchEventComponent (Touch events)

  • Inconclusive - Seems to do nothing. Not sure what it expects to happen.

• [x] BasicTestApp.VirtualizationComponent (Virtualization)

  • OK

• [x] BasicTestApp.VirtualizationDataChanges (Virtualization data changes)

  • OK

[javiercn]

• This one looks like a genuine bug in WebForms hosting. If we want the sync context semantics to be the same across all hosting models, looks like more work is needed.

We want the semantics to be the same, otherwise libraries won't work properly.

[captainsafia]

Lazy loading isn't really applicable on WebView, but we should consider whether we want it to work anyway so that the same components would work across WebAssembly and WebView

Yeah, lazy-loading won't practically work in WebView the same way it does on WASM. We can probably have an implementation of WebViewLazyAssemblyLoader that virtually no-ops when invoked so that components are cross-compat.

[Eilon]

Looking pretty good in Android so far...

[Eilon]

❓ = unknown / not tested	❌ = doesn't work	✔ = works

Scenario	WPF	WinForms	MAUI WinUI	MAUI MacCat	MAUI iOS	MAUI Droid	Comments
1. AddRemoveChildComponents (Add/remove child components)			✔			✔
2. AfterRenderInteropComponent (After-render interop component)			✔			✔
3. AsyncEventHandlerComponent (Async event handlers)			✔			✔
4. AuthTest.AuthRouter (Auth cases)			❌			❌
5. AuthTest.CascadingAuthenticationStateParent (Cascading authentication state)			❌			❌
6. BindCasesComponent (bind cases)						✔
7. CascadingValueTest.CascadingValueSupplier (Cascading values)						✔
8. ComponentRefComponent (Component ref component)						✔
9. ConcurrentRenderParent (Concurrent rendering)						✔
10. ConfigurationComponent (Configuration)						❌
11. ContentEditable (Content-editable)						✔
12. CounterComponent (Counter)						✔
13. CounterComponentUsingChild (Counter using child component)						✔
14. CounterComponentWrapper (Counter wrapped in parent)						✔
15. CulturePicker (Culture Picker)						❌
16. DataDashComponent (data-* attribute rendering)						✔
17. DispatchingComponent (Dispatching to sync context)
18. DuplicateAttributesComponent (Duplicate attributes)
19. DynamicComponentRendering (DynamicComponent rendering)
20. ElementFocusComponent (Element focus component)
21. ElementRefComponent (Element ref component)
22. ErrorComponent (Error throwing)
23. ErrorBoundaryTest.ErrorBoundaryContainer (Error boundary cases)
24. EventBubblingComponent (Event bubbling)
25. EventCallbackTest.EventCallbackCases (EventCallback)
26. EventCustomArgsComponent (Event custom arguments)
27. EventDisablingComponent (Event disabling)
28. EventDuringBatchRendering (Event during batch rendering)
29. EventPreventDefaultComponent (Event preventDefault)
30. ExternalContentPackage (External content package)
31. FocusEventComponent (Focus events)
32. Forms.InputFocusComponent (Input Focus)
33. Forms.NotifyPropertyChangedValidationComponent (INotifyPropertyChanged validation)
34. Forms.SimpleValidationComponent (Simple validation)
35. Forms.SimpleValidationComponentUsingExperimentalValidator (Simple validation using experimental validator)
36. Forms.TypicalValidationComponent (Typical validation)
37. Forms.TypicalValidationComponentUsingExperimentalValidator (Typical validation using experimental validator)
38. Forms.InputFileComponent (Input file)
39. NavigateOnSubmit (Navigate to submit)
40. GlobalizationBindCases (Globalization Bind Cases)
41. GracefulTermination (Graceful Termination)
42. HierarchicalImportsTest.Subdir.ComponentUsingImports (Imports statement)
43. HtmlBlockChildContent (ChildContent HTML Block)
44. HtmlEncodedChildContent (ChildContent HTML Encoded Block)
45. HtmlMixedChildContent (ChildContent Mixed Block)
46. HttpClientTest.BinaryHttpRequestsComponent (Binary HttpClient tester)
47. HttpClientTest.CookieCounterComponent (HttpClient cookies)
48. HttpClientTest.HttpRequestsComponent (HttpClient tester)
49. InputEventComponent (Input events)
50. InteropComponent (Interop component)
51. InteropOnInitializationComponent (Interop on initialization)
52. JsonSerializationCases (JSON serialization)
53. KeyCasesComponent (Key cases)
54. KeyPressEventComponent (Key press event)
55. LaggyTypingComponent (Laggy typing)
56. LimitCounterComponent (Limit counter component)
57. LocalizedText (Localized Text)
58. LogicalElementInsertionCases (Logical element insertion cases)
59. LoggingComponent (Logging)
60. LongRunningInterop (Long running interop)
61. MarkupBlockComponent (Markup blocks)
62. MouseEventComponent (Mouse events)
63. MovingCheckboxesComponent (Moving checkboxes diff case)
64. MultipleChildContent (Multiple child content)
65. NavigationFailureComponent (Navigation failure)
66. ParentChildComponent (Parent component with child)
67. PropertiesChangedHandlerParent (Parent component that changes parameters on child)
68. RazorTemplates (Razor Templates)
69. Reconnection.ReconnectionComponent (Reconnection server-side blazor)
70. RedTextComponent (Red text)
71. ReliabilityComponent (Server reliability component)
72. RenderFragmentToggler (Render fragment renderer)
73. RenderAttributesBeforeConnectedCallback (Render attributes before ConnectedCallback)
74. ReorderingFocusComponent (Reordering focus retention)
75. RouterTest.NavigationManagerComponent (NavigationManager Test)
76. RouterTest.TestRouter (Router)
77. RouterTest.TestRouterWithOnNavigate (Router with OnNavigate)
78. RouterTest.TestRouterWithLazyAssembly (Router with dynamic assembly)
79. RouterTest.TestRouterWithAdditionalAssembly (Router with additional assembly)
80. SelectVariantsComponent (Select with component options)
81. SignalRClientComponent (SignalR client)
82. StringComparisonComponent (StringComparison)
83. SvgComponent (SVG)
84. TextOnlyComponent (Plain text)
85. ToggleEventComponent (Toggle Event)
86. TouchEventComponent (Touch events)
87. VirtualizationComponent (Virtualization)
88. VirtualizationDataChanges (Virtualization data changes)
89. HotReload.RenderOnHotReload (Render on hot reload)

[mkArtakMSFT]

The pending work here is to automate using Photino in the dotnet/aspnetcore repo.

[ghost]

Thanks for contacting us.

We're moving this issue to the .NET 9 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s). If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[ghost]

Thanks for contacting us.

We're moving this issue to the .NET 9 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s). If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.