bUnit-dev / bUnit

bUnit is a testing library for Blazor components that make tests look, feel, and runs like regular unit tests. bUnit makes it easy to render and control a component under test’s life-cycle, pass parameter and inject services into it, trigger event handlers, and verify the rendered markup from the component using a built-in semantic HTML comparer.
https://bunit.dev
MIT License
1.13k stars 105 forks source link

FragmentContainer was not found in async test #1143

Closed uecasm closed 1 year ago

uecasm commented 1 year ago

Describe the bug

I have a relatively simple component that just renders different content depending on the state of a Task. The code of both component and test is very similar to the async example except that instead of awaiting the task on init I'm using task.ContinueWith(...) => InvokeAsync(StateHasChanged) and using the razor test syntax.

When using bUnit 1.16.2 and not using WaitForAssertion, the tests almost always pass. (I did very very rarely observe the same waiting failure as below.) When using later versions of bUnit, the tests will more frequently intermittently fail (showing the waiting content rather than the done content). When I tried adding WaitForAssertion (in 1.16.2) it started instead failing with:

Bunit.Extensions.WaitForHelpers.WaitForFailedException : The assertion did not pass within the timeout period. Check count: 2. Component render count: 2. Total render count: 5.
  ----> Bunit.Rendering.ComponentNotFoundException : A component of type FragmentContainer was not found in the render tree.
   at Bunit.RenderedFragmentWaitForHelperExtensions.WaitForAssertion(IRenderedFragmentBase renderedFragment, Action assertion, Nullable`1 timeout) in /_/src/bunit.core/Extensions/WaitForHelpers/RenderedFragmentWaitForHelperExtensions.cs:line 72

I haven't been able to replicate precisely this behaviour in a MCVE test, but what I did manage to reproduce is described below.

(Oddly, the MCVE code always fails (with waiting content, not the exception) when not using WaitForAssertion. While not exactly surprising due to async, it's odd that it's different; though it's likely that this is due to the real component being a bit more complex.)

Example: Testing this component:

@if (Task != null)
{
    @if (Task.IsCompleted)
    {
        <span>done</span>
    }
    else
    {
        <span>waiting</span>
    }
}

@code {

    [Parameter] public Task? Task { get; set; }

    private Task? _RegisteredTask;

    protected override void OnParametersSet()
    {
        var task = Task;
        if (task != _RegisteredTask)
        {
            _RegisteredTask = task;

            _ = task?.ContinueWith((t, o) =>
            {
                if (t == Task)
                {
                    _ = InvokeAsync(StateHasChanged);
                }
            }, null);
        }

        base.OnParametersSet();
    }

}

With this test:

@using NUnit.Framework
@using Bunit
@*@inherits Bunit.TestContext*@
@inherits BunitTestContext  /* this uses TestContextWrapper */
@code {

    [Test]
    public void Cancel1()
    {
        var tcs = new TaskCompletionSource();

        using var cut = Render(@<MyComponent Task="@tcs.Task"/>);

        cut.MarkupMatches(@<span>waiting</span>);

        tcs.SetCanceled();

        cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>));
    }

    [Test]
    public void Cancel2()
    {
        var tcs = new TaskCompletionSource();

        using var cut = Render(@<MyComponent Task="@tcs.Task"/>);

        cut.MarkupMatches(@<span>waiting</span>);

        tcs.SetCanceled();

        cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>));
    }

    [Test]
    public void Cancel3()
    {
        var tcs = new TaskCompletionSource();

        using var cut = Render(@<MyComponent Task="@tcs.Task"/>);

        cut.MarkupMatches(@<span>waiting</span>);

        tcs.SetCanceled();

        cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>));
    }

}

(Note that this is three identical copies of the same test.)

Expected behavior:

All tests should pass.

Actual behavior:

The first test always passes. The other two tests intermittently fail with the exception above.

If I run the tests in the debugger (without stopping on any breakpoints or exceptions), all tests pass.

Version info:

uecasm commented 1 year ago

Ok, I don't completely understand this (since the code ought to be equivalent, with the exception of SynchronizationContext, except that shouldn't matter to calling InvokeAsync), but replacing the OnParametersSet with the below seems to avoid the exception and the tests all consistently pass:

    protected override void OnParametersSet()
    {
        var task = Task;
        if (task != _RegisteredTask)
        {
            _RegisteredTask = task;

            task?.GetAwaiter().OnCompleted(() =>
            {
                if (task == Task)
                {
                    StateHasChanged();
                }
            });
        }

        base.OnParametersSet();
    }

So, yay I guess?


This version of the MCVE code brings back the exceptions:

    private Task<object?>? _WrappedTask;

    protected override void OnParametersSet()
    {
        var task = Task;
        if (task != _RegisteredTask)
        {
            _RegisteredTask = task;
            _WrappedTask = task == null ? null : Wrap(task);

            var localTask = _WrappedTask;
            localTask?.ConfigureAwait(false).GetAwaiter().OnCompleted(() =>
            {
                if (localTask == _WrappedTask)
                {
                    _ = InvokeAsync(StateHasChanged);
                }
            });
        }

        base.OnParametersSet();
    }

    private static async Task<object?> Wrap(Task task)
    {
        await task;
        return null;
    }

Amending Wrap to also use ConfigureAwait(false) fixes them (and tests pass). It all seems a bit fragile, though.

egil commented 1 year ago

Thanks for reporting this @uecasm.

Do you mind trying this with the latest release?

uecasm commented 1 year ago

1.21.9 is the latest release, isn't it? At least it's the newest on nuget.org...

linkdotnet commented 1 year ago

I found an intersting issue on the ASP.NET Core GitHub Tracker that has a similar problem: https://github.com/dotnet/aspnetcore/issues/43364

That would to some extend explain some of the phenomena you discovered.

As said note: The Blazor team encourages not to use ContinueWith - that makes sense, as the thread gets scheduled on the thread pool without any SynchronizationContext. async/ await would be preferrable here.

egil commented 1 year ago

1.21.9 is the latest release, isn't it? At least it's the newest on nuget.org...

Did the latest release fix the issue?

uecasm commented 1 year ago

1.21.9 is the latest release, isn't it? At least it's the newest on nuget.org...

Did the latest release fix the issue?

No, I think you misunderstood. This was the latest release the whole time, at least for the MCVE (see the bottom of the original post).

egil commented 1 year ago

I tried and was able to reproduce using NUnit, but not XUnit.:

This is my test case:

@inherits TestContext
@code {
#if NET6_0_OR_GREATER
    [NUnit.Framework.Test()]
    [NUnit.Framework.Repeat(1000)]
    public void CancelNUnit()
    {
        using var ctx = new TestContext();
        var tcs = new TaskCompletionSource();

        using var cut = ctx.Render(@<InvokeAsyncInsideContinueWith Task="@tcs.Task"/>);

        cut.MarkupMatches(@<span>waiting</span>);

        tcs.SetCanceled();

// The exception is being thrown when the render fragment passed to MarkupMatches
// is rendered before getting passed to the diffing algorithm.                
        cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>));
    }

    [Fact]
    [Repeat(1000)]
    public void CancelXunit()
    {
        var tcs = new TaskCompletionSource();

        using var cut = Render(@<InvokeAsyncInsideContinueWith Task="@tcs.Task"/>);

        cut.MarkupMatches(@<span>waiting</span>);

        tcs.SetCanceled();

        cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>));
    }
#endif
}

Full exception output:

Bunit.Extensions.WaitForHelpers.WaitForFailedException : The assertion did not pass within the timeout period. Check count: 2. Component render count: 2. Total render count: 5.
  ----> Bunit.Rendering.ComponentNotFoundException : A component of type FragmentContainer was not found in the render tree.
   at Bunit.RenderedFragmentWaitForHelperExtensions.WaitForAssertion(IRenderedFragmentBase renderedFragment, Action assertion, Nullable`1 timeout) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\RenderedFragmentWaitForHelperExtensions.cs:line 72
   at Bunit.TestContextBaseTest.Cancel1() in C:\dev\bUnit-dev\bUnit\tests\bunit.core.tests\TestContextBaseTest.razor:line 17
--ComponentNotFoundException
   at Bunit.Rendering.TestRenderer.FindComponent[TComponent](IRenderedFragmentBase parentComponent) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Rendering\TestRenderer.cs:line 152
   at Bunit.Extensions.TestContextBaseRenderExtensions.RenderInsideRenderTree(TestContextBase testContext, RenderFragment renderFragment) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\TestContextBaseRenderExtensions.cs:line 45
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, RenderFragment expected, String userMessage) in C:\dev\bUnit-dev\bUnit\src\bunit.web\Asserting\MarkupMatchesAssertExtensions.cs:line 303
   at Bunit.TestContextBaseTest.<>c__DisplayClass20_0.<Cancel1>b__2() in C:\dev\bUnit-dev\bUnit\tests\bunit.core.tests\TestContextBaseTest.razor:line 17
   at Bunit.Extensions.WaitForHelpers.WaitForAssertionHelper.<>c__DisplayClass6_0.<.ctor>b__0() in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForAssertionHelper.cs:line 31
   at Bunit.Extensions.WaitForHelpers.WaitForHelper`1.OnAfterRender(Object sender, EventArgs args) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForHelper.cs:line 153

I am still not sure why this happens. The FragmentContainer is something we add into the render tree to separate any components that have been added to the TestContext.RenderTree, and there should not be any reason for this to fail as far as I can tell. It could be an unhandled exception that is staying around from the initial render though or from the previous MarkupMatches call.

Will continue to investigate. @linkdotnet if you can take a look too it would be very helpful.

linkdotnet commented 1 year ago

I did run your test 25k times without any issue - how does your InvokeAsyncInsideContinueWith look like? Is it from the original post?

Edit: I did run the test on the current main as well as the latest v1.21 release

egil commented 1 year ago

My InvokeAsyncInsideContinueWith is this

@@ -0,0 +1,35 @@
@if (Task != null)
{
    @if (Task.IsCompleted)
    {
        <span>done</span>
    }
    else
    {
        <span>waiting</span>
    }
}
@code {
    [Parameter] public Task? Task { get; set; }

    private Task? registeredTask;

    protected override void OnParametersSet()
    {
        var task = Task;
        if (task != registeredTask)
        {
            registeredTask = task;

            _ = task?.ContinueWith((t, o) =>
            {
                if (t == Task)
                {
                    _ = InvokeAsync(StateHasChanged);
                }
            }, null);
        }

        base.OnParametersSet();
    }
}
linkdotnet commented 1 year ago

Okay - I used the same and for me it doesn't fail - even after 100000 runs

egil commented 1 year ago

Pushed branch 1143-fragmentcontainer-not-found that has the test that fail. I'm on my slow laptop. It happens just after running the test once with its own repeat attribute.

linkdotnet commented 1 year ago

Interesting - on your branch it fails for me but when I locally did the same it did not. The only difference was that I did not take the razor syntax! @egil Can you recreate the test in plain old csharp code?

egil commented 1 year ago

It only happens with razor syntax. The error happens when MarkupMatches converts the render fragment to markup.

I also saw the xunit test fail now, so it does not matter that NUnit doesn't use a sync context, it seems.

linkdotnet commented 1 year ago

I guess I am stating some obvious stuff, but nevertheless:

cut.WaitForAssertion(() => cut.MarkupMatches("<span>done</span>")); // Works
cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>)); // Doesn't work

The obvious difference is that the second one, the one that does accept a RenderFragment does trigger a new render cycle:

public static void MarkupMatches(this IRenderedFragment actual, RenderFragment expected, string? userMessage = null)
{
    // ....

    var testContext = actual.Services.GetRequiredService<TestContextBase>();
    var renderedFragment = (IRenderedFragment)testContext.RenderInsideRenderTree(expected);
    MarkupMatches(actual, renderedFragment, userMessage);
}

So what follows is that the FragmentContainer that could not be found is not from the component under test but from the expected RenderFragment (here @<span>done</span>).

egil commented 1 year ago

That is my guess too. In addition, I guess that the render of the markup matches fragment actually isn't able to run (renderer is locked) because the async triggered render by the TCS is blocking the renderer because it is rendering.

linkdotnet commented 1 year ago

As way forward #1018 becomes a valid candidate - when MarkupMatches has its own renderer, that is completely detached from the cut, this issue does not persist.

egil commented 1 year ago

As way forward #1018 becomes a valid candidate - when MarkupMatches has its own renderer, that is completely detached from the cut, this issue does not persist.

Yes and no. This behavior could indicate that an assumption bUnit has is not correct, so I do want to understand this fully.

uecasm commented 1 year ago

FWIW after I changed my real app to use the latest MCVE version (OnComplete with double ConfigureAwait(false)), while I've never managed to get it to fail again on my machine, it still happens occasionally on a slower machine. But it's a lot rarer than previously.

egil commented 1 year ago

FWIW after I changed my real app to use the latest MCVE version (OnComplete with double ConfigureAwait(false)), while I've never managed to get it to fail again on my machine, it still happens occasionally on a slower machine. But it's a lot rarer than previously.

If you want to eliminate it completely (until (if) we find a fix), don't pass a RenderFragment to MarkupMatches, just pass a string. If you are using c# 11 you can use the """ quote strings to get around having to escape attributes, etc., i.e.:

@using NUnit.Framework
@using Bunit
@*@inherits Bunit.TestContext*@
@inherits BunitTestContext  /* this uses TestContextWrapper */
@code {

    [Test]
    public void Cancel1()
    {
        var tcs = new TaskCompletionSource();

        using var cut = Render(@<MyComponent Task="@tcs.Task"/>);

        cut.MarkupMatches(@<span>waiting</span>);

        tcs.SetCanceled();

        cut.WaitForAssertion(() => cut.MarkupMatches("""<span>done</span>"""));
    }
egil commented 1 year ago

@linkdotnet I am pretty certain I understand what is going on. Here is my thinking:

  1. The TaskCompletionSource is completed, causing a render cycle to start for CUT.
  2. That render cycle of InvokeAsyncInsideContinueWith is kicked off, causing UpdateDisplayAsync to be called, which then triggers the WaitForHandler's checker and causes it to attempt the assertion again.
  3. The assertion needs to render the expected markup fragment. This happens on the same thread as the current UpdateDisplayAsync is running on, which means the render is queued up internally in the renderer's pending task queue.
  4. That means the expected markup render does not complete and causes the ComponentNotFoundException to be thrown.
  5. The check fails, and the previous render cycle of the CUT completes, allowing the render cycle of the expected markup to complete.

This seems unique to using a rendering a render fragment in a WaitForX checker. Doing other things like calling FindComponent from a checker is OK since that is just accessing the current render tree of the CUT.

UPDATE confirmed by logging this in the renderer when IsBatchInProgress == true.

Suggestions?

Bunit.Extensions.WaitForHelpers.WaitForFailedException: The assertion did not pass within the timeout period. Check count: 2. Component render count: ...

Bunit.Extensions.WaitForHelpers.WaitForFailedException
The assertion did not pass within the timeout period. Check count: 2. Component render count: 2. Total render count: 5.
   at Bunit.RenderedFragmentWaitForHelperExtensions.WaitForAssertion(IRenderedFragmentBase renderedFragment, Action assertion, Nullable`1 timeout) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\RenderedFragmentWaitForHelperExtensions.cs:line 72
   at Bunit.TestContextBaseTest.CancelXunit() in C:\dev\bUnit-dev\bUnit\tests\bunit.core.tests\TestContextBaseTest.razor:line 49
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

Bunit.Rendering.ComponentNotFoundException
A component of type FragmentContainer was not found in the render tree.
   at Bunit.Rendering.TestRenderer.FindComponent[TComponent](IRenderedFragmentBase parentComponent) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Rendering\TestRenderer.cs:line 152
   at Bunit.Extensions.TestContextBaseRenderExtensions.RenderInsideRenderTree(TestContextBase testContext, RenderFragment renderFragment) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\TestContextBaseRenderExtensions.cs:line 45
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, RenderFragment expected, String userMessage) in C:\dev\bUnit-dev\bUnit\src\bunit.web\Asserting\MarkupMatchesAssertExtensions.cs:line 303
   at Bunit.TestContextBaseTest.<>c__DisplayClass21_0.<CancelXunit>b__2() in C:\dev\bUnit-dev\bUnit\tests\bunit.core.tests\TestContextBaseTest.razor:line 49
   at Bunit.Extensions.WaitForHelpers.WaitForAssertionHelper.<>c__DisplayClass6_0.<.ctor>b__0() in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForAssertionHelper.cs:line 31
   at Bunit.Extensions.WaitForHelpers.WaitForHelper`1.OnAfterRender(Object sender, EventArgs args) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForHelper.cs:line 153

2023-07-10 09:27:23 (0018) [Debug] Initializing root component 0 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0018) [Debug] Rendering component 0 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:23 (0018) [Debug] Initializing component 1 ("Bunit.Rendering.FragmentContainer") as child of 0 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0018) [Debug] Rendering component 1 of type "Bunit.Rendering.FragmentContainer"
2023-07-10 09:27:23 (0018) [Debug] Initializing component 2 ("Bunit.TestAssets.SampleComponents.InvokeAsyncInsideContinueWith") as child of 1 ("Bunit.Rendering.FragmentContainer")
2023-07-10 09:27:23 (0018) [Debug] Rendering component 2 of type "Bunit.TestAssets.SampleComponents.InvokeAsyncInsideContinueWith"
2023-07-10 09:27:23 (0018) [Debug] Component 0 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] The initial render of component 0 is completed.
2023-07-10 09:27:23 (0018) [Debug] Initializing root component 3 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0018) [Debug] Rendering component 3 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:23 (0018) [Debug] Initializing component 4 ("Bunit.Rendering.FragmentContainer") as child of 3 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0018) [Debug] Rendering component 4 of type "Bunit.Rendering.FragmentContainer"
2023-07-10 09:27:23 (0018) [Debug] Component 0 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] Component 1 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] Component 3 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] The initial render of component 3 is completed.
--- tcs.SetCanceled(); is called---
--- cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>)); is called ---
2023-07-10 09:27:23 (0018) [Debug] Checking the wait condition for component 1.
2023-07-10 09:27:23 (0018) [Debug] Initializing root component 5 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0018) [Debug] Rendering component 5 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:23 (0018) [Debug] Initializing component 6 ("Bunit.Rendering.c") as child of 5 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0018) [Debug] Rendering component 6 of type "Bunit.Rendering.FragmentContainer"
2023-07-10 09:27:23 (0018) [Debug] Component 0 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] Component 1 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] Component 3 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] Component 4 has been rendered.
2023-07-10 09:27:23 (0018) [Debug] Component 5 has been rendered.
--- render of @<span>done</span> done (child of FragmentContainer with id 5) ---
2023-07-10 09:27:23 (0018) [Debug] The initial render of component 5 is completed.
2023-07-10 09:27:23 (0018) [Debug] The checker for component 1 throw an exception.
Bunit.HtmlEqualException: HTML comparison failed. 

The following errors were found:
  1: The text in span(0) > #text(0) is different.

Actual HTML: 
<span>waiting</span>

Expected HTML: 
<span>done</span>

   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(INodeList actual, INodeList expected, String userMessage) in C:\dev\bUnit-dev\bUnit\src\bunit.web\Asserting\MarkupMatchesAssertExtensions.cs:line 237
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, IRenderedFragment expected, String userMessage) in C:\dev\bUnit-dev\bUnit\src\bunit.web\Asserting\MarkupMatchesAssertExtensions.cs:line 132
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, RenderFragment expected, String userMessage) in C:\dev\bUnit-dev\bUnit\src\bunit.web\Asserting\MarkupMatchesAssertExtensions.cs:line 304
   at Bunit.TestContextBaseTest.<>c__DisplayClass21_0.<CancelXunit>b__2() in C:\dev\bUnit-dev\bUnit\tests\bunit.core.tests\TestContextBaseTest.razor:line 49
   at Bunit.Extensions.WaitForHelpers.WaitForAssertionHelper.<>c__DisplayClass6_0.<.ctor>b__0() in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForAssertionHelper.cs:line 31
   at Bunit.Extensions.WaitForHelpers.WaitForHelper`1.OnAfterRender(Object sender, EventArgs args) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForHelper.cs:line 153
2023-07-10 09:27:23 (0024) [Debug] Rendering component 2 of type "Bunit.TestAssets.SampleComponents.InvokeAsyncInsideContinueWith"
2023-07-10 09:27:23 (0024) [Debug] Component 0 has been rendered.
--- Component 1 finished rendering after component 7, which causes the WaitForHandler to attempt to verify
     condition again. This verification happens inside the render lock to avoid further renders while
     checks are being performed. However, the check involves rendering the fragment passed to MarkupMatches
     which is blocked and thus never completes. ---
2023-07-10 09:27:23 (0024) [Debug] Checking the wait condition for component 1.
2023-07-10 09:27:23 (0024) [Debug] Initializing root component 7 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0024) [Debug] The initial render of component 7 is completed.
2023-07-10 09:27:23 (0024) [Debug] The checker for component 1 throw an exception.
Bunit.Rendering.ComponentNotFoundException: A component of type FragmentContainer was not found in the render tree.
   at Bunit.Rendering.TestRenderer.FindComponent[TComponent](IRenderedFragmentBase parentComponent) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Rendering\TestRenderer.cs:line 152
   at Bunit.Extensions.TestContextBaseRenderExtensions.RenderInsideRenderTree(TestContextBase testContext, RenderFragment renderFragment) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\TestContextBaseRenderExtensions.cs:line 45
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, RenderFragment expected, String userMessage) in C:\dev\bUnit-dev\bUnit\src\bunit.web\Asserting\MarkupMatchesAssertExtensions.cs:line 303
   at Bunit.TestContextBaseTest.<>c__DisplayClass21_0.<CancelXunit>b__2() in C:\dev\bUnit-dev\bUnit\tests\bunit.core.tests\TestContextBaseTest.razor:line 49
   at Bunit.Extensions.WaitForHelpers.WaitForAssertionHelper.<>c__DisplayClass6_0.<.ctor>b__0() in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForAssertionHelper.cs:line 31
   at Bunit.Extensions.WaitForHelpers.WaitForHelper`1.OnAfterRender(Object sender, EventArgs args) in C:\dev\bUnit-dev\bUnit\src\bunit.core\Extensions\WaitForHelpers\WaitForHelper.cs:line 153
2023-07-10 09:27:23 (0024) [Debug] Component 1 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 3 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 4 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 5 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 6 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Rendering component 7 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:23 (0024) [Debug] Initializing component 8 ("Bunit.Rendering.FragmentContainer") as child of 7 ("Bunit.Rendering.RootComponent")
2023-07-10 09:27:23 (0024) [Debug] Rendering component 8 of type "Bunit.Rendering.FragmentContainer"
2023-07-10 09:27:23 (0024) [Debug] Component 0 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 1 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 3 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 4 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 5 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 6 has been rendered.
2023-07-10 09:27:23 (0024) [Debug] Component 7 has been rendered.
--- rendering of component 8 (used in the second wait for check) completes are the check has failed) ---
2023-07-10 09:27:53 (0012) [Debug] The waiter for component 1 timed out.
2023-07-10 09:27:53 (0018) [Debug] The waiter for component 1 disposed.
2023-07-10 09:27:53 (0018) [Debug] Disposing component 0 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 1 of type "Bunit.Rendering.FragmentContainer"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 2 of type "Bunit.TestAssets.SampleComponents.InvokeAsyncInsideContinueWith"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 3 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 4 of type "Bunit.Rendering.FragmentContainer"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 5 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 6 of type "Bunit.Rendering.FragmentContainer"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 7 of type "Bunit.Rendering.RootComponent"
2023-07-10 09:27:53 (0018) [Debug] Disposing component 8 of type "Bunit.Rendering.FragmentContainer"
egil commented 1 year ago

Fund the assumption that we have had that didn't hold:

https://github.com/bUnit-dev/bUnit/blob/674a5559d1cc572de0bea9dbb203b6f7358316d0/src/bunit.core/Rendering/TestRenderer.cs#L379-L382

linkdotnet commented 1 year ago

Fund the assumption that we have had that didn't hold:

https://github.com/bUnit-dev/bUnit/blob/674a5559d1cc572de0bea9dbb203b6f7358316d0/src/bunit.core/Rendering/TestRenderer.cs#L379-L382

Hmmm my way of thinking is that we have the dispatcher before, which result we for sure await (GetAwaiter().GetResult()): For sure that should block until the synchronous part of the render cycle is done - when there is another render cycle happening, this should block thanks to the dispatcher, not?

egil commented 1 year ago

Here is all the code of the Render<TResult> that was partly linked above.

https://github.com/bUnit-dev/bUnit/blob/674a5559d1cc572de0bea9dbb203b6f7358316d0/src/bunit.core/Rendering/TestRenderer.cs#L349-L389

Even after the renderTask is completed there is still a chance that the root component has not rendered yet.

If you download the 1143-fragmentcontainer-not-found branch you will see that there are a few other tests besides the Cancel one that are also failing due to the InvalidOperationException being thrown.

egil commented 1 year ago

Correction, no other tests are failing, if I move AssertNoUnhandledExceptions above the render count check.

The CancelXunit tests do fail though with the The root component did not complete it's initial render cycle. exception.

linkdotnet commented 1 year ago

Suggestions?

We could instantiate our own renderer-instance for MarkupMatches that is completely detached from the cut.

egil commented 1 year ago

Suggestions?

We could instantiate our own renderer-instance for MarkupMatches that is completely detached from the cut.

That will likely work. Should we attempt to reuse renderer instances if they are not currently blocked?

What about Services? We register a renderer in there that is getting pulled out in certain circumstances, should that be a transient registration instead?

linkdotnet commented 1 year ago

Probably newing up a renderer might be "good enough". The average use case is not the pass in another Blazor component that needs registered services and has async lifecycle. If the assumption doesn't hold true, we still can get the DI container from TestContextBase and feed all necessary services into the new renderer.

EDIT: As the extension method and renderer life in different assemblies, it will get rather complex to grab all the information necessary.

uecasm commented 1 year ago

It would definitely need to make services from the TestContext available to the markup renderer. While often you're comparing against HTML primitives, sometimes you're not -- for example I have quite a few tests that compare a large component against smaller components that end up generating complex SVG internally (but I don't want to write that level of detail into the test source, even if it internally compares at that level).

Granted, I don't think I currently have any tests comparing against components that have complex service dependencies (and certainly not anything async), but it wouldn't surprise me if someone does, even if only a logger.

It does make sense to me for the cut and the MarkupMatches to be using entirely independent renderers, though.

linkdotnet commented 1 year ago

Making it transient should work but might have side effects when you use render or findcomponent while there might be still ongoing async operations. So I would refrain from doing that for now

egil commented 1 year ago

Thanks for the input folks.

@uecasm if you need to compare with the output of a RenderFragment, you could do the following:

@using NUnit.Framework
@using Bunit
@*@inherits Bunit.TestContext*@
@inherits BunitTestContext  /* this uses TestContextWrapper */
@code {

    [Test]
    public void Cancel1()
    {
        var tcs = new TaskCompletionSource();
        var expectedMarkup = Render(@<span>done</span>);
        var cut = Render(@<MyComponent Task="@tcs.Task"/>);

        cut.MarkupMatches(@<span>waiting</span>);

        tcs.SetCanceled();

        cut.WaitForAssertion(() => cut.MarkupMatches(expectedMarkup.Markup));
    }
}
linkdotnet commented 1 year ago

I try to find some time to provide a fix

linkdotnet commented 1 year ago

@egil I pushed a commit on your branch that fixes the test in question - it isn't the cleanest solution, but I guess this is a way forward. Let me know your thoughts and I spent some time to make it nice

egil commented 1 year ago

@uecasm can you try again with the 1.22.16-preview release and see if that solves the issue for you.

uecasm commented 1 year ago

The MCVE code using ContinueWith(..., TaskScheduler.FromCurrentSynchronizationContext()) and the DelegateTo that does not ConfigureAwait(false) still fails with the same exception in 1.22.16-preview.

Same when using .ConfigureAwait(false).GetAwaiter().OnCompleted and not using ConfigureAwait(false) inside DelegateTo.

I'm not really seeing any difference between the two versions.

Just for the sake of only-await (even though most of the above should have been equivalent), I also tried this variant of the MCVE:

    protected override void OnParametersSet()
    {
        var task = Task;
        if (task != _RegisteredTask)
        {
            _RegisteredTask = task;
            _DelegatedTask = task == null ? null : DelegateTo(task);
            RenderWhenDone();
        }

        base.OnParametersSet();
    }

    private async void RenderWhenDone()
    {
        var task = _DelegatedTask;
        if (task != null)
        {
            _ = await Task.WhenAny(task).ConfigureAwait(false);

            if (task == _DelegatedTask)
            {
                _ = InvokeAsync(StateHasChanged);
            }
        }
    }

    private static async Task<object?> DelegateTo(Task task)
    {
        await task;//.ConfigureAwait(false);
        return null;
    }

This fails (in both versions). If the ConfigureAwaits are changed at all (added or removed) then it passes (although one of these combinations passed on my machine but failed on a slower machine in another test, not checked with this new variant).

egil commented 1 year ago

@uecasm just to double check. Does the test fail with the "FragmentContainer was not found" error or does the test just fail its assertion?

uecasm commented 1 year ago

Yes, it's the same FragmentContainer exception.

linkdotnet commented 1 year ago

With the latest version, this component:

@if (Task != null)
{
    @if (Task.IsCompleted)
    {
        <span>done</span>
    }
    else
    {
        <span>waiting</span>
    }
}
@code {
    [Parameter] public Task? Task { get; set; }

    private Task? registeredTask;

    protected override void OnParametersSet()
    {
        var task = Task;
        if (task != registeredTask)
        {
            registeredTask = task;

            _ = task?.ContinueWith((t, o) =>
            {
                if (t == Task)
                {
                    _ = InvokeAsync(StateHasChanged);
                }
            }, TaskScheduler.FromCurrentSynchronizationContext());
        }

        base.OnParametersSet();
    }
}

Passes the following test:

[Fact]
public void MarkupMatchesShouldNotBeBlockedByRendererComplex()
{
    var tcs = new TaskCompletionSource<object?>();

    var cut = Render(@<InvokeAsyncInsideContinueWith Task="@tcs.Task"/> );

        cut.MarkupMatches(@<span>waiting</span>);

    tcs.SetResult(true);

    cut.WaitForAssertion(() => cut.MarkupMatches(@<span>done</span>));
}

Even the more complex version:

@if (Task != null)
{
    @if (Task.IsCompleted)
    {
        <span>done</span>
    }
    else
    {
        <span>waiting</span>
    }
}
@code {
    [Parameter] public Task? Task { get; set; }

    private Task? registeredTask;
    private Task? delegatedTask;

    protected override void OnParametersSet()
    {
        var task = Task;
        if (task != registeredTask)
        {
            registeredTask = task;
            delegatedTask = task == null ? null : DelegateTo(task);
            RenderWhenDone();
        }

        base.OnParametersSet();
    }

    private async void RenderWhenDone()
    {
        var task = delegatedTask;
        if (task != null)
        {
            _ = await Task.WhenAny(task).ConfigureAwait(false);

            if (task == delegatedTask)
            {
                _ = InvokeAsync(StateHasChanged);
            }
        }
    }

    private static async Task<object?> DelegateTo(Task task)
    {
        await task;//.ConfigureAwait(false);
        return null;
    }
}

Passes - I am a bit puzzled what is going on. @uecasm can you check whether or not some cached bUnit version was taken?

uecasm commented 1 year ago

Are you running the test more than once in a single session? Note the use of three identical test cases in the original post. The first test always passes; it's only the second and third that fail.

Also, while it shouldn't matter, I'm using SetCanceled and not SetResult.

For reference, here's the exception trace from the preview build:

Bunit.Extensions.WaitForHelpers.WaitForFailedException : The assertion did not pass within the timeout period. Check count: 2. Component render count: 2. Total render count: 5.
  ----> Bunit.Rendering.ComponentNotFoundException : A component of type FragmentContainer was not found in the render tree.
   at Bunit.RenderedFragmentWaitForHelperExtensions.WaitForAssertion(IRenderedFragmentBase renderedFragment, Action assertion, Nullable`1 timeout) in /_/src/bunit.core/Extensions/WaitForHelpers/RenderedFragmentWaitForHelperExtensions.cs:line 72
   at BunitTests.MyComponentTest.Cancel2() in D:\Projects\BunitTests\BunitTests\MyComponentTest.razor:line 34
--ComponentNotFoundException
   at Bunit.Rendering.TestRenderer.FindComponent[TComponent](IRenderedFragmentBase parentComponent) in /_/src/bunit.core/Rendering/TestRenderer.cs:line 155
   at Bunit.Extensions.TestContextBaseRenderExtensions.RenderInsideRenderTree(TestContextBase testContext, RenderFragment renderFragment) in /_/src/bunit.core/Extensions/TestContextBaseRenderExtensions.cs:line 45
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, RenderFragment expected, String userMessage) in /_/src/bunit.web/Asserting/MarkupMatchesAssertExtensions.cs:line 303
   at BunitTests.MyComponentTest.<>c__DisplayClass2_0.<Cancel2>b__2() in D:\Projects\BunitTests\BunitTests\MyComponentTest.razor:line 34
   at Bunit.Extensions.WaitForHelpers.WaitForAssertionHelper.<>c__DisplayClass6_0.<.ctor>b__0() in /_/src/bunit.core/Extensions/WaitForHelpers/WaitForAssertionHelper.cs:line 31
   at Bunit.Extensions.WaitForHelpers.WaitForHelper`1.OnAfterRender(Object sender, EventArgs args) in /_/src/bunit.core/Extensions/WaitForHelpers/WaitForHelper.cs:line 153

On the upside, the workaround of pre-rendering the expected content beforehand does do the trick; I haven't had any failures with that regardless of where the ConfigureAwaits are.

linkdotnet commented 1 year ago

@uecasm there should be a new pre-release available on NuGet: 1.22.18-preview It took exactly your test as the basis for the fix - so fingers crossed. Let me know whether or not that fixes your situation.

uecasm commented 1 year ago

Preview 18 does indeed seem to pass the MCVE, in several attempts and variations. (Though it still fails #1159.)