search

bUnit-dev / bUnit

bUnit is a testing library for Blazor components that make tests look, feel, and runs like regular unit tests. bUnit makes it easy to render and control a component under test’s life-cycle, pass parameter and inject services into it, trigger event handlers, and verify the rendered markup from the component using a built-in semantic HTML comparer.
https://bunit.dev
MIT License
1.14k stars 105 forks source link

MarkupMatches partially broken since 1.16.2 (regression) #1045

Closed candritzky closed 1 year ago

candritzky commented 1 year ago

Describe the bug

Comparison of the markup of a rendered component is broken for medium complex markup since version 1.16.2.

Example: Here's the actual markup that got rendered (taken from cut.Markup):

<div class="main" b-l9zz05p28t><div class="pane master-pane" b-l9zz05p28t><div class="header" b-l9zz05p28t><div class="zui-header" style="font: var(--zui-typography-h2); " b-9gghzsuirm>Custom Metadata Definitions</div>
            <zui-button id="add-button" emphasis="primary-highlight" blazor:onclick="1" disabled style="width: 100%;">Add new<zui-icon-symbols-plus slot="icon" size="s32"></zui-icon-symbols-plus></zui-button></div><div style="display: flex; align-items: center; align-self: start;"><zui-progress-ring size="s16" emphasis="highlight" mode="activity" style="margin-right: 5px;"></zui-progress-ring>
    <em>Loading...</em></div></div>
    <div class="details-pane-outer" b-l9zz05p28t><div class="placeholder-container" b-nfzpgxci6j><div class="placeholder-header" b-nfzpgxci6j>No custom metadata definition selected</div>
    <div class="placeholder-body" b-nfzpgxci6j>Please select a custom metadata definition from the list to view the details</div></div></div></div>

And this is the markup that it is compared to using cut.MarkupMatches. This worked up to bUnit version <= 1.15.5.

<div class="main">
                    <div class="pane master-pane">
                        <div class="header">
                            <div class="zui-header" style="font: var(--zui-typography-h2);">Custom Metadata Definitions</div>
                            <zui-button id="add-button" emphasis="primary-highlight" disabled style="width: 100%;">
                                Add new
                                <zui-icon-symbols-plus slot="icon" size="s32"></zui-icon-symbols-plus>
                            </zui-button>
                        </div>
                        <div diff:ignoreAttributes>
                            <zui-progress-ring diff:ignoreAttributes></zui-progress-ring>
                            <em>Loading...</em>
                        </div>
                    </div>
                    <div class="details-pane-outer">
                        <div class="placeholder-container" diff:ignoreChildren></div>
                    </div>
                </div>

Since version 1.16.2 it fails with the following error message: An attribute source map cannot be created unless a element comparison source is provided. (Parameter 'elementSource')

CustomMetadataPageTests.Render_ItemsIsNull_ShouldRenderBasicLayout threw exception: 
System.ArgumentException: An attribute source map cannot be created unless a element comparison source is provided. (Parameter 'elementSource')
    at AngleSharp.Diffing.Core.SourceMap..ctor(ComparisonSource& elementSource)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareElementAttributes(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareElement(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareNode(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.<CompareNodes>b__11_0(Comparison comparison)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareElement(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareNode(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.<CompareNodes>b__11_0(Comparison comparison)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareElement(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareNode(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.<CompareNodes>b__11_0(Comparison comparison)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareElement(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareNode(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.<CompareNodes>b__11_0(Comparison comparison)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareElement(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.CompareNode(Comparison& comparison)
   at AngleSharp.Diffing.Core.HtmlDifferenceEngine.<CompareNodes>b__11_0(Comparison comparison)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
   at System.Collections.Generic.SparseArrayBuilder`1.ReserveOrAdd(IEnumerable`1 items)
   at System.Linq.Enumerable.Concat2Iterator`1.ToArray()
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Bunit.CompareToExtensions.CompareTo(INodeList actual, INodeList expected) in /_/src/bunit.web/Asserting/CompareToExtensions.cs:line 106
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(INodeList actual, INodeList expected, String userMessage) in /_/src/bunit.web/Asserting/MarkupMatchesAssertExtensions.cs:line 234
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, String expected, String userMessage) in /_/src/bunit.web/Asserting/MarkupMatchesAssertExtensions.cs:line 113
   at CustomMetadataPageTests.Render_ItemsIsNull_ShouldRenderBasicLayout()

Expected behavior:

It should work the same way as before.

Version info:

Additional context:

I first assumed this problem was caused by the upgrade of AngleSharp.Diffing, but bUnit 1.15.5 and 1.16.2 both use AngleSharp.Diffing 0.17.1. So it seems to be a problem somewhere within bUnit.

linkdotnet commented 1 year ago

Hey @candritzky,

Thanks for the detailed report. Indeed at first glance, that seems like a regression. If we have a look at the diff there is only one addition: BlazorDiffingHelpers.UnknownElementMatch and indeed if I remove the matcher your test works.

Not sure why that happens, but once I have a bit more time to spare, I will investigate further.

Update 1: The reason the unknownmatcher is invoked is because of zui-button and zui-icon-symbols-plus. With web components in mind these are valid use cases.

linkdotnet commented 1 year ago

PR coming...

linkdotnet commented 1 year ago

PR is merged. Thanks @egil for the swift response and @candritzky for the detailed issue description - just superb!

If you want, you can test preview builds:https://github.com/bUnit-dev/bUnit/discussions/209

There should soon be a new preview version available (version 1.19.11-preview). If you have some time to spare, maybe you can check if your specific case is solved with the newest version - if so we can release a new stable version to the public.

candritzky commented 1 year ago

@linkdotnet and @egil Thanks for working on this so quickly, that's amazing!

I tested with bunit 1.19.11-preview, but unfortunately this still results in the same failures.

  Failed Render_ItemsIsNull_ShouldRenderBasicLayout [24 ms]
  Error Message:
   Test method ZENManagementServer.UnitTests.CustomMetadata.CustomMetadataPageTests.Render_ItemsIsNull_ShouldRenderBasicLayout threw exception:
Bunit.HtmlEqualException: HTML comparison failed.

The following errors were found:
  1: The expected element at div(0) > div(1) > div(1) > zui-button(3) and the actual element at div(0) > div(0) > div(0) > div(0) are different.
  2: The value of the attribute div(0) > div(1) > div(1) > zui-button(3)[style] and actual attribute div(0) > div(0) > div(0) > div(0)[style] are different.
  3: The expected text at div(0) > div(1) > div(1) > zui-button(3) > #text(0) and the actual text at div(0) > div(0) > div(0) > div(0) > #text(0) is different.
  4: The element at div(0) > div(1) > div(1) > zui-button(3) > zui-icon-symbols-plus(1) is missing.
  5: The element at div(0) > div(1) > div(1) > div(1) is missing.
  6: The element at div(0) > div(0) > div(0) > zui-button(2) was not expected.
  7: The attribute at div(0) > div(1) > div(1) > zui-button(3)[id] is missing.
  8: The attribute at div(0) > div(1) > div(1) > zui-button(3)[emphasis] is missing.
  9: The attribute at div(0) > div(1) > div(1) > zui-button(3)[disabled] is missing.
  10: The attribute at div(0) > div(0) > div(0) > div(0)[class] was not expected.

Actual HTML:
<div class="main" >
  <div class="pane master-pane" >
    <div class="header" >
      <div class="zui-header" style="font: var(--zui-typography-h2); " >Custom Metadata Definitions</div>
      <zui-button id="add-button" emphasis="primary-highlight"  disabled="" style="width: 100%;">Add new<zui-icon-symbols-plus slot="icon" size="s32"></zui-icon-symbols-plus>
      </zui-button>
    </div>
    <div style="display: flex; align-items: center; align-self: start;">
      <zui-progress-ring size="s16" emphasis="highlight" mode="activity" style="margin-right: 5px;"></zui-progress-ring>
      <em>Loading...</em>
    </div>
  </div>
  <div class="details-pane-outer" >
    <div class="placeholder-container" >
      <div class="placeholder-header" >No custom metadata definition selected</div>
      <div class="placeholder-body" >Please select a custom metadata definition from the list to view the details</div>
    </div>
  </div>
</div>

Expected HTML:
<div class="main">
  <div class="pane master-pane">
    <div class="header">
      <div class="zui-header" style="font: var(--zui-typography-h2);">Custom Metadata Definitions</div>
      <zui-button id="add-button" emphasis="primary-highlight" disabled="" style="width: 100%;">
        Add new
        <zui-icon-symbols-plus slot="icon" size="s32"></zui-icon-symbols-plus>
      </zui-button>
    </div>
    <div diff:ignoreattributes="">
      <zui-progress-ring diff:ignoreattributes=""></zui-progress-ring>
      <em>Loading...</em>
    </div>
  </div>
  <div class="details-pane-outer">
    <div class="placeholder-container" diff:ignorechildren=""></div>
  </div>
</div>

  Stack Trace:
      at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(INodeList actual, INodeList expected, String userMessage) in /_/src/bunit.web/Asserting/MarkupMatchesAssertExtensions.cs:line 237
   at Bunit.MarkupMatchesAssertExtensions.MarkupMatches(IRenderedFragment actual, String expected, String userMessage) in /_/src/bunit.web/Asserting/MarkupMatchesAssertExtensions.cs:line 113
   at CustomMetadata.CustomMetadataPageTests.Render_ItemsIsNull_ShouldRenderBasicLayout()
linkdotnet commented 1 year ago

Hmm at least we are one step further here. The original exception was because we handled attributes of custom elements - and anglesharp did not like this.

Now we have "just" different HTML output. Let me check

linkdotnet commented 1 year ago

@candritzky do you have a small reproducible example?

candritzky commented 1 year ago

Not yet. Isn't the HTML dump sufficient? If not, I can try to isolate this - tomorrow.

linkdotnet commented 1 year ago

After fiddling around - I got a very small working sample on my own with a proper fix (works also with your given example)

linkdotnet commented 1 year ago

There should be another preview version available soon. I would love to get your feedback on the newest version

candritzky commented 1 year ago

@linkdotnet Hooray, tested with 1.19.12-preview and it works. Thank you so much. Looking forwared for an official release.

egil commented 1 year ago

Good work @linkdotnet. Feel free to create a release.

linkdotnet commented 1 year ago

@candritzky release is out! You can now use the stable 1.19 version.