[Bug]: Custom messages from expect() are being displayed on HTML report even when there are NOT errors

marcosvfranco commented 8 months ago

Version

1.41.2

Steps to reproduce

clone my repo https://github.com/marcosvfranco/saucedemo-playwright.git
install it with npm i
run npm test
Check the HTML report on this Test Case: "then the products are displayed with their respective prices"
- https://github.com/marcosvfranco/saucedemo-playwright/blob/96a6f0c369ebc135815249f182951a6e06a2af6a/tests/e2e/products.spec.ts#L8
Check the penultimate line of the checks

Expected behavior

A normal description of the expect() should be displayed as normal, as we didn't have the Custom Error Message, like expect.toHaveLength()

Actual behavior

The Custom Error Message is being displayed on the HTML Report even when the Expect is successful, this is also happening on Custom Reports like Allure, but principally on our HTML Playwright Report.

Additional context

No response

Environment

System:
    OS: macOS 14.3.1
    CPU: (10) arm64 Apple M2 Pro
    Memory: 60.23 MB / 16.00 GB
  Binaries:
    Node: 20.9.0 - /usr/local/bin/node
    npm: 10.1.0 - /usr/local/bin/npm
  IDEs:
    VSCode: 1.86.1 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.41.2 => 1.41.2

vladkrasn commented 8 months ago

I think it's by design. "Custom message" you use is meant for custom description of the expect in reporter. Like, if "expect.toHaveLength" is too confusing, you write "That array is of expected length 6".

Custom error messages are handled inside of custom assertions

mxschmitt commented 8 months ago

As per above, this is by design, in the next version we'll adjust our wording and call it just message instead.

Folding into https://github.com/microsoft/playwright/issues/29378.

marcosvfranco commented 8 months ago

As per above, this is by design, in the next version we'll adjust our wording and call it just message instead.

Folding into https://github.com/microsoft/playwright/issues/29378.

Thanks, the documentation will be more clear like this way

microsoft / playwright