Test Marked As Passed Even If Different Error Type Is Returned

[mr-feek]

👋 I noticed that my Codeception tests pass even if I expect a different type of error from psalm.

Steps To Reproduce Write a gherkin test snippet and include Then I see these errors in the test. Expect a specific error such as | MissingFile | Cannot find file | when require_once file/that/doesn't/exist.php is analyzed.

An example is here: https://github.com/psalm/psalm-plugin-laravel/pull/99/commits/d69377b08fd16567de5ea706e8dc3e336afc196e#diff-dc52f1aabf7e9388e148c112a4b349c1R32

Expected

• Test fails because UnresolvableInclude was returned by psalm instead of the expected MissingFile

Actual

• Test passes

[mr-feek]

I just realized I have the wrong syntax. I didn't include | Type | Message in my assertion. Should we complain loudly about that to avoid a false positive like what I ran into?

[weirdan]

First line of the table element is ignored because I thought it would be useful to have headers. | Type | Message | is just a convention.

Perhaps the module should complain when:

• there's just one row in the table
• the first cell contains a known error type (indicating that header row was omitted)