lchrusciel
commented
4 years ago PHP Matcher has a backtrace feature added recently. I will take a look at what I can do about it
Closed ostrolucky closed 3 years ago
lchrusciel
commented
4 years ago PHP Matcher has a backtrace feature added recently. I will take a look at what I can do about it
ostrolucky
commented
4 years ago Can you go into more details how would that feature help here? What's your idea?
pbowyer
commented
4 years ago If we could pass options into \Diff() - or our own instance of \Diff - that would help. I get lots of false positives because the indentation of my JSON files is different to PHP's JSON_PRETTY_PRINT output (2 spaces vs 4). Being able to tell the diff to ignore whitespace would be helpful in my case.
lchrusciel
commented
4 years ago For sure it needs some improvement. Backtrace functionality was not that useful (or perhaps I didn't found a proper solution with it). Any ideas are more than welcome :)
norberttech
commented
3 years ago This is known issue of PHPMatcher, as @lchrusciel mentioned there is a Backtrace feature that keeps a record of all actions taken by matcher. The problem is that in case of failure, there is no easy way to say which matcher exactly failed.
Matcher comes with couple predefined matchers that are used recursively. Meaning that there is for example ScalarMatcher, @string@,@integer@, that is also used inside of ArrayMatcher @array@ that is additionally used by @json@ matcher.
Beside those obvious matchers, there are also few custom ones.
Matcher internally is iterating over all registered matchers, checking if it can match given pattern. However json patterns for example are perfectly matchable by @string@ and @json@, obviously @string@ matcher will always fail but it comes first so it will be visible in the backtrace.
If you look at default example in the sandbox https://php-matcher.norbert.tech/ (submit) you should be able to find following failures:
#19 Matcher Coduo\PHPMatcher\Matcher\ScalarMatcher failed to match value
#24 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (scalars) failed to match value
#50 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (scalars) failed to match value
#53 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (array) failed to match value
#76 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (scalars) failed to match value
#79 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (array) failed to match value
#154 Matcher Coduo\PHPMatcher\Matcher\ScalarMatcher failed to match value
#159 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (scalars) failed to match value
#163 Matcher Coduo\PHPMatcher\Matcher\TextMatcher failed to match value
#165 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (array) failed to match valueso as you see, there is no easy way to select the right failure and in this case, nothing even really failed. If you change the @string@.isDateTime() into @integer@ in the same example it will fail but you will find multiple interesting failures:
#142 Matcher Coduo\PHPMatcher\Matcher\ScalarMatcher failed to match value "2014-01-01" with "@integer@" pattern
#143 Matcher Coduo\PHPMatcher\Matcher\ScalarMatcher error: "2014-01-01" does not match "@integer@".
#147 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (scalars) failed to match value "2014-01-01" with "@integer@"
#148 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (scalars) error: "2014-01-01" does not match "@integer@".
#151 Matcher Coduo\PHPMatcher\Matcher\TextMatcher failed to match value "2014-01-01" with "@integer@" pattern
#153 Matcher Coduo\PHPMatcher\Matcher\ChainMatcher (array) failed to match value "2014-01-01" with "@integer@"
#155 Matcher Coduo\PHPMatcher\Matcher\ArrayMatcher failed to match value "Array(1)" with "Array(1)" patternLong Story Short I don't know how to make error messages more detailed in a generic way that would work for all matchers. Backtrace didn't help here but it still brings values since thanks to it I'm able to visualize matching order.
It was already reported some time ago https://github.com/coduo/php-matcher/issues/197 here, and here https://github.com/coduo/php-matcher/issues/191
I'm open for any ideas about how to improve error message verbosity and reduce it only to different values. I'm not even expecting pull requests or fully baked solutions, ideas are more than welcome, thanks in advance!
norberttech
commented
3 years ago actually I think I found a way to make those error messages a bit more descriptive, https://github.com/coduo/php-matcher/pull/225 - please let me know what do you think, thanks!
norberttech
commented
3 years ago It's live, you can play see the results here - guess this issue can be closed now?
ostrolucky
commented
3 years ago You posted link to localhost, did you intend to do that?
norberttech
commented
3 years ago lol of course not, it's fixed :P
ostrolucky
commented
3 years ago Yep looks awesome. Thanks! Indeed I think we can close :)
lchrusciel
commented
3 years ago @ostrolucky have you checked it already on ApiTestCase? I'm wondering if is there anything we need to provide support for it. Perhaps VoidBacktrace will break it.
norberttech
commented
3 years ago Nah, it should be good out of the box, it's not related to the Backtrace at all
Every time test case fails, it's massive PITA to figure out which field is actually different when using
@integer@notations, because during diffs (which is triggered on failure only) it uses them as literals. So output is like following. So bad in figuring out what is actually different:It's a big pain for us with this library. Anything we can do clean output out of false positives?