Misleading error message using != (not equal) operator

Thomasdezeeuw commented 5 months ago

What is the current bug behavior?

The != (not equal) operator creates an error message where it "expects" the value we don't want to match. See

error: Assert failure
    --> FILE.hurl:3682:0
3682 | jsonpath "$PATH" != "NOT EXPECTED"
     |   actual:   none
     |   expected: string <NOT EXPECTED>

Steps to reproduce

Create webserver that returns some JSON data, doesn't matter what, as long as it doesn't match below.
Create Hurl file that makes a request and has the following assertion:

jsonpath "$.field" != "Value we don't expect"

What is the expected correct behavior?

An error message that says expected anything but value. For example

error: Assert failure
    --> FILE.hurl:3682:0
3682 | jsonpath "$PATH" != "NOT EXPECTED"
     |   actual:   none
     |   expected anything but: string <NOT EXPECTED>

Execution context

Hurl Version (hurl --version):

hurl 4.3.0 (x86_64-pc-linux-gnu) libcurl/8.8.0 OpenSSL/3.3.0 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libssh2/1.11.0 nghttp2/1.62.1 quic/nghttp3/1.3.0
Features (libcurl):  alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 IDN IPv6 Largefile libz NTLM SPNEGO SSL TLS-SRP UnixSockets zstd
Features (built-in): brotli

Possible fixes

Thomasdezeeuw commented 5 months ago

The error is also misleading if the condition fails (i.e. we get the value that we didn't want):

error: Assert failure
    --> FILE.hurl:2038:0
     |
     | GET URL
     | ...
2038 | jsonpath "$.data" != "text"
     |   actual:   string <text>
     |   expected: string <text>
     |

fabricereix commented 5 months ago

Thanks @Thomasdezeeuw for reporting the bug. The error message is indeed misleading.

We could simply display the expected type along with the predicate

error: Assert failure
    --> FILE.hurl:3682:0
3682 | jsonpath "$PATH" != "NOT EXPECTED"
     |   actual:   none
     |   expected: string != "NOT EXPECTED"

fabricereix commented 5 months ago

or repeating the predicate may be overkilling

error: Assert failure
    --> FILE.hurl:3682:0
3682 | jsonpath "$PATH" != "NOT EXPECTED"
     |   actual:   none
     |   expected: string

fabricereix commented 5 months ago

with the exists predicate, we display the following error message

error: Assert failure
  --> tests_failed/predicate.hurl:43:0
   |
   | GET http://localhost:8000/predicate/error/type
   | ...
43 | jsonpath "$.not-exist" exists
   |   actual:   none
   |   expected: something
   |

We could also reuse the same message (something, similar to your anything).

Orange-OpenSource / hurl