Erroneous expected mismatch in 'Request with a multipart body (negative case)'

JP-Ellis commented 1 year ago

I my testing, I believe the following scenario has an incorrect "expected mismatch":

https://github.com/pact-foundation/pact-compatibility-suite/blob/d22d4667c0bda76d408676044cb33db834e7167e/features/V1/http_consumer.feature#L222-L228

When testing it using the Rust FFI in Pact Python, I'm getting the following error message which is similar, but not a match:

MIME part '1': Actual body [application/octet-stream, 50 bytes, starting with 7b0a2020202022626f6479223a2022546869732069732074686520626f647920] is not equal to the expected body [application/octet-stream, 97 bytes, starting with 3c68746d6c3e0a20203c686561643e0a20203c2f686561643e0a20203c626f64]

The full mismatch is:

[
  {
    "method": "PUT",
    "mismatches": [
      {
        "actual": "{\n    \"body\": \"This is the body of the message.\"\n}",
        "expected": "<html>\n  <head>\n  </head>\n  <body>\n    <p>This is the body of the message.</p>\n  </body>\n</html>\n",
        "mismatch": "MIME part '1': Actual body [application/octet-stream, 50 bytes, starting with 7b0a2020202022626f6479223a2022546869732069732074686520626f647920] is not equal to the expected body [application/octet-stream, 97 bytes, starting with 3c68746d6c3e0a20203c686561643e0a20203c2f686561643e0a20203c626f64]",
        "path": "$",
        "type": "BodyMismatch"
      }
    ],
    "path": "/multipart",
    "type": "request-mismatch"
  }
]

rholshausen commented 1 year ago

How is that erroneous? I can't see the difference between what you are getting and the expected error.

This scenario is testing comparing two multiparts bodies where one of the parts have different content.

JP-Ellis commented 1 year ago

The error (at least as far as I can tell), is that the scenario requires the mismatch to be:

Actual body [application/octet-stream, 50 bytes, starting with 7b0a2020202022626f6479223a2022546869732069732074686520626f647920] is not equal to the expected body [application/octet-stream, 97 bytes, starting with 3c68746d6c3e0a20203c686561643e0a20203c2f686561643e0a20203c626f64]

whereas Pact returns:

MIME part '1': Actual body [application/octet-stream, 50 bytes, starting with 7b0a2020202022626f6479223a2022546869732069732074686520626f647920] is not equal to the expected body [application/octet-stream, 97 bytes, starting with 3c68746d6c3e0a20203c686561643e0a20203c2f686561643e0a20203c626f64]

The difference is specifically at the start, the MIME part '1': descriptor.

Want me to make a PR to fix this quickly?

rholshausen commented 1 year ago

No, just ignore that and check that it contains the string from the suite.

JP-Ellis commented 1 year ago

Is it always the case that the mismatch error should be a substring of the full message?

rholshausen commented 1 year ago

Yes, different implementations can generate different prefixes to the errors. I'm pretty sure that if we change that to include the MIME part '1': descriptor it will break the suite for Pact-JVM.

JP-Ellis commented 8 months ago

I'll close this as it's been answered :)

pact-foundation / pact-compatibility-suite

Erroneous expected mismatch in 'Request with a multipart body (negative case)' #4