postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.86k stars 1.16k forks source link

Newman fails with "Parse Error" and no other details when Imperva inserts "bot detection (classification)" cookie into repsonse. #2747

Open noahlz opened 3 years ago

noahlz commented 3 years ago
  1. Newman Version (can be found via newman -v):
  2. OS details (type, version, and architecture):
  3. Command / script used to run Newman:
    $ docker run postman/newman:4-alpine -v
    4.6.1
  4. Are you using Newman as a library, or via the CLI? - CLI
  5. Did you encounter this recently, or has this bug always been there: Recently
  6. Expected behaviour: Print more details about the error / hints at resolution

We were blocked for weeks because newman would fail with only the following:

Parse error

No details at all about what couldn't be parsed. I used htmlextra reporter to try to capture information about the response such as headers, body. This yielded absolutely nothing:

image

Finally we contacted our CDN+WAF, Imperva (formerly Incapsula). They confirmed that sometimes to detect bots they insert a "classification cookie" into their responses. Explanation: Browsers can handle the cookie, but Bots cannot. When they disabled this "classification cookie" feature, the Parse Error issue went away.

See: https://www.imperva.com/blog/how-incapsula-client-classification-challenges-bots/

I'm creating this issue primarily in hopes that Imperva will work with Postman Labs & Co to resolve this issue, or at least so other people can see they aren't alone in if they encounter this issue (perhaps future me).

Ideally, the Parse Error should include more verbosity. For example, if the cookie was corrupted / invalid, it should report that rather than failing without any details.

(Imperva mentioned that they intentionally send a bad cookie that browsers discard but bots choke on).

noahlz commented 3 years ago

See also #2725