Diffy doesn't output differences when endpoints have responses with non-200 status code

[joaoccgomes]

I'm trying to use Diffy but I have a strange behaviour when the endpoints have responses with a status code different from 200OK. Seems that Diffy only checks responses with status code 200OK, if they are 4xx I cannot see any request in the Diffy UI. Is it supposed just to work with success responses?

[puneetkhanduri]

That is strange. Diffy is designed to work with 4XX responses just as well as it does with 200. I just tested some endpoints hardcoded to return 400 status codes and am able to see them on the UI. Are you able to verify the networking and see that your primary, secondary and candidate instances are receiving traffic from Diffy?

[lbatalha]

(coworker here) We have found that if the Primary/Secondary returns a 404 and the Candidate returns a 200, nothing is shown in the Diffy webUI. The opposite appears to work just fine. Is this working as intended?

[puneetkhanduri]

No. You should be able to see the differences in both cases. I also just verified this behavior. Can you share the differences.log file generated by diffy.

[puneetkhanduri]

you should be seeing something like this for your scenario.

[lbatalha]

deleted and re-created docker container (diffy/diffy), cleaned up differences.log with echo "" > differences.log (it seems to have some test data already there from when the container was built?), verified that it was empty. Sent a request with an endpoint that only exists in the candidate (/ping). Nothing is written to differences.log, nothing in webui. Admin web ui shows successful requests to all 3 backends. Sent another requests to an endpoint that does not exist in both candidate and current backends and that worked (there was a diff in the webui and in differences.log)

Also (but unrelated since same behaviour happens without it) we have nginx reverse proxying all requests on port 80 back to the candidate node with a mirror option that then mirrors the request to diffy (as this allows us to see a response when submitting the requests). Running the /ping request replies with the appropriate response (pong).

The image you linked is what shows up when we have a primary return 200 and candidate return 404, just not the opposite for some reason.

EDIT: our docker run arguments:

  -p 8080:8080 -p 8881:8881 -p 8888:8888 \
  artifacts.slu.bskyb.com:5003/diffy/diffy \
    -candidate=CANDIDATE:80 \
    -master.primary=OLDAPI01:4031 \
    -master.secondary=OLDAPI01:4031 \
    -service.protocol=http \
    -serviceName="some-api" \
    -proxy.port=:8080 \
    -admin.port=:8881 \
    -http.port=:8888 \
    -rootUrl=localhost:8888 -excludeHttpHeadersComparison=true

Requesting directly (without nginx):

curl -v localhost:8080/ping
* About to connect() to localhost port 8080 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /ping HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Length: 0
<

Diffy Admin UI:

[puneetkhanduri]

Can you ping me at puneet.khanduri@gmail.com. It might be easier to debug this with screen sharing via hangouts.

[puneetkhanduri]

Turns out this was a case of unsupported content-type where the response header did not contain application/json or text/html. Issue was fixed by having the downstream targets supply the expected content-type header values.

[lbatalha]

After a very helpful hangouts, it was discovered that the specific endpoint we were using to test this "issue" was returning a content type of text/plain on the candidate side. After enabling DEBUG logging for ai.diffy.lifter.HttpLifter the stack traced showed a parse error. @puneetkhanduri confirmed that Diffy does not support that content type at this point, only the usual HTML and JSON suspects.

For our use case this is fine because all normal application endpoints use one of those.