microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.62k stars 303 forks source link

Enable structured logging for RESTler request sequences #829

Closed marina-p closed 1 year ago

marina-p commented 1 year ago

This change adds structured logging for all of the network traffic that is currently logged to the network.*.txt log. The data is written to a 'trace database'. The default format is newline-delimited json, but custom logging formats are supported via a module specified in the engine settings (similar to checkers).

Below is an example for demo_server:

{"sent_timestamp": "2023-11-02T10:10:59.544824+00:00", "received_timestamp": null, "request": "GET /api/blog/posts?page=1&per_page=1 HTTP/1.1\r\nAccept: application/json\r\nHost: localhost\r\nContent-Length: 0\r\nx-restler-sequence-id: fe2172ab-151c-419f-b918-f3e7483b3230\r\n\r\n", "response": null, "request_json": "null", "response_json": null, "tags": {"request_id": "27f9653431313fdc3fecc4a890b72b80b4ce1e59", "sequence_id": "fe2172ab-151c-419f-b918-f3e7483b3230", "combination_id": "1950cbddab7726489624c3d346d3426561c921ad_1", "hex_definition": "6daf8d22c7a6b3472fc83c9f08f290b3507c3ff3", "origin": "main_driver"}}

{"sent_timestamp": null, "received_timestamp": "2023-11-02T10:10:59.597613+00:00", "request": null, "response": "HTTP/1.1 400 Bad Request\r\ndate: Thu, 02 Nov 2023 10:10:58 GMT\r\nserver: uvicorn\r\ncontent-length: 41\r\ncontent-type: application/json\r\n\r\n{\"detail\":\"per_page must be at least 2.\"}", "request_json": null, "response_json": null, "tags": {"request_id": "27f9653431313fdc3fecc4a890b72b80b4ce1e59", "sequence_id": "fe2172ab-151c-419f-b918-f3e7483b3230", "combination_id": "1950cbddab7726489624c3d346d3426561c921ad_1", "hex_definition": "6daf8d22c7a6b3472fc83c9f08f290b3507c3ff3", "origin": "main_driver"}}

Testing:

marina-p commented 1 year ago

Just fixed a few issues - please get the latest iteration for testing


In reply to: 1711381128

wilbaker commented 1 year ago

Just fixed a few issues - please get the latest iteration for testing

In reply to: 1711381128

Thanks! I will give this a try.

marina-p commented 1 year ago

TODO: test replay mode with trace DB & update replay sequence to generate a unique sequence ID. #Resolved

marina-p commented 1 year ago

@wilbaker updated the PR with the origin and checker changes.


In reply to: 1792953323