search

microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.52k stars 283 forks source link

[Feature Request] Additional bug bucketing configurations #820

Open gitmadness opened 9 months ago

gitmadness commented 9 months ago

đź’ˇ Idea

We are looking for more control over RESTler’s bug bucketing functionality to enable additional analysis of RESTler result with more configurable bug bucketing.

We would like to configure the following bucketing behavior.

  1. Maximum number of sequences to look back - e.g. A 0 means only look at the last request + response.
  2. Exclude response or request properties when bucketing - e.g. Exclude request body
  3. Exclude response or request properties for normalization when bucketing - e.g. Don’t normalize request body

Design Notes

No response

marina-p commented 9 months ago

Hello @gitmadness,

Could you please clarify (1) - would 0 mean there would be no bucketization at all?

Would it work for you to provide a custom bug bucketing function? This would work similarly to checkers, where you would specify a python file in the engine settings, which implements an interface that takes the required information for generating a bug bucket and returns the bug hash.

Thanks,

Marina

gitmadness commented 9 months ago

Could you please clarify (1) - would 0 mean there would be no bucketization at all?

Maybe now that you mentioned it. I was just thinking the bucket is based on the last request - response rather than sequence of request-responses.

For example, assume there are 4 request-responses like below:

Request 1 - HTTP 200 Request 2 - HTTP 200 Request 3 - HTTP 200 Request 4 - HTTP 500

If 1 is specified, the bucketing algorithm will only evaluate whether Request 3 is part of the sequence leading to the HTTP 500 hit by Request 4.

If 0 is specified, the bucketing algorithm will not evaluate Requests 1 to 3.

Would it work for you to provide a custom bug bucketing function? This would work similarly to checkers, where you would specify a python file in the engine settings, which implements an interface that takes the required information for generating a bug bucket and returns the bug hash.

Good idea. This might be a more flexible approach.