search

microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.6k stars 297 forks source link

Question regarding providing dynamic HTTP headers to the fuzzer #396

Open david-lazar opened 2 years ago

david-lazar commented 2 years ago

Is it possible to give the fuzzer a script to run before each request that updates some of the HTTP headers? For instance, I must sign each request in a specific manner (which is different for each request) and add it to the HTTP header. The signature is dynamic (depends on the request content) so I can't add it as a static field.

Thanks!

marina-p commented 2 years ago

This is not possible now, but we will add this.

In case you or others want to contribute this, the implementation could be very similar to how --token_refresh_cmd is implemented, where an arbitrary command would run taking the entire request text as input, and returning the updated request in stdout.

PatGod commented 2 years ago

Another (perhaps simpler) option would be to write a new checker for this: after each request (possibly failing because of a wrong header), the checker is called and then it re-writes the header and re-send the request with the correct header. See https://github.com/microsoft/restler-fuzzer/blob/main/docs/user-guide/Checkers.md#create-your-own-checker for more information on checkers.

marina-p commented 2 years ago

@PatGod A new checker is not going to work, because once a request fails in the main algorithm, dependent requests will not be run.

PatGod commented 2 years ago

@marina-p This is a good point - thanks for the clarification.

PatGod commented 2 years ago

@marina-p Here is a related suggestion:

the checker API should be extended so that checkers can notify the core engine that they found a valid rendering from a previously invalid request, so that the request (with that new rendering) can be considered as valid and the search can go on.

Today, the PayloadBodyChecker could use this new mechanism to increase the number of valid requests. And in the future, some new 'body-fixer' checker or some new 'header-repair' checker for David Lazar's scenario above could also use this mechanism.

What do you think? Should we file a separate item for this suggestion?

Thanks again!

marina-p commented 2 years ago

@PatGod This new API would not be able to fix the dynamic header scenario in this issue, because here a header's values are a function of the request content and have to be re-computed for every different payload sent. So, for example, as soon as more values are plugged in during fuzzing, or by a different checker, the previous valid header values added by the new checker would be invalidated.

I think we should add a new issue for your suggestion, which would address cases where specific re-usable valid payloads (e.g. particular set of / values of parameters) could be dynamically generated by a checker.