search

microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.59k stars 297 forks source link

Examples sample #224

Open LYDE1234 opened 3 years ago

LYDE1234 commented 3 years ago

Could you provide a sample in the repo that shows how examples can be used? I've read the Examples.md documentation but it would be useful to see demonstration of using an external example file and how the DiscoverExamples=true works with inline examples.

marina-p commented 3 years ago

Thanks for the suggestion. We will add this.

marina-p commented 3 years ago

@LYDE1234 In case this will unblock you, there is an example of this in unit tests which you can use for now as a reference. It shows a referenced payload example: /src/compiler/Restler.Compiler.Test/swagger/array_example.json and the referenced example /src/compiler/Restler.Compiler.Test/swagger/examples/make_order_descriptions.json.

(Note it's a relative path so you will need to put it in an 'examples' sub-directory or update this path in the swagger json)

marina-p commented 3 years ago

DiscoverExamples=true does not apply to inline examples in the specification, only external examples of an entire payload. This is because there is already a way to specify individual values in the restler_custom_payload section of the dictionary.

Question: would it be useful for you if when DiscoverExamples is set to true, those inline example values would be added to the dictionary in the restler_custom_payload section? If yes, can you describe your use case in more detail? Thanks!

LYDE1234 commented 3 years ago

Our use case is to try out RAFT in enough detail to make a decision on how to use it going forward. We're using the API that you can find a slightly outdated version of here:

https://app.swaggerhub.com/apis/AndersBC/XP_API_Test/1.0.0

as our test input. To answer your question, yes, I think if DiscoverExamples is true, all examples in the API should be used by RESTler. If that can be done by putting them in restler_custom_payload, then that's a good idea.

mbiuki commented 3 years ago

DiscoverExamples=true does not apply to inline examples in the specification, only external examples of an entire payload. This is because there is already a way to specify individual values in the restler_custom_payload section of the dictionary.   Question: would it be useful for you if when DiscoverExamples is set to true, those inline example values would be added to the dictionary in the restler_custom_payload section? If yes, can you describe your use case in more detail? Thanks!  

Sorry to meddle in. But, I believe this somehow reflects on general improvements for the Compiler suggested here. It'd be great to see that the first time that RESTler compiles the spec, it can generate a baseline body model to populate examples.json. This way, a user would have an idea about how the body examples can be constructed by using the examples that the RESTler compiler created as a sample from the inline examples of the open API.    Then, if the user wants to use those body examples, she can set the DiscoverExamples flag to true in config.json. In essence, the body examples that are taken from the open API spec to construct the examples.json should not confuse RESTler with restler_custom_payload available in dict.json. Rather it should ameliorate a thorough fuzzing.