microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.52k stars 283 forks source link

Changing state of resources #821

Closed aptly-io closed 8 months ago

aptly-io commented 9 months ago

Description

Certain methods on REST resources only pass if the object is in a correct state first. E.g. before a resource can be delete, it needs to be in a disabled state or before a resource can be acted on, it needs to be in an enabled state.

Is it possible to describe this for Restler when running in the test (and next fuzz mode)? Currently I use openapi's link feature to make sure a certain resource (and its dependencies) exists before updating or deleting it. But how to make sure that it is in the correct state? I think this cannot be described through openapi's links? Is there an alternative feature inside Restler to make this happen? I don't see something inside grammar.json or dependencies.json that can help or am I wrong (is there a description of the syntax of these grammer & dependencies JSON objects)?

Thank you! Kind regards, francis

marina-p commented 9 months ago

Hello @aptly-io,

You are correct, RESTler does not have a specific feature for a general "valid state" pre/post condition on operations. We will look into supporting this.

Could you please give an example of what specific correct state you would like to check for? Perhaps there is a way to express it using RESTler's supported options, and if not this would be great to have as data for designing the feature.

Thanks,

Marina

aptly-io commented 9 months ago

In my use case, as described above, a REST resource can only be deleted if it is in a disabled state. The state is just a flag value in a database table; manipulated with a REST update request. Similarly a REST resource can only be signed (another REST request) if it is in an enabled state and the key REST resource is in an enabled state. In the Restler test phase it fails early on delete, sign, validate etc requests due to the resource not being in the correct state. It would be nice to configure (going beyond the openapi links feature) a certain state before a REST request is fuzzed. I guess it is difficult to express this in JSON though? I hope this clarifies my use case?

marina-p commented 9 months ago

Thank you, yes, that clarifies it.

Could you please give the specific sequence or perhaps share your OpenAPI spec if possible? e.g., for the sequence for signing, here is the level of detail I need to know to check if the existing features of RESTler can be used to unblock your use case:

POST /key
POST /resource
PUT /resource/{id} {... state: enabled }
PUT /resource/{id}/sign

There might be a way to unblock some of these requests using the existing producer-consumer dependencies (as described in Annotations.md) + example payloads.

aptly-io commented 9 months ago

I cannot share the OpenAPI spec unfortunately. I'm going to look into the annotations.md and its examples. The sequence you describe with PUT is exactly the one I'm looking for (POST the key and resource is handled through OpenAPI's links feature). Thank you for that suggestion!

marina-p commented 9 months ago

It sounds like the following setup might work for you:

aptly-io commented 9 months ago

@marina-p Thank you very much, I was looking into how to get the PUT/PATCH updated value in place while you proposed a solution (above). I will try your solution soonish!

aptly-io commented 8 months ago

@marina-p I succeeded fuzzing following your suggestion by creating an annotation and custom dict for a limited subset of the full REST API. E.g. this scenario:

POST resourceA
PATCH resourceA {state:disable}
DELETE resourceA

Is my understanding correct that I should create different configurations to test other inter-dependent scenarios? Because setting the status to disable might be necessary for resourceA in one scenario but it needs to be enabled in another scenario. (the dict.json associates a specific value with a method and path without understanding the scenario context, right?)