Sequence of requests

[qixin5]

Hi RESTler authors,

I'm using RESTler to test a small local-hosted service, for which RESTler doesn't seem to infer a lot of producer-consumer dependencies (please see attached dependencies.json.txt). In the fuzzing phase, however, I still see a lot of sequences of requests generated in various lengths (please see attached network.testing*.txt). I wonder why? If there is no dependency relationship, would a request be appended to any existing sequence? Is it because of checkers? Thanks! dependencies.json.txt network.testing.140060271036160.1.txt

[PatGod]

In fuzz mode, RESTler will try to generate all kinds of request sequences, including requests that may not be related with dependencies.

Still, for this job, adding annotations or fixing the Swagger spec should increase the meaningfulness of the tests performed by RESTler. For more on annotations, please see https://github.com/microsoft/restler-fuzzer/blob/main/docs/user-guide/Annotations.md

[qixin5]

@PatGod, thanks!

[qixin5]

Sorry, I understand the issue was closed, but I have a few more questions. I wonder when dependency is not inferred, what rules RESTler uses to chain requests together to produce sequences? Would it try all combinations? Also, I wonder what rules RESTler uses to infer dependencies (does it consider any of the property/parameter names, types, endpoints, and the HTTP verbs; if it compares strings, does it support any fuzzing matching)? Thanks! I'm trying to have a deeper understanding of how RESTler works in my case.

[PatGod]

If you want to understand how RESTler works, please read the research publications we wrote about it. Start with https://patricegodefroid.github.io/public_psfiles/icse2019.pdf Links to all 4 relevant publications can be found in https://github.com/microsoft/restler-fuzzer

[marina-p]

@qixin5 Exact matching is used to compare strings when inferring dependencies, but identifiers are split per the convention of the specification (e.g. camelCaseName will be split into 3 words). It's possible that your spec uses a convention not supported by RESTler - see 'NamingConvention' in the documentation /docs/user-guide/CompilerConfig.md

[marina-p]

@qixin5 Can you attach your OpenAPI spec? Dependencies won't be inferred if your spec is missing the full response schemas. Your naming convention is supported, so that's not the issue.

[qixin5]

@PatGod , thanks, I did read the papers (including the icse19 one) and enjoyed doing so, but still had the questions regarding details about dependency inference and handling non-dependent requests (according to icse19 paper Fig 3, lines 14-20, it seems RESTler would only append a request to a sequence with which it has a dependency relationship, so I had my previous question regarding what rules are used for chaining non-dependency requests). Sorry if I miss anything from the papers.

@marina-p , thanks, I attached the spec (swagger.json). I understand that the spec is not perfect and could be improved. But I was curious about the inference algorithm, which I was not sure after reading the papers. And thanks for your explanation. A quick follow-up question: In my case, I wonder why it doesn't infer that "GET /products/{productName}/features" depends on "GET /products/{productName}" which returns an object "Product" which has a property "name". Wouldn't "productName" match "name" after word splitting? swagger.json.txt

[marina-p]

@qixin5 Great, thanks. I see from the spec that the POST doesn't return any schema. Is that by design? Without this, RESTler will not know that to create the product you need to call a POST. It could infer a producer from the GET - there is an compiler option for this in config.json called 'AllowGetProducers'. It's not on by default because it is mainly used in cases where there is no POST and someone is testing against an existing pre-created set of resources. If the POST of your service returns details with the "name", I suggest adding the response schema, and the dependency should be inferred then.

        "responses": {
          "default": {
            "description": "successful operation"
          }

[qixin5]

@marina-p I see, and thanks! The service in its implementation does not return the created resource after a post, but this could be improved.