Test cases render problem

[painsAgains]

Sorry to bother you When I read this paper [RESTTESTGEN: Automated Black-Box Testing of RESTful APIs], I was a bit confused about the test case s generation module. According to the paper, RestTestGen constructs an ODG graph based on the OpenAPI, and then test cases are generated according to the order of the ODG graph. I would like to know how test sequences are generated in RestTestGen by traversing the ODG graph. Specifically, is the test sequence generated by selecting an ODG node according to the dependency and then rendering the next node according to the ODG graph if the current node is successful？ Or construct a test sequence template by traversing the node order of the ODG graph and rendering them all for testing?

[davidecorradini]

Hello, test sequences are dynamically built based on the feedback from the API, therefore, the final order depends on the actual interaction of the API. In practice, RTG builds the ODG (there is an edge for every data dependency, i.e., parameter). Among the operations that could not be successfully tested (no 2XX status code was observed), RTG picks the next candidate operation to test based on the following orderings: number of unsatisfied dependencies, CRUD semantics, testing attempts. For instance, the first operations getting tested are those that have less unsatisfied dependencies, that have READ crud semantics (GET HTTP method), that have been tested less during the testing session. Hopefully this is clear. Please get in touch in case you still need help. Best, Davide

[painsAgains]

Hi. Thank you very much for your reply, can I understand your answer this way: the test sequence is generated by first selecting the nodes in the ODG graph that match the dependencies and expanding them according to the dependencies. However, when this node does not receive a 200 response due to parameters or other reasons, the RTG will either tag these operations or add them to a collection for separate processing and re-render them in the test sequence as you have described. But I still have a question for you, when this node fails in the test sequence expansion phase, does it affect the subsequent expansion process? For example, in ODG, there is A->B, when operation B fails the test, will operation A not be tested? If it is convenient, can you tell me where in the project the test sequence generation and parameter filling is implemented. I think I will get a better understanding by reading the code. Best, PAG

[davidecorradini]

All operations will be tested. If operations A fails the test, then we will store a variable testingAttempts = 1. Operation B has the same variable set to 0, so it will be eventually tested. To better understand the algorithm you might look at the ODG-based opertations sorter class: https://github.com/SeUniVr/RestTestGen/blob/main/src/main/java/io/resttestgen/implementation/operationssorter/GraphBasedOperationsSorter.java

[painsAgains]

Thanks a lot ! I will look the code

[painsAgains]

Hello. I would like to add the following feature to RTG: when rendering the request sequence, first render the "required" parameters for testing and then render the other non-required parameters for testing. Which part of the code do I need to change? Thank you very much!

[davidecorradini]

Hello, you will need to create a custom version of the NominalFuzzer. In particular, at line 158 we set the value for all the mandatory parameters, plus the value of non mandatory parameters with a give probability. You can set that probability to 0.