search

SeUniVr / RestTestGen

A framework for automated black-box testing of RESTful APIs.
Apache License 2.0
36 stars 9 forks source link

Fuzzing Time Duration #14

Closed bungdanar closed 1 year ago

bungdanar commented 1 year ago

Hello, I have questions regarding RTG.

How long is the fuzzing duration performed by RTG and is there any configuration to set how long the fuzzing duration is?

davidecorradini commented 1 year ago

Hello,

currently, RestTestGen does not incorporate a time-based budget feature. The determination of how long to test an API remains within the realm of testing strategies.

Nonetheless, within the NominalAndErrorStrategy, there exists the flexibility to adjust certain parameters. For instance, you can modify the number of attempts for which RestTestGen generates test interactions for a specific operation. This adjustment can either extend or curtail the overall testing time.

The relevant parameter can be found on line 36 within the NominalAndErrorStrategy class. Currently set at 20, this value holds potential for adjustment—perhaps to 100 or even 500, as an illustration.

In instances where your API poses considerable challenges for automated testing, we recommend a slight increment in the parameter on line 24 of the GraphBasedOperationsSorter class (maximumAttempts), from 10 to 15-20. This particular parameter dictates the number of attempts undertaken to test an operation that hasn't yielded a successful status code before giving up on its testing.

Feel free to inform me if I can be of help with regards to the other testing strategies provided by RestTestGen.

bungdanar commented 1 year ago

Hello,

currently, RestTestGen does not incorporate a time-based budget feature. The determination of how long to test an API remains within the realm of testing strategies.

Nonetheless, within the NominalAndErrorStrategy, there exists the flexibility to adjust certain parameters. For instance, you can modify the number of attempts for which RestTestGen generates test interactions for a specific operation. This adjustment can either extend or curtail the overall testing time.

The relevant parameter can be found on line 36 within the NominalAndErrorStrategy class. Currently set at 20, this value holds potential for adjustment—perhaps to 100 or even 500, as an illustration.

In instances where your API poses considerable challenges for automated testing, we recommend a slight increment in the parameter on line 24 of the GraphBasedOperationsSorter class (maximumAttempts), from 10 to 15-20. This particular parameter dictates the number of attempts undertaken to test an operation that hasn't yielded a successful status code before giving up on its testing.

Feel free to inform me if I can be of help with regards to the other testing strategies provided by RestTestGen.

Thanks for the suggestion, I will try the parameter adjustments so they can extend the fuzzing duration.

I have another question. Some of my endpoints require complex request body such as nested object and list of object. When I run rtg, I got ParameterCreationException and rtg generated empty request body {} for those endpoints. Is there any workaround or configuration to fix this exception?

For note, rtg generated correct request body for endpoints that do not require complex request body (simple object).

davidecorradini commented 1 year ago

Hey there,

I will proceed by closing this issue as this is only related to fuzzing time.

Could you do me a favor? If you're experiencing the ParameterCreationException issue, could you open a new issue and explain what's been going on? Honestly, I haven't come across that exception before. It would be super helpful if you could also share the RestTestGen log and the OpenAPI specification file that's causing the problem.

Thanks a bunch for your patience and time.

Take care, Davide