ParameterCreationException

[bungdanar]

Hello,

In my app, I have endpoints (/api/product-tag-category and /api/product-tag-category-coupon) that require complex request body such as nested object and list of object. When I run rtg, I got ParameterCreationException and rtg generated empty request body {} for those endpoints. Is there any workaround or configuration to fix this exception?

For other endpoint (/api/product) that do not require complex request body, rtg generated correct request body.

These are the openapi spec, fuzzing log, and test result: rtg-ouput.zip

[davidecorradini]

Hey there,

I've gone through the log and the OpenAPI spec you provided. It seems the issue boils down to the combined schema "allOf" used in the requests. Right now, RestTestGen isn't equipped to handle combined schemas (like oneOf, allOf, anyOf, etc.) for generating test cases or fuzzing. However, our custom OpenAPI parser can handle them just fine. I'm planning to double-check if the parser is doing its job accurately.

On another note, we're actively working on adding support for combined schemas, and this new feature should be ready in about a month. Meanwhile, if you're up for testing your API, you might want to consider manually providing schemas without these combinations. For instance, you could create a new schema that includes all the individual subschemas from the allOf combined schema.

Oh, one last thing: could you confirm whether the provided spec actually complies with the OpenAPI standard? The use of "allOf" in this unconventional way is something I haven't come across before. Consider the following request body schema:

 "ProductTagCategoryCreatePayload": {
    "type": "object",
    "required": [
      "tags",
      "category"
    ],
    "allOf": [
      {
        "$ref": "#/components/schemas/ProductCreatePayload"
      }
    ],
    "properties": {
      "tags": {
        "$ref": "#/components/schemas/TagsCreatePayload"
      },
      "category": {
        "$ref": "#/components/schemas/CategoryCreatePayload"
      }
    }
}

I think that allOf should be the only child of ProductTagCategoryCreatePayload, and the list of allOf should contain multiple schemas. In my experience I've never met such syntax. Can you please confirm it is legit?

Thank you again.

Best, Davide

[bungdanar]

Thanks very much for the answer. I will try using non combined schemas and run the rtg again.

Regarding my openapi spec, I use https://editor.swagger.io/ to check its validity. It seems my openapi spec is valid and is rendered correctly. Also, there is no error message shown in the editor.

[bungdanar]

I've run the rtg again using spec that doesn't contain combined schema. I don't get any more ParameterCreationException. However, I get another error:

Despite this error, rtg still managed to generate test results. Can this error be ignored?

One last thing, how do we know the number of bugs/errors that have been found by rtg? Is it by manually counting the number of requests that produce response code 500 in the Report folder?

[davidecorradini]

Thanks a lot for getting back to me!

I noticed that there's a class cast exception happening in the ConstraintViolationMutator class. I think I've figured out what's causing it, but I'd really appreciate it if I could test my fix using your API. Could you send me your new OpenAPI specification and the source code or even an executable of the API? This way, I can give my fix a try before it goes live for everyone.

This exception pops up during the error testing phase. Error testing is performed after nominal testing, so, the results you are seeing are the nominal testing results, plus the partial error testing results before the exception occurs.

If you're wondering how many errors RTG found, you have a couple of options. You could manually count them from the report files. Alternatively, there's also the option to check out the built-in coverage metrics in the CoverageReport folder (results might not be as detailed as those from the reports). By the way, we're actively working on making the results display even better, and we've got a cool web GUI in the works that's about to be released soon, where all this information can be seen quickly.

[davidecorradini]

Alternatively, this is the source code of the fixed version of the class ConstraintViolationMutator: ConstraintViolationMutator.zip

Please replace it in your project and let me know if RestTestGen terminates its execution successfully. Thanks!

[bungdanar]

I have updated the ConstraintViolationMutator class but I get build error:

resttestgen-rtg-1  | 
resttestgen-rtg-1  | Welcome to Gradle 7.6.2!
resttestgen-rtg-1  | 
resttestgen-rtg-1  | Here are the highlights of this release:
resttestgen-rtg-1  |  - Added support for Java 19.
resttestgen-rtg-1  |  - Introduced `--rerun` flag for individual task rerun.
resttestgen-rtg-1  |  - Improved dependency block for test suites to be strongly typed.
resttestgen-rtg-1  |  - Added a pluggable system for Java toolchains provisioning.
resttestgen-rtg-1  | 
resttestgen-rtg-1  | For more details see https://docs.gradle.org/7.6.2/release-notes.html
resttestgen-rtg-1  | 
resttestgen-rtg-1  | Starting a Gradle Daemon (subsequent builds will be faster)
resttestgen-rtg-1  | > Task :compileJava
resttestgen-rtg-1  | /app/src/main/java/io/resttestgen/implementation/mutator/ConstraintViolationMutator.java:76: error: cannot find symbol
resttestgen-rtg-1  |         parameter.setValueWithProvider(valueProvider);
resttestgen-rtg-1  |                  ^
resttestgen-rtg-1  |   symbol:   method setValueWithProvider(ParameterValueProvider)
resttestgen-rtg-1  |   location: variable parameter of type LeafParameter
resttestgen-rtg-1  | /app/src/main/java/io/resttestgen/implementation/mutator/ConstraintViolationMutator.java:102: error: cannot find symbol
resttestgen-rtg-1  |             parameter.setValueManually(((String) parameter.getConcreteValue()).substring(0, chosenLength.get()));
resttestgen-rtg-1  |                      ^
resttestgen-rtg-1  |   symbol:   method setValueManually(String)
resttestgen-rtg-1  |   location: variable parameter of type StringParameter
resttestgen-rtg-1  | /app/src/main/java/io/resttestgen/implementation/mutator/ConstraintViolationMutator.java:107: error: cannot find symbol
resttestgen-rtg-1  |             parameter.setValueManually(parameter.getConcreteValue() +
resttestgen-rtg-1  |                      ^
resttestgen-rtg-1  |   symbol:   method setValueManually(String)
resttestgen-rtg-1  |   location: variable parameter of type StringParameter
resttestgen-rtg-1  | /app/src/main/java/io/resttestgen/implementation/mutator/ConstraintViolationMutator.java:131: error: invalid method reference
resttestgen-rtg-1  |         chosenValue.ifPresent(parameter::setValueManually);
resttestgen-rtg-1  |                               ^
resttestgen-rtg-1  |   cannot find symbol
resttestgen-rtg-1  |     symbol:   method setValueManually()
resttestgen-rtg-1  |     location: class NumberParameter
resttestgen-rtg-1  | 4 errors
resttestgen-rtg-1  | 
resttestgen-rtg-1  | > Task :compileJava FAILED
resttestgen-rtg-1  | 1 actionable task: 1 executed
resttestgen-rtg-1  | 
resttestgen-rtg-1  | FAILURE: Build failed with an exception.
resttestgen-rtg-1  | 
resttestgen-rtg-1  | * What went wrong:
resttestgen-rtg-1  | Execution failed for task ':compileJava'.
resttestgen-rtg-1  | > Compilation failed; see the compiler error output for details.
resttestgen-rtg-1  | 
resttestgen-rtg-1  | * Try:
resttestgen-rtg-1  | > Run with --stacktrace option to get the stack trace.
resttestgen-rtg-1  | > Run with --info or --debug option to get more log output.
resttestgen-rtg-1  | > Run with --scan to get full insights.
resttestgen-rtg-1  | 
resttestgen-rtg-1  | * Get more help at https://help.gradle.org
resttestgen-rtg-1  | 
resttestgen-rtg-1  | BUILD FAILED in 1m 9s

Here, if you still want to use my app for testing, you can run it with docker compose. The app may take while before it is ready. Just check the app log to ensure the app is already running. app.zip

[davidecorradini]

Right, the error is caused by some different methods that we have added in the development branch and that are not public, so they are incompatible with your version. I will test it locally with your app and in a couple days I will release RTG v23.08 with the fix. Thanks again

[davidecorradini]

Bug fixed with release v23.08 coming soon.

[davidecorradini]

@bungdanar Released v23.09 with fix.

[bungdanar]

Thanks for the update. I've already tried the new version and didn't get StringParameter error anymore.