Custom generator works only for the first request

[marcellomaugeri]

Description

I am trying to use a custom generator to dynamically generate the body of a POST request to the path /request, however, only the first request contains the value of the custom_generator.py. All the rest of the queries are random strings.

Steps to reproduce

Swagger.json

{
    "openapi": "3.0.0",
    "info": {
        "title": "Test custom",
        "version": "1.0.0"
    },
    "servers": [
        {
        "url": "http://127.0.0.1:8080",
        "description": "Endpoint"
        }
    ],
    "paths": {
        "/test": {
            "post": {
                "summary": "Execute a request",
                "operationId": "execute",
                "requestBody": {
                    "content": {
                        "application/json": {
                            "schema": {
                                "type": "object",
                                "parameters": {
                                    "__body__": {}
                                }
                            }
                        }
                    }
                },
                "responses": {
                "200": {
                    "description": "Successful response",
                    "content": {
                    "application/json": {
                        "schema": {
                        "type": "object"
                        }
                    }
                    }
                }
                }
            }
        }
    }
}

Dict.json

{
  "restler_custom_payload": { "/test/post/__body__": ["{\n\"test\" : \"test\"\n}"] },
  "restler_custom_payload_uuid4_suffix": {},
  "restler_custom_payload_header": {},
  "restler_custom_payload_query": {}
}

custom_generator.py

def gen_restler_custom_payload__request_post__body_(**kwargs):
    while True:
        yield '''{
                    "test": "test1"
                }'''
        yield '''{
                    "test": "test2"
                }'''

value_generators = {
    "restler_custom_payload": {
        "/request/post/__body__": gen_restler_custom_payload__request_post__body_
    }
}

engine_settings.json

{
  "max_combinations": 20,
  "max_request_execution_time": 90,
  "max_async_resource_creation_time": 60,
  "global_producer_timing_delay": 2,
  "dyn_objects_cache_size":20,
  "custom_value_generators": "[FULL PATH TO]/custom_generator.py",
  "per_resource_settings": {}
}

config.json

{
  "SwaggerSpecFilePath": [
    "[FULL PATH TO]/swagger.json"
  ],
  "DataFuzzing": true,
  "CustomDictionaryFilePath": "dict.json",
  "EngineSettingsFilePath": "engine_settings.json",
  "AnnotationFilePath": "annotations.json"
}

Commands:

Restler compile config.json
Restler fuzz-lean --grammar_file Compile/grammar.py --dictionary_file Compile/dict.json --no_ssl --settings Compile/engine_settings.json

Expected results

Alternating between the two bodies defined in custom_generator.py

Actual results

Only the first value defined in custom_generator is sent.

Environment details

Ubuntu 22.04, Python 3.10.12, dotnet 6.0.126, RESTler 9.2.3

[alaendro]

I have experienced the same problem some months ago without a solution

[marcellomaugeri]

@marina-p could you please confirm if this is a bug? I would be glad to fix it with some guidance, thank you.

[marina-p]

Hello @marcellomaugeri,

Apologies for the delay in replying and thank you for the self-contained repro steps. In the above files, the dictionary and custom value generators have a different path:

        "/request/post/__body__": gen_restler_custom_payload__request_post__body_
        "/test/post/__body__": ["{\n\"test\" : \"test\"\n}"] 

If I modify both keys to match the Swagger (/test/post/__body__), RESTler sends both values generated by the custom value generator function.

Thanks,

Marina

[marina-p]

@marcellomaugeri leaving this issue open to further brainstorm how to improve diagnosing this error. There isn't a warning issued today for unused value generators since one use case is to have the same value_gen.py for many Swagger files, which would result in noisy warnings.

[marcellomaugeri]

@marina-p thank you for your solution. Actually, I made a mistake in providing the MRE, but the paths are all correct in my code. Still, it's sending the first value in the yield: { "test": "test1" } at the beginning, all the remaining payloads are just random strings, not taking into consideration neither dict nor other values in the generator. I was expecting Restler to send { "test": "test1" } and { "test": "test2" } alternated. In brief, I would like to have full control of the payload sent, is it possible?