handling different string formats in RESTler

[marina-p]

Any value can be used as the "format" value for string types in an OpenAPI specification. To better support domain-specific formats, RESTler could handle this in a generic way by generating a new dictionary entry, which the user could then populate. If empty, it would be treated as a restler_fuzzable_string.

For example: restler_custom_payload_format("date") restler_custom_payload_format("uri")

and in the dictionary:

"restler_custom_payload_format": {
  "date": ["2021-10-02"],
  "uri": ["https://example.com"]
}

[mbiuki]

• Is this implemented?
• Is this in addition to datetime and date-time?
• What if those particular string types do not exist while we have them added in the restler_custom_payload_format, are they going to be ignored during compile/run time?
• Is this feature added to the latest container version of RESTler, i.e., 8.1.0?
• In my spec file, I see that format: uri is defined in the responses section. Does it matter? Would RESTler be intelligent enough to pick those no matter we define uri in dict or not? Example:

      '404':
        description: Can't find ....
        x-klm-docs-reviewed: true
        content:
          application/json:
            schema:
              type: object
              required:
                - error
              properties:
                error:
                  type: string
                message:
                  type: string
                someId:
                  type: string
                  format: uuid
                code:
                  type: string
                createdAt:
                  type: string
                  format: date-time
                requestId:
                  type: string
                  format: uuid
                docUrl:
                  type: string
                  format: uri

[mbiuki]

In addition to the above questions, it looks like this feature of dictionary is not yet implemented, I get the following error:

$/Restler/Compile$ cat StdErr.txt
Unhandled exception. System.ArgumentException: Could not read dictionary: ERROR: Cannot deserialize mutations dictionary.  An exception occurred during deserialization: Newtonsoft.Json.JsonSerializationException: Could not find member 'restler_custom_payload_format' on object of type 'MutationsDictionary'. Path 'restler_custom_payload_format', line 37, position 34.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at Microsoft.FSharpLu.Json.Compact.tryDeserializeFile[T](String file)
   at Restler.Workflow.generateGrammarFromSwagger(String grammarOutputDirectoryPath, FSharpOption`1 swaggerDoc, FSharpOption`1 specMetadata, Config config) in /usr/local/bin/restler-fuzzer/src/compiler/Restler.Compiler/Workflow.fs:line 134
   at Restler.Workflow.generateRestlerGrammar(FSharpOption`1 swaggerDoc, Config config) in /usr/local/bin/restler-fuzzer/src/compiler/Restler.Compiler/Workflow.fs:line 270
   at Program.main(String[] argv) in /usr/local/bin/restler-fuzzer/src/compiler/Restler.CompilerExe/Program.fs:line 37

$ Restler --version RESTler version: 8.1.0

[marina-p]

Hi Mehdi,

The issue is open, so this feature is not yet implemented.

Any dictionary entries used today will continue to work for backwards compatibility (such as restler_fuzzable_datetime). Yes, the string formats that do not exist in the spec but do exist in the dictionary would be unused.

For your last question, let's assume the above schema is for the request body instead of the response. Then, yes, RESTler would generate those entries even if they are not defined in the dictionary (similar to how today it generates restler_custom_payload_uuid4_suffix entries).

Thanks,

Marina