Open marina-p opened 3 years ago
Is there any property to use for DateTime strings such that their name from/to that is from the swagger translated to proper date-time string format?
For example, OpenAPI YAML:
- in: query
name: to
description: You can find things that were raised before this time.
docs-review: true
schema:
type: string
format: datetime
instead of being translated to fuzzstring;
"restler_fuzzable_string": [
"fuzzstring"
],
Could it be translated to:
"restler_fuzzable_datetime_string": [
"2021-03-09T16:01:13Z"
],
And what is the short-term solution to bypass 400 errors b/c of the above problem?
Hello @mbuki,
restler_fuzzable_datetime
can be used in the dictionary. Are you saying that in the grammar.py these are restler_fuzzable_string
? If yes, this looks like a bug - could you please copy/paste the grammar.py snippet you get for the above schema?
As a workaround, you can use restler_custom_payload
, but values need to be specified separately for each of the parameters. So, in the above:
restler_custom_payload {
"to": ["2021-03-09T16:01:13Z", "2021-03-10T16:01:13Z"],
"from": ["2021-03-08T16:01:13Z", "2021-03-11T16:01:13Z"]
}
Another less precise/verbose workaround would be to enter these values into the list of restler_fuzzable_string
values. This will cause more parameter combinations to be tried, but eventually RESTler should plug in those values into the query parameter.
Yes, in the grammar.py these are restler_fuzzable_string.
primitives.restler_static_string("from="),
primitives.restler_fuzzable_string("fuzzstring", quoted=False),
primitives.restler_static_string("&"),
primitives.restler_static_string("to="),
primitives.restler_fuzzable_string("fuzzstring", quoted=False),
primitives.restler_static_string("&"),
There are some additional notes here, so I am explaining from the beginning. When I run the following command from the root directory of my project where the YAML file is:
Restler compile --api_spec openapi.yaml
I get a new folder called `Compile' with all the default grammar, dict, etc. files.
Looking into that generated dict.json, I also see that the datetime property is not in the right format, i.e.,
"restler_fuzzable_datetime": [
"6/25/2019 12:00:00 AM"
]
This is while the server only accepts ISO timestamps. e.g. "2021-02-09T16:54:13Z".
This is my openapi:
- in: query
name: from
description: runtime after this time.
schema:
type: string
format: datetime
- in: query
name: to
description: runtime before this time.
schema:
type: string
format: datetime
So I think issues are two here. One is datetime from schema to produce correct ISO format in dict.json and the second is to have this format create a correct datetime in grammar.py.
As a workaround, you can use restler_custom_payload, but values need to be specified separately for each of the parameters. So, in the above:
Even if I use:
restler_custom_payload {
"to": ["2021-03-09T16:01:13Z", "2021-03-10T16:01:13Z"],
"from": ["2021-03-08T16:01:13Z", "2021-03-11T16:01:13Z"]
}
The former issue persists, i.e., not ISO compliant.
Above all, note that our server does not need quotes around strings, which I see the quoted flag in grammar.py is set to False. So this should be fine as long as the problem from the above matters could be resolved, thanks.
Thanks for these details, this is very helpful.
Could you please clarify, when you change the dictionary (e.g. to add a restler_custom_payload
section), are you re-compiling with
Restler compile config.json
where the config.json points to your new, updated dictionary?
Whenever you add new properties in the dictionary, you need to recompile because RESTler may need to update the grammar. Specifically, when you use restler_custom_payload
, RESTler will replaced the fuzzable string grammar elements with restler_custom_payload("to")
and use the values in the restler_custom_payload
part of the dictionary.
We will update the docs to make this more clear.
I will open a separate bug for these:
One is datetime from schema to produce correct ISO format in dict.json and the second is to have this format create a correct datetime in grammar.py.
Hi @marina-p,
Yes, that's correct. When I change dict.json in Compile folder I do the following. I go to the Compile folder and run:
Restler compile config.json
That generates a new grammar.py based on changes in dict.json. My config.json looks like this:
{
"SwaggerSpecFilePath": [
"/root/openapi.yaml"
],
"GrammarOutputDirectoryPath": "/root/Compile",
"GrammarInputFilePath": "/root/Compile/grammar.json",
"CustomDictionaryFilePath": "/root/Compile/dict.json",
"EngineSettingsFilePath": "/root/Compile/restler_user_settings.json",
"IncludeOptionalParameters": true,
"UseBodyExamples": true,
"UseQueryExamples": true,
"DiscoverExamples": true,
"ExamplesDirectory": "/root/Compile/Example",
"DataFuzzing": true,
"ReadOnlyFuzz": false,
"ResolveQueryDependencies": true,
"ResolveBodyDependencies": true,
"UseRefreshableToken": true,
"AllowGetProducers": false
}
My dict.json looks like this:
{
"restler_fuzzable_string": [
"fuzzstring"
],
"restler_fuzzable_string_unquoted": [],
"restler_fuzzable_datetime": [
"6/25/2019 12:00:00 AM"
],
"restler_fuzzable_datetime_unquoted": [],
"restler_fuzzable_uuid4": [
"903bcc44-30cf-4ea7-968a-d9d0da7c072f"
],
"restler_fuzzable_uuid4_unquoted": [],
"restler_fuzzable_int": [
"0",
"1"
],
"restler_fuzzable_number": [
"0.1",
"1.2"
],
"restler_fuzzable_bool": [
"true"
],
"restler_fuzzable_object": [
"{}"
],
"restler_custom_payload": {
"to": ["2021-03-09T16:01:13Z", "2021-03-10T16:01:13Z"],
"from": ["2021-03-08T16:01:13Z", "2021-03-11T16:01:13Z"]
},
"restler_custom_payload_unquoted": {},
"restler_custom_payload_uuid4_suffix": {}
}
And so when I run the compile command stated above with the settings described, I still get the following:
....
primitives.restler_static_string("&"),
primitives.restler_static_string("from="),
primitives.restler_fuzzable_string("fuzzstring", quoted=False),
primitives.restler_static_string("&"),
primitives.restler_static_string("to="),
primitives.restler_fuzzable_string("fuzzstring", quoted=False),
primitives.restler_static_string("&"),
primitives.restler_static_string("sort="),
primitives.restler_static_string("["),
primitives.restler_fuzzable_string("fuzzstring", quoted=True),
primitives.restler_static_string("]"),
primitives.restler_static_string("&"),
....
I see 4 issues here:
style: form
explode: false
in the open api for object properties, we have to create a list without brackets, and quotations. See Query Parameters table, second row, in https://swagger.io/docs/specification/serialization/.
I found this today, and it seems the compiler only supports the date-time format for strings, of the ones stated in the swagger specification. For me, this caused issues with 'date'. I'm on 7.4, but from the code this seems to still be a problem in 8. I just added 'date' to the 'date-time' case in the swaggervisitor in my local source tree, but presumably this is not the fully correct solution (and mainly works because I know how the API I'm testing works).
Hello @larsivi, thanks for bringing this up. We have added 'date' recently, it's included in 8.1.0 (yep, adding it is a bit more involved at this point than just adding the case to the code... we're looking into to improve the implementation to avoid having to extend the engine for every type of format, see issue #369. If you read the description in that issue, let us know if you have feedback.)
Thanks!
Marina
For the following Swagger defined properties:
Currently, these are compiled to
restler_fuzzable_string
.It should instead be
restler_fuzzable_date_span
andrestler_fuzzable_uri
.