Query stripped from endpoints

[cloudtest-anon]

Description

RESTler is stripping query from endpoints when trying different combinations for an endpoint (the last combination). In HTTP traffic logs (network.testing.xxxx.txt), the log for such requests has the query present as 'restler_static_string' but it is not part of the actual request sent. Here is the log for an example request.

This makes the errors in speccov.json very misleading because speccov.json includes the last request combination for an endpoint and the last combination is the one with its query stripped.

The following is the log in speccov.json for the same request. The error message says that the mandatory HTTP header "x-ms-blob-type" is not specified for this request. This error does not correspond to the endpoint that is being tested i.e., "/{containerName}/{blob}?comp=seal" because it is generated for the endpoint "/{containerName}/{blob}", without query.

["29431971b3bea3460dbcf6ad5662cac3e7783dad_2": {
        "verb": "PUT",
        "endpoint": `"/{containerName}/{blob}?comp=seal",
        "verb_endpoint": "PUT /{containerName}/{blob}?comp=seal",
        "valid": 0,
        "matching_prefix": "None",
        "invalid_due_to_sequence_failure": 0,
        "invalid_due_to_resource_failure": 0,
        "invalid_due_to_parser_failure": 0,
        "invalid_due_to_500": 0,
        "status_code": "400",
        "status_text": `"An HTTP header that's mandatory for this request is not specified.",
        "error_message": "\ufeff<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<Error><Code>MissingRequiredHeader</Code><Message>An HTTP header that&apos;s mandatory for this request is not specified.\nRequestId:a3243340-f01e-009c-323f-145ad8000000\nTime:2022-12-20T06:54:12.4169971Z</Message><HeaderName>x-ms-blob-type</HeaderName></Error>",
        "request_order": 6,
        "sample_request": {
            "request_sent_timestamp": null,
            "response_received_timestamp": "2022-12-20 06:54:12",
            "request_verb": "PUT",
            "request_uri": "/mycontainer905e6e8e45/test_file",
            "request_headers": [
                "Accept: application/json",
                "Host: x.blob.core.windows.net",
                "x-ms-version: 2021-12-02",
                "AUTHORIZATION TOKEN"
            ],
            "request_body": null,
            "response_status_code": "400",
            "response_status_text": "An HTTP header that's mandatory for this request is not specified.",
            "response_headers": [
                "Content-Length: 301",
                "Content-Type: application/xml",
                "Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
                "x-ms-error-code: MissingRequiredHeader",
                "x-ms-request-id: a3243340-f01e-009c-323f-145ad8000000",
                "x-ms-version: 2021-12-02",
                "Date: Tue, 20 Dec 2022 06:54:12 GMT"
            ],
            "response_body": "\ufeff<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<Error><Code>MissingRequiredHeader</Code><Message>An HTTP header that&apos;s mandatory for this request is not specified.\nRequestId:a3243340-f01e-009c-323f-145ad8000000\nTime:2022-12-20T06:54:12.4169971Z</Message><HeaderName>x-ms-blob-type</HeaderName></Error>"
        },
        "tracked_parameters": {}
   }]

Steps to reproduce

• Use RESTler to compile Azure Storage swagger specs.
• Run RESTler in Test mode.

Expected results

Query should not be removed from endpoints.

Actual results

Query removed from combinations of certain requests.

Environment details

OS: Windows, Python: 3.8.2, .NET version: 6.0.404, RESTler version: 9.1.0

[marina-p]

Hello @anna-mazhar,

Could you please try to run your test using the code in the mp/diag branch? I believe this issue should be fixed there (it will also give you a more helpful coverage investigation file at the top level of the logs - see doc updates of that change for details). Please let me know if the issue is still present and I'll try to address it ASAP.

Thanks,

Marina

[cloudtest-anon]

Thank you for replying.

I ran it with code in the mp/diag branch and found the issue to still persist. The last combination request still has its query removed as per the network logs. However, 'speccov.json' is now logging the first combination of the request so from the perspective of improving coverage, speccov.json is no longer misleading. Also, on a side note, speccov-min.json is a good addition to help quick investigation!