[qualys_vmdr] Ensure rate-limit headers are honored

[andrewkroh]

The qualys API service uses rate-limiting headers. Let's ensure that the integration honors these to avoid hitting 409s.

Example headers in response:

x-ratelimit-limit: 300
x-ratelimit-window-sec: 3600
x-concurrency-limit-limit: 2
x-concurrency-limit-running: 0
x-ratelimit-towait-sec: 0
x-ratelimit-remaining: 299

References

• https://cdn2.qualys.com/docs/qualys-api-vmpc-user-guide.pdf (search for X-RateLimit-Limit)

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[efd6]

Initial sketch for the rate limit header value construction. Still thinking though.

mito src.cel
! stderr .
cmp stdout want.txt

-- src.cel --
{
    "X-RateLimit-Limit": ["300"],
    "X-RateLimit-Window-Sec": ["3600"],
    "X-Concurrency-Limit-Limit": ["2"],
    "X-Concurrency-Limit-Running": ["0"],
    "X-RateLimit-ToWait-Sec": ["0"],
    "X-RateLimit-Remaining": ["299"],
}.as(headers, 
    rate_limit(headers.with({
        "X-RateLimit-Reset": [string(headers[?"X-RateLimit-ToWait-Sec"][0].orValue("3600"))],
    }), 'X-RateLimit', false, true, duration(string(headers[?"X-RateLimit-Window-Sec"][0].orValue("3600"))+"s"), 0)
).as(rate_headers, rate_headers.with({
    // Work around inf detection in input.
    ?"rate": rate_headers.rate == double("Infinity") ? optional.of("inf") : optional.none(),
    ?"next": rate_headers.next == double("Infinity") ? optional.of("inf") : optional.none(),
}))
.with({"_now":now})
-- want.txt --
{
    "_now": "2024-06-19T05:06:14.429471807Z",
    "burst": 1,
    "headers": "X-RateLimit-Limit=\"300\" X-RateLimit-Remaining=\"299\" X-RateLimit-Reset=\"0\"",
    "next": 0.08333333333333333,
    "rate": "inf",
    "reset": "2024-06-19T05:06:14.429450357Z"
}