search

ThreeMammals / Ocelot

.NET API Gateway
https://www.nuget.org/packages/Ocelot
MIT License
8.32k stars 1.63k forks source link

Errors when using rate limiting #659

Closed skyflyer closed 5 years ago

skyflyer commented 5 years ago

Expected Behavior / New Feature

No errors present in the log file (stdout output)

Actual Behavior / Motivation for New Feature

An error is present in the log output:

fail: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0]
      requestId: 0HLHFFP8GNIEK:00000001, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack:    at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response)
         at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context)
         at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context)
         at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) RequestId: 0HLHFFP8GNIEK:00000001, exception: System.NullReferenceException: Object reference not set to an instance of an object.

Steps to Reproduce the Problem

I have configured a ReRoute with the following config:

{
            "DownstreamPathTemplate": "/{everything}",
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/{everything}",
            "UpstreamHttpMethod": [
                "GET",
                "POST"
            ],
            "RateLimitOptions": {
                "ClientWhiteList": [],
                "EnableRateLimiting": true,
                "Period": "5s",
                "Limit": 1,
                "PeriodTimespan": 10
            }
        }

I then sent several successive requests to the ocelot gateway and observed the error in the log file: for i in {1..5} ; do curl -i http://localhost:5000/ ; done.

Full log file:

Full log file ``` Using launch settings from /gateway/src/Foobar.Gateway/Properties/launchSettings.json... Hosting environment: Development Content root path: /gateway/src/Foobar.Gateway Now listening on: http://localhost:5000 Application started. Press Ctrl+C to shut down. info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1] Request starting HTTP/1.1 GET http://localhost:5000/ info: Ocelot.Authentication.Middleware.AuthenticationMiddleware[0] requestId: 0HLHFFVQP5L6F:00000001, previousRequestId: no previous request id, message: No authentication needed for / info: Ocelot.Authorisation.Middleware.AuthorisationMiddleware[0] requestId: 0HLHFFVQP5L6F:00000001, previousRequestId: no previous request id, message: /{everything} route does not require user to be authorised info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2] Request finished in 349.1088ms 200 text/html; charset=utf-8 info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1] Request starting HTTP/1.1 GET http://localhost:5000/ info: Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware[0] requestId: 0HLHFFVQP5L6G:00000001, previousRequestId: no previous request id, message: Request get:/ from ClientId client has been blocked, quota 1/5s exceeded by 2. Blocked by rule /{everything}, TraceIdentifier 0HLHFFVQP5L6G:00000001. fail: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0] requestId: 0HLHFFVQP5L6G:00000001, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack: at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) RequestId: 0HLHFFVQP5L6G:00000001, exception: System.NullReferenceException: Object reference not set to an instance of an object. at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2] Request finished in 32.8502ms 429 info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1] Request starting HTTP/1.1 GET http://localhost:5000/ info: Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware[0] requestId: 0HLHFFVQP5L6H:00000001, previousRequestId: no previous request id, message: Request get:/ from ClientId client has been blocked, quota 1/5s exceeded by 3. Blocked by rule /{everything}, TraceIdentifier 0HLHFFVQP5L6H:00000001. fail: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0] requestId: 0HLHFFVQP5L6H:00000001, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack: at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) RequestId: 0HLHFFVQP5L6H:00000001, exception: System.NullReferenceException: Object reference not set to an instance of an object. at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2] Request finished in 0.9687ms 429 info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1] Request starting HTTP/1.1 GET http://localhost:5000/ info: Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware[0] requestId: 0HLHFFVQP5L6I:00000001, previousRequestId: no previous request id, message: Request get:/ from ClientId client has been blocked, quota 1/5s exceeded by 4. Blocked by rule /{everything}, TraceIdentifier 0HLHFFVQP5L6I:00000001. fail: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0] requestId: 0HLHFFVQP5L6I:00000001, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack: at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) RequestId: 0HLHFFVQP5L6I:00000001, exception: System.NullReferenceException: Object reference not set to an instance of an object. at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2] Request finished in 1.497ms 429 info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1] Request starting HTTP/1.1 GET http://localhost:5000/ info: Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware[0] requestId: 0HLHFFVQP5L6J:00000001, previousRequestId: no previous request id, message: Request get:/ from ClientId client has been blocked, quota 1/5s exceeded by 5. Blocked by rule /{everything}, TraceIdentifier 0HLHFFVQP5L6J:00000001. fail: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0] requestId: 0HLHFFVQP5L6J:00000001, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack: at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) RequestId: 0HLHFFVQP5L6J:00000001, exception: System.NullReferenceException: Object reference not set to an instance of an object. at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2] Request finished in 1.9146ms 429 ```

According to the documentation, I understand that there is 1 request per 5 seconds allowed. But sending multiple requests in succession does not result in rate limiting.

Specifications

TomPallister commented 5 years ago

@skyflyer can you share your program / startup.cs

skyflyer commented 5 years ago

@TomPallister,

here is a simple repro program: ocelotratelimiterror.zip

I'm running a server on localhost:8000 so that Ocelot can proxy it. And I'm issuing requests from bash with for i in {1..5} ; do curl -i http://localhost:5000/api/ocelot.json ; done. Alternatively, wrk http://localhost:5000/api/ocelot.json does the same trick.

FWIW, I'm running this on Mac.

TomPallister commented 5 years ago

@skyflyer i think this is probably a bug, maybe when Ocelot does the rate limiting it isn't setting something required by the responder middleware.

NicolasPrivatViseo commented 5 years ago

Hi all,

I am facing the same issue, I have an error in the logs when reaching the rate limit. There is a NullReferenceExceptionin the SetResponseOnHttpContext method of Ocelot.Responder.HttpContextResponder class. The responseparameter is null.

Here is the stacktrace I got:

Ocelot.Errors.Middleware.ExceptionHandlerMiddleware:Error: requestId: 0HLJ41QOS0LJ1:00000003, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack:    at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) in C:\src\Ocelot\Responder\HttpContextResponder.cs:line 27
   at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) in C:\src\Ocelot\Responder\Middleware\ResponderMiddleware.cs:line 45
   at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) in C:\src\Ocelot\Middleware\Pipeline\MapWhenMiddleware.cs:line 40
   at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) in C:\src\Ocelot\Errors\Middleware\ExceptionHandlerMiddleware.cs:line 53 RequestId: 0HLJ41QOS0LJ1:00000003, exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response) in C:\src\Ocelot\Responder\HttpContextResponder.cs:line 27
   at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context) in C:\src\Ocelot\Responder\Middleware\ResponderMiddleware.cs:line 45
   at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context) in C:\src\Ocelot\Middleware\Pipeline\MapWhenMiddleware.cs:line 40
   at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) in C:\src\Ocelot\Errors\Middleware\ExceptionHandlerMiddleware.cs:line 53

I guess the method should not be called when response is null (in the Invoke method of the Ocelot.Responder.Middleware.ResponderMiddlewareclass)

Despite the error message my api is not called when the rate limit is reached, so that's ok on this side.

rwestonbutler commented 5 years ago

FWIW, I'm getting a similar exception when trying to return an Unauthorized HttpResponseMessage from a delegating handler.

Code:

protected override async Task<HttpResponseMessage> SendAsync(
            HttpRequestMessage request,
            CancellationToken cancellationToken)
        {
         HttpResponseMessage unauthorizedResult =
                new HttpResponseMessage(HttpStatusCode.Unauthorized) { RequestMessage = request };

            if (!request.Headers.TryGetValues("someheader", out IEnumerable<string> someheaderValues))
            {
                TaskCompletionSource<HttpResponseMessage> tsc = new TaskCompletionSource<HttpResponseMessage>();
                tsc.SetResult(unauthorizedResult);
                return await tsc.Task;
            return await base.SendAsync(request, cancellationToken);
            }

Stack Trace:

Ocelot.Errors.Middleware.ExceptionHandlerMiddleware:Error: requestId: 0HLJRUNOU5HNK:00000001, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack:    at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response)
   at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context)
   at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context)
   at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) RequestId: 0HLJRUNOU5HNK:00000001, exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response)
   at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context)
   at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context)
   at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context)
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request finished in 1544.0562ms 500
stefankip commented 5 years ago

Same issue here:

Error: requestId: 0HLKNSPSC2JI0:0000001F, previousRequestId: no previous request id, message: Exception caught in global error handler, exception message: Object reference not set to an instance of an object., exception stack: at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response)
at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context)
at MyCustomCode.Startup.<>c.<<Configure>b__4_3>d.MoveNext() in /src/Startup.cs:line 158
--- End of stack trace from previous location where exception was thrown ---
at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context)
at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) RequestId: 0HLKNSPSC2JI0:0000001F, exception: System.NullReferenceException: Object reference not set to an instance of an object.
at Ocelot.Responder.HttpContextResponder.SetResponseOnHttpContext(HttpContext context, DownstreamResponse response)
at Ocelot.Responder.Middleware.ResponderMiddleware.Invoke(DownstreamContext context)
at MyCustomCode.Startup.<>c.<<Configure>b__4_3>d.MoveNext() in /src/Startup.cs:line 158
--- End of stack trace from previous location where exception was thrown ---
at Ocelot.Middleware.Pipeline.MapWhenMiddleware.Invoke(DownstreamContext context)
at Ocelot.Errors.Middleware.ExceptionHandlerMiddleware.Invoke(DownstreamContext context) (test-vergelijken.kpnnetwerk.nl)

Line 158 is inside the PreErrorResponderMiddleware: await next.Invoke();