Bug: Downstream request of url/{param} is not working

[msvaillant]

Good day, I have encountered a problem with downstream url forwarding. Let me express it on example:

send GET request to Ocelot by url like http://localhost:5100/api/vacancy/1 it should be downstreamed to http://localhost:5002/api/v1/vacancy/1 but it is http://localhost:5002/api/v1/vacancy/{vacancyid}

configuration looks like this:

    {
      "DownstreamPathTemplate": "/api/v1/vacancy",
      "DownstreamScheme": "http",
      "DownstreamPort": 5002,
      "DownstreamHost": "localhost",
      "UpstreamPathTemplate": "/vacancy",
      "UpstreamHttpMethod": [ "Options",  "Put", "Get", "Post", "Delete" ],
      "ServiceName": "botCore",
      "LoadBalancer": "LeastConnection"
    },
    {
      "DownstreamPathTemplate": "/api/v1/vacancy/{vacancyId}",
      "DownstreamScheme": "http",
      "DownstreamPort": 5002,
      "DownstreamHost": "localhost",
      "UpstreamPathTemplate": "/vacancy/{vacancyId}",
      "UpstreamHttpMethod": [ "Options",  "Put", "Get", "Post", "Delete" ],
      "ServiceName": "botCore",
      "LoadBalancer": "LeastConnection"
    },

I attach Logs to approve behavior.

Ocelot apigateway logs:

dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[3]
      Hosting starting
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[4]
      Hosting started
Hosting environment: Production
Content root path: <path>
Now listening on: http://localhost:5100
Application started. Press Ctrl+C to shut down.
dbug: Microsoft.AspNetCore.Server.Kestrel[1]
      Connection id "0HL8FTULJKHK0" started.
dbug: Microsoft.AspNetCore.Server.Kestrel[1]
      Connection id "0HL8FTULJKHK1" started.
dbug: Microsoft.AspNetCore.Server.Kestrel[6]
      Connection id "0HL8FTULJKHK1" received FIN.
dbug: Microsoft.AspNetCore.Server.Kestrel[10]
      Connection id "0HL8FTULJKHK1" disconnecting.
dbug: Microsoft.AspNetCore.Server.Kestrel[7]
      Connection id "0HL8FTULJKHK1" sending FIN.
dbug: Microsoft.AspNetCore.Server.Kestrel[8]
      Connection id "0HL8FTULJKHK1" sent FIN with status "0".
dbug: Microsoft.AspNetCore.Server.Kestrel[2]
      Connection id "0HL8FTULJKHK1" stopped.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://localhost:5100/api/vacancy/1
dbug: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0]
      ocelot pipeline started : OcelotRequestId - not set
dbug: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0]
      calling next middleware : OcelotRequestId - not set
fail: Ocelot[0]
      Error occured with request Microsoft.AspNetCore.Http.Internal.DefaultHttpRequest
dbug: Ocelot.Request.Middleware.DownstreamRequestInitialiserMiddleware[0]
      started calling request builder middleware : OcelotRequestId - not set
dbug: Ocelot.Request.Middleware.DownstreamRequestInitialiserMiddleware[0]
      calling next middleware : OcelotRequestId - not set
dbug: Ocelot.DownstreamRouteFinder.Middleware.DownstreamRouteFinderMiddleware[0]
      upstream url path is /api/vacancy/1 : OcelotRequestId - not set
dbug: Ocelot.DownstreamRouteFinder.Middleware.DownstreamRouteFinderMiddleware[0]
      downstream template is Ocelot.Values.PathTemplate : OcelotRequestId - not set
dbug: Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware[0]
      EndpointRateLimiting is not enabled for Ocelot.Values.PathTemplate : OcelotRequestId - not set
dbug: Ocelot.Authorisation.Middleware.AuthorisationMiddleware[0]
      /api/v1/vacancy/{vacancyId} route does not require user to be authorised : OcelotRequestId - not set
dbug: Ocelot.DownstreamUrlCreator.Middleware.DownstreamUrlCreatorMiddleware[0]
      downstream url is http://localhost:5002/api/v1/vacancy/{vacancyId} : OcelotRequestId - not set
dbug: Ocelot.Request.Middleware.HttpRequestBuilderMiddleware[0]
      started calling request builder middleware : OcelotRequestId - not set
dbug: Ocelot.Request.Middleware.HttpRequestBuilderMiddleware[0]
      setting upstream request : OcelotRequestId - not set
dbug: Ocelot.Request.Middleware.HttpRequestBuilderMiddleware[0]
      calling next middleware : OcelotRequestId - not set
dbug: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      started calling requester middleware : OcelotRequestId - not set
dbug: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      setting http response message : OcelotRequestId - not set
dbug: Ocelot.Requester.Middleware.HttpRequesterMiddleware[0]
      returning to calling middleware : OcelotRequestId - not set
dbug: Ocelot.Request.Middleware.HttpRequestBuilderMiddleware[0]
      succesfully called next middleware : OcelotRequestId - not set
dbug: Ocelot.Request.Middleware.DownstreamRequestInitialiserMiddleware[0]
      succesfully called next middleware : OcelotRequestId - not set
dbug: Ocelot.Responder.Middleware.ResponderMiddleware[0]
      no pipeline errors, setting and returning completed response : OcelotRequestId - not set
dbug: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0]
      succesfully called middleware : OcelotRequestId - not set
dbug: Ocelot.Errors.Middleware.ExceptionHandlerMiddleware[0]
      ocelot pipeline finished : OcelotRequestId - not set
dbug: Microsoft.AspNetCore.Server.Kestrel[9]
      Connection id "0HL8FTULJKHK0" completed keep alive response.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 340.5324ms 404

Resource server logs:

[18:00:39 INF] Request starting HTTP/1.1 GET http://localhost:5002/api/v1/vacancy/%7BvacancyId%7D  0
[18:00:39 INF] Request finished in 16.6574ms 404

And as I have debugged your code - I see that solution must be somewhere there:

[msvaillant]

Am I right? Is it bug or some uncorrect configuration from me? If it is bug - i can fix it by adding some additional logic to replacer.

this issue happens just sometimes

[msvaillant]

I have investigated a little with your acceptance test and have found the difference in the results of executing FindDownstreamRoute of DownsteamRouteFinder. On acceptance tests it returns templateVariableNameAndValues for {someid}, but for my case result is just empty collection

[TomPallister]

@msvaillant looks like it might be a bug..ill have a look ASAP!

[msvaillant]

@TomPallister , I have encountered bug - if you use Ocelot with request of http://localhost:5100/api/vacancy/1 - bug reproduces in templateVariableNameAndValues, if http://localhost:5100/vacancy/1 - everything works right!

[msvaillant]

@TomPallister - so yes, it is so - on acceptance test there is also such behavior - the placeholder is not recognized when passing request with API before main part. (But strangely - test is OK and you will not see fault, but in debugger I see no placeholder data - actually failing in production).

As I see - this not really bug, but undocumented behavior. I think so because:

1. No api prefix in configuration.json
2. Existance of working results: when API exists but no placeholders are in request - everything fine.
3. It is possible to avoid such a problem if not to use API prefix in Ocelot Gateway, or add API prefix in configuration.json ( I suppose it will help).

[msvaillant]

For myself - I will now use requests to Ocelot with no API prefix, or will configure ocelot explicitly to use this one (hope it will help if it would be needed).

To fix this behavior or document it? I don't know, but I can be not the only who received this problem. It should be blocked to use API implicitly, or fixed to use it implicitly with no care or at least documented to avoid implicitness, because it fails in some cases.

[msvaillant]

Would be nice to look if there are any cases close to mine to avoid bugs, and write acceptance test for my case :)

[TomPallister]

@msvaillant I'm trying to write a test to reproduce your issue but I am failing. This request is being forwarded to the downstream service with a path of "/api/v1/vacancy/1" see test below. What am I missing? :(

   [Fact]
        public void bug()
        {
            var configuration = new FileConfiguration
            {
                ReRoutes = new List<FileReRoute>
                {
                    new FileReRoute
                    {
                        DownstreamPathTemplate = "/api/v1/vacancy",
                        DownstreamScheme = "http",
                        DownstreamHost = "localhost",
                        DownstreamPort = 51879,
                        UpstreamPathTemplate = "/vacancy/",
                        UpstreamHttpMethod = new List<string> { "Options",  "Put", "Get", "Post", "Delete" },
                        ServiceName = "botCore",
                        LoadBalancer = "LeastConnection"
                    },
                    new FileReRoute
                    {
                        DownstreamPathTemplate = "/api/v1/vacancy/{vacancyId}",
                        DownstreamScheme = "http",
                        DownstreamHost = "localhost",
                        DownstreamPort = 51879,
                        UpstreamPathTemplate = "/vacancy/{vacancyId}",
                        UpstreamHttpMethod = new List<string> { "Options",  "Put", "Get", "Post", "Delete" },
                        ServiceName = "botCore",
                        LoadBalancer = "LeastConnection"
                    }
                }
            };

            this.Given(x => x.GivenThereIsAServiceRunningOn("http://localhost:51879/api/v1/vacancy/1", 200, "Hello from Laura"))
                .And(x => _steps.GivenThereIsAConfiguration(configuration))
                .And(x => _steps.GivenOcelotIsRunning())
                .When(x => _steps.WhenIGetUrlOnTheApiGateway("/vacancy/1"))
                .Then(x => _steps.ThenTheStatusCodeShouldBe(HttpStatusCode.OK))
                .And(x => _steps.ThenTheResponseBodyShouldBe("Hello from Laura"))
                .BDDfy();
        }

The other thing is..

@TomPallister , I have encountered bug - if you use Ocelot with request of http://localhost:5100/api/vacancy/1 - bug reproduces in templateVariableNameAndValues, if http://localhost:5100/vacancy/1 - everything works right!

If you dont have /api/vacancy/{vacancyId} in your config then Ocelot should not route your request to the downstream service at all...it should 404 unless I introduced a defect.....should have a test for this :(

[msvaillant]

@TomPallister .When(x => _steps.WhenIGetUrlOnTheApiGateway("api/vacancy/1"))

And on debugging that test you will get empty collection (but strangely this test was not failing - maybe something do not depends on url data)

[TomPallister]

OK I think I understand.

That should return a 404 because you do not have the following set up..

 {
      "DownstreamPathTemplate": "/api/v1/vacancy/{vacancyId}",
      "DownstreamScheme": "http",
      "DownstreamPort": 5002,
      "DownstreamHost": "localhost",
      **"UpstreamPathTemplate": "api/vacancy/{vacancyId}",**
      "UpstreamHttpMethod": [ "Options",  "Put", "Get", "Post", "Delete" ],
      "ServiceName": "botCore",
      "LoadBalancer": "LeastConnection"
    },

However it might have got broken...

[msvaillant]

Yes, it is so :) so now you have one more acceptance test, to check on 404 when somebody tries to use request with API, but without expected configuration.

[TomPallister]

@msvaillant yep...this is definitely a bug, guess I missed a test.