Swagger UI not working as expected while service behind Nginx reverse-proxy

[deepforest]

Trying to run a simple asp.net core 2.1 service behind Nginx reverse-proxy, the swagger UI fail to find the .json file.

When configuring Nginx to forward all calls to a service, where not using the default route, for example: localhost:8080/simpleapp/ --> internal-address:80/, the service itself works as expected, swagger JSON file provided back, but when trying to access the swagger UI, it tries to locate the .json file under localhost:8080/ instead of localhost:8080/simpleapp/ which causes an error to be displayed. The following project demonstrates this issue: https://github.com/deepforest/simpleapp

[RicoSuter]

Required fixes:

• Fix HTTP Location header redirect as it redirects to the internal url (RedirectMiddleware)
• Fix swagger.json path parameter in index.html as it points to the internal path (SwaggerUiIndexMiddleware)

Maybe we can just solve this with the MiddlewareBasePath and use it only for redirect and the param but not for route matching

[OculiViridi]

@deepforest, @RSuter I temporary solved the problem by using this settings.

Application Program class (Note the UseUrls method.)

myWebHost = new WebHostBuilder()
    .UseKestrel()
    .UseUrls("http://*:5001")
    .UseStartup<Startup>()
    .UseSerilog()
    .Build();

Application Startup class

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UsePathBase(new PathString("/my_custom_path"));

    if (env.IsDevelopment())
    {
        // ...
    }

    // ...
}

Swagger configuration (Note the ../ prefix on Add method.)

app.UseSwaggerWithApiExplorer(config =>
{
    config.GeneratorSettings.OperationProcessors.TryGet<ApiVersionProcessor>().IncludedVersions = new[] { "1.0" };
    config.SwaggerRoute = "/v1.0.json";
});

app.UseSwaggerUi3(config =>
{
    config.SwaggerRoutes.Add(new SwaggerUi3Route("v1.0", "../v1.0.json"));
});

@RSuter Anyway, I still have a problem with the UI that I don't actually know if it would be solved by your proposed fixes. Please look at the attached screenshot.

The strange thing here is that you actually take the final hostname (devtest.e...i.com/control/swagger in the screenshot) while it is not configured on my application in any way. The key point seems to be how do you resolve the hostname?

Maybe you should use the Request Header information when generating the URL of UI requests (checking eventually also X-Forwarded-Host HTTP header). In this way the UI commands are fine to be sent from a client-side point of view. I'm assuming that UI API calls (by clicking Try --> Execute) are fired from the client browser.

This is the extract of the previous screenshot swagger JSON file.

[deepforest]

And what about: app.UsePathBase(new PathString("/my_custom_path")); Is it related to your solution -- is this the "virtual route" configured in nginx?

[OculiViridi]

@deepforest There're 2 cases:

CASE 1: Reverse proxy without URL rewrite (my actual case)

Final URL from outside is: http://dev.mydomain.com/my_custom_path/swagger

and the reverse proxy forward requests to: http://myservername:5001/my_custom_path/swagger

because there's no URL rewrite settings on proxy.

So, using app.UsePathBase(new PathString("/my_custom_path")); is needed to allow Kestrel to intercept the full path. Not using the URL rewrite also gives the possibility to the application to know the original request full path coming from the client (outside). The original host is also available at application level from the X-Forwarded-Host HTTP header. Application has all information needed to eventually generate valid URLs for client-side point of view.

CASE 2: Reverse proxy with URL rewrite

Final URL from outside is: http://dev.mydomain.com/my_custom_path/swagger

and the reverse proxy forward requests to: http://myservername:5001/swagger

because using URL rewrite settings on proxy.

In this case, app.UsePathBase(new PathString("/my_custom_path")); is not needed. However, the application now has no chance to retrieve the original full path that is valid from client-side point of view. Only the proxy can implement a Response rewrite rule acting on HTTP Response links. This is good for UI (HTML tags...) but when application outputs JSON data, the proxy is not manipulating it. For this reason I don't see how Swagger could work properly with URL rewrite... (for JSONs) In any case it is still possible to send to the client the correct hostname (that is on the Request header), as in the case 1.

In conclusion I think that the solution would be:

• Do not use URL rewrite on reverse proxy
• Use app.UsePathBase(new PathString("/my_custom_path"));
• Resolve the original client hostname using HTTP headers correctly (X-Forwarded-Host and not only Host)

[deepforest]

Good capture, though knowing your "custom path" as part of your code, is a cons. This is the advantage of URL rewrite, you give devops the opportunity to manage the route how they sees fits in the organization.

[RicoSuter]

The host, schemes, etc. are set here based on the ASP.NET Core values: https://github.com/RSuter/NSwag/blob/master/src/NSwag.AspNetCore/Middlewares/SwaggerMiddleware.cs#L87-L89

… and can be changed with UseSwagger's PostProcess. (v12)

[OculiViridi]

Good capture, though knowing your "custom path" as part of your code, is a cons. This is the advantage of URL rewrite, you give devops the opportunity to manage the route how they sees fits in the organization.

@deepforest I can agree with you. My post is just about my experience with that problems till now. I'm using IIS as reverse proxy, just because in my case we're doing a first experiment with .NET Core, Kestrel and so on... and I've no time at the moment to try/learn different solutions (like nginx for example, that I'd like to use instead). Suppose you're creating a RESTful API where listing methods manage paging by returning not only the requested bunch of objects, but also prev/next page link (as best practices suggest). They have to be rewritten accordingly to domain name, etc. In IIS I didn't find a way to tell the URL rewrite rule to manipulate JSON data in this way...

[OculiViridi]

The host, schemes, etc. are set here based on the ASP.NET Core values: https://github.com/RSuter/NSwag/blob/master/src/NSwag.AspNetCore/Middlewares/SwaggerMiddleware.cs#L87-L89

… and can be changed with UseSwagger's PostProcess. (v12)

@RSuter Sorry! I didn't notice the (v12) line... 😛 Can you please post an example of what we're supposed to do in the PostProcess?

[RicoSuter]

I've added a PR with a new TransformToExternalPath setting + an Nginx sample based on @deepforest 's sample app: https://github.com/RSuter/NSwag/pull/1728

[RicoSuter]

Please review and check whether this solves your scenario

[RicoSuter]

You can clone the repo, switch to the PR branch and test yourself with the solution in

NSwag\samples\WithMiddleware\Sample.AspNetCore21.Nginx

instructions in Startup.cs

[ptr1120]

Hello,

I am also having issues with Nginx reverse proxy and NSwag. I set in nginx:

proxy_set_header X-Forwarded-Server   $host;
proxy_set_header X-Forwarded-Path     $request_uri;

Nginx rewrites the path from /api/myservice/openapi (external) to /api/openapi (internal). I applied the instruction from the Sample.AspNetCore21.Nginx, after that I can access swagger document via /api/myservice/openapi/v1/openapi.json. But in SwaggerUi (/api/myservice/openapi) I get the following error :

Parser error on line 13
end of the stream or a document separator is expected

probably because of a wrong swaggerDocument route for the UI. This is the same for Redoc /api/myservice/redoc/ becomes to /api/myservice/redoc//index.html?url=/api/myservice/redoc/.

My configuration

 app.UseSwagger(options =>
                {
                    options.DocumentName = "v1";
                    options.Path = "/openapi/{documentName}/openapi.json";
                    options.PostProcess = (document, request) =>
                    {
                        if (!new[] { "X-Forwarded-Host", "X-Forwarded-Path" }.All(k => request.Headers.ContainsKey(k)))
                        {
                            return;
                        }
                        document.Host = request.Headers["X-Forwarded-Host"].First();
                        document.BasePath = request.Headers["X-Forwarded-Path"].First();
                    };
                });

                app.UseSwaggerUi3(options =>
                {
                    options.Path = "/openapi";
                    options.DocumentPath = "/openapi/{documentName}/openapi.json";
                    // The header X-Forwarded-Path is set in the reverse proxy
                    options.TransformToExternalPath = (internalUiRoute, request) => request.Headers.ContainsKey("X-Forwarded-Path") ? request.Headers["X-Forwarded-Path"].First() : string.Empty;
                });

                // add api doc
                app.UseReDoc(options =>
                {
                    options.Path = "/redoc";
                    options.DocumentPath = "/openapi/v1/openapi.json";
                    // The header X-Forwarded-Path is set in the reverse proxy
                    options.TransformToExternalPath = (internalUiRoute, request) => request.Headers.ContainsKey("X-Forwarded-Path") ? request.Headers["X-Forwarded-Path"].First() : string.Empty;
                });

What can I do to get these UI's working?

Thank you and best regards, Peter

[deepforest]

Please review and check whether this solves your scenario

@RSuter looks like this fixes the issue, yet requires messing up with nginx config, adding X-External-Host and X-External-Path headers. Not so clean, but a legit workaround. One thing still unclear. Setting 'proxy_set_header X-External-Host localhost:8080;' in nginx.conf, this works if you put localhost:8080 in the web-browser, but apparently, not running locally, this must not be localhost. So do we need to replace 'localhost' with host address per deployed environment?

[OculiViridi]

@deepforest By chance have you tried also with other reverse proxies, different from nginx? Why are the X-External-Host and X-External-Path headers used instead of X-Forwarded-Host, X-Forwarded-For and X-Forwarded-Proto? I'm using IIS as reverse proxy.

[RicoSuter]

Why are the X-External-Host and X-External-Path headers used instead of X-Forwarded-Host, X-Forwarded-For and X-Forwarded-Proto?

This is just an example of how to use TransformToExternalRoute

[deepforest]

@OculiViridi havn't tried that yet, sry.

[OculiViridi]

@RSuter So, do I just need to replace the X-External-Host and X-External-Path headers with the X-Forwarded ones to match my case?

[deepforest]

@OculiViridi, based on MDN, X-Forwarded-Host is a de-facto standard XFH header, identifying the original host, useful to determine which Host was originally used. So based on @RSuter nginx conf file, you can replace X-External-Host with X-Forwarded-Host. As for the X-External-Path, if you consider the X-Forwarded-Host to also contain the path: '/externalpath', then you can also omit X-External-Path, but then, it will be a bit "harder" to extract the path in the Startup.cs file, unless @RSuter will update the TransformToExternalPath config to accept only the Host, and do the magic inside.

[deepforest]

@OculiViridi, @RSuter, eventually, I've left using only standard headers in my nginx.conf file:

proxy_set_header   X-Forwarded-Host localhost:8080/externalpath;
proxy_set_header   X-Forwarded-Proto $scheme;
proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;

(last one is optional)

With the following changes to the Startup code:

public class Startup
{
    public Startup(IConfiguration configuration)
    {
        Configuration = configuration;
    }

    public IConfiguration Configuration { get; }

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
        services.AddSwaggerDocument();
    }

    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.UseMvc();
        app.UseAuthentication();

        // There are two ways to run this app: 
        // 1. Run docker-compose and access http://localhost:8080/externalpath/swagger
        // 2. Run Sample.AspNetCore21.Nginx and access http://localhost:59185/swagger/
        // both URLs should be correctly served...

        // Config with support for multiple documents
        app.UseSwagger(config => config.PostProcess = (document, request) =>
        {
            // Change document server settings to public
            document.Host = ExtractHost(request);
            document.BasePath = ExtractPath(request);
        });

        app.UseSwaggerUi3(config =>
            config.TransformToExternalPath =
                (route, request) => ExtractPath(request) + route);
    }

    private string ExtractHost(HttpRequest request) =>
        request.Headers.ContainsKey("X-Forwarded-Host") ?
            new Uri($"{ExtractProto(request)}://{request.Headers["X-Forwarded-Host"].First()}").Host :
                request.Host.Host;

    private string ExtractProto(HttpRequest request) =>
        request.Headers["X-Forwarded-Proto"].FirstOrDefault() ?? request.Protocol;

    private string ExtractPath(HttpRequest request) =>
        request.Headers.ContainsKey("X-Forwarded-Host") ?
            new Uri($"{ExtractProto(request)}://{request.Headers["X-Forwarded-Host"].First()}").AbsolutePath :
            string.Empty;
}

[RicoSuter]

I dont want to couple TransformToExternalPath to this specific scenario and the project is also just a sample of how to use this transform setting. Maybe we should update the sample to use the default headers

[deepforest]

I dont want to couple TransformToExternalPath to this specific scenario and the project is also just a sample of how to use this transform setting. Maybe we should update the sample to use the default headers

yea, see my prev comment, we posted same time :)

[deepforest]

@OculiViridi, @RSuter, I've integrated the last version of NSwag into our product source code, which is hosted in K8s. In our case, Nginx is the "cluster's ingress", deployed as a K8s service of type "LoadBalancer" with no extra configuration. The "virtual path" in our case is actually each K8s service dns name. So for example accessing http://my-org.com/serviceA/swagger, internally routed to http://serviceA:8080/swagger service. I've only set the 'document.BasePath' to be the service name in our case, provided to each Pod as Environment Variable, and didn't touch 'document.Host' at all. This works like a charm.

Thank you guys!!

[youngcm2]

I had to add these lines in the PostProcess to have it add the https to the schemes.

document.Schemes.Clear();                   
var httpScheme = ExtractProto(request) == "http" ? SwaggerSchema.Http : SwaggerSchema.Https;
document.Schemes.Add(httpScheme);

[youngcm2]

I also had to change the following.

private string ExtractProto(HttpRequest request) =>
        request.Headers["X-Forwarded-Proto"].FirstOrDefault() ?? request.Scheme;

[OculiViridi]

@deepforest I've opened a new issue about reverse proxy configuration. Would you mind to take a look at it (#1892)? Thanks! 😃

[anandkadu]

When can we expect this issue to be solved. Its very common issue. I am facing similar issue. My v2/api-docs i.e. json are on correct url (example: https://{host}/{variable}/{context_path}/{api_prefix)/v2/api-docs). But my rest apis are hitting on https://{host}/{context_path}/{api_prefix)/{api_url}.

[RicoSuter]

See https://github.com/RicoSuter/NSwag/pull/2196

[yetanotherchris]

I wrote a small extension method to get NSwag working with ProxyKit. I'm not sure if it works with a sidecar/k8 setup but it works with different ports:

// Inside Startup.cs:
public override void Configure(IApplicationBuilder app)
{
    app.UseSwaggerWithReverseProxySupport();
    // etc.
}

public static class ApplicationBuilderExtensions
{
    public static IApplicationBuilder UseSwaggerWithReverseProxySupport(this IApplicationBuilder app)
    {
        app.UseSwagger(config => config.PostProcess = (document, request) =>
        {
            string pathBase = request.Headers["X-Forwarded-PathBase"].FirstOrDefault();
            document.BasePath = pathBase;
            document.Host = request.Headers["X-Forwarded-Host"].FirstOrDefault();
        });

        app.UseSwaggerUi3(settings =>
        {
            settings.TransformToExternalPath = (route, request) =>
            {
                string pathBase = request.Headers["X-Forwarded-PathBase"].FirstOrDefault();

                if (!string.IsNullOrEmpty(pathBase))
                    return $"{pathBase}{route}";

                return route;
            };
        });

        return app;
    }
}

[RicoSuter]

Maybe we should also automatically support this header https://github.com/RicoSuter/NSwag/blob/master/src/NSwag.AspNetCore/HttpRequestExtension.cs#L53

[yetanotherchris]

Just to make the code even more interesting to read, there's this header too:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded

(which doesn't add a path prefix option for some reason, like Prefix and PathBase do)

[nosshar]

Ran into the problem with Nginx reverse proxy: Unable to infer base url. After some research I figured out the request to the path <schema>://<host>:<port>/v2/api-docs don't work as I expect. It serves content partially (1st chunk) then closes the connection. Curl output: curl: (18) transfer closed with outstanding read data remaining. Googling this trouvaille I managed to solve the problem with the following Nginx config:

upstream backend {
  server 127.0.0.1:8090;
}

server {
  listen my-public-internet-host-or-ip-address:8090;
  proxy_pass http://backend;
  proxy_set_header Host my-public-internet-host-or-ip-address:8090; # it's important to explicitly specify this otherwise upstream will try to use backend as a host name
  proxy_buffering off; # the most important option helps to fix the problem easily reproduced with the curl tool mentioned above
}

[Simonl9l]

@RicoSuter per the extension above, if one runs behind a reverse proxy that rewrites what is the best setup...is there now built in behavior - how is that configured ?

As an aside Im using Envoy (that also now is behind as AppMesh on AWS)

Thanks

[jeremyVignelles]

@Simonl9l I didn't understand your question, but commenting on closed issue is not a good way to get help. Please open your own issue and describe with code what you're trying to do and what doesn't work.

[Simonl9l]

@jeremyVignelles - as suggested see here https://github.com/RicoSuter/NSwag/issues/3192

[mathiash98]

NSwag has support for this by default now, BUT you need to configure the Dotnet application to actually use the forwardHeaders like described here: https://nickkell.medium.com/swagger-service-url-behind-reverse-proxy-3a2795229100

https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/proxy-load-balancer?view=aspnetcore-6.0#forwarded-headers-middleware-order

Works perfectly without any other settings on UseOpenApi()