search

microsoft / reverse-proxy

A toolkit for developing high-performance HTTP reverse proxy applications.
https://microsoft.github.io/reverse-proxy
MIT License
8.52k stars 836 forks source link

PathPattern transform results in 301 from IIS because it strips trailing slash #2531

Open gplindauer opened 4 months ago

gplindauer commented 4 months ago

Describe the bug

If I don't need a PathPattern Transform, YARP passes my request through with a trailing slash and all is well. However, if I insert a PathPattern Transform, even if it is a test transform that shouldn't (if I understand the docs correctly) change anything, YARP strips off the trailing slash, causing IIS to send a 301. Unfortunately, IIS's 301 includes the destination host, and so the browser (chrome v126) starts trying to send packets directly to the destination, which fails.

I googled a lot trying to resolve this, and re-read the docs about config files, routing, and transforms carefully, but never found a solution online. Still, this could quite likely my user error. If so I apologize but maybe the docs could be clarified?

To Reproduce

In the appsettings.json file [this is a release build, not a development build, so not appsettins.development.json]:

  "Urls": "https://outwardfacing.debugsite.com",
  "ReverseProxy": {
    // Routes tell the proxy which requests to forward
    "Routes": {
      "test-route": {
        "ClusterId": "TEST",
        "AuthorizationPolicy": "Default",
        "Match": {
          "Path": "/REQUESTS/{**path}"
        },
     "Transforms": [{ "PathPattern": "/requests/{**path}"        }]
      }
    }, // end of routes

    // Clusters tell the proxy where and how to forward requests
    "Clusters": {
      "TEST": {
        "Destinations": {
          "outpath": {

            "Address": "http://protected.debug.website.com/lab/"
          }
        }
      }
    } // End clusters
  }

Further technical details

ASP.Net Log for above shows:
      Request starting HTTP/2 GET https://outwardfacing.debugsite.com/lab/requests/ - - -
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1001]
      1 candidate(s) found for the request path '/requests/'
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1005]
      Endpoint 'test-route' with route pattern '/REQUESTS/{**path}' is valid for the request path '/requests/'
dbug: Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware[1]
      Request matched endpoint 'test-route'
dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[15]
      Static files was skipped as the request already matched an endpoint.
dbug: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[8]
      AuthenticationScheme: Identity.Application was successfully authenticated.
dbug: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[1]
      Authorization was successful.
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
      Executing endpoint 'test-route'
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[9]
      Proxying to http://protected.debug.website.com/lab/requests HTTP/2 RequestVersionOrLower 
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[56]
      Received HTTP/1.1 response 301.

NOTE THAT IN THE Proxying to .../lab/requests HTTP/2 ... LINE ABOVE, THERE IS NO TRAILING / AFTER requests

If I comment out the "transforms" line above, everything works. [However in the final I need the transform.] Here's the log:

    Request starting HTTP/2 GET https://outwardfacing.debugsite.com/lab/requests/ - - -
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1001]
      1 candidate(s) found for the request path '/requests/'
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1005]
      Endpoint 'test-route' with route pattern '/requests/{**path}' is valid for the request path '/requests/'
dbug: Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware[1]
      Request matched endpoint 'test-route'
dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[15]
      Static files was skipped as the request already matched an endpoint.
dbug: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[8]
      AuthenticationScheme: Identity.Application was successfully authenticated.
dbug: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[1]
      Authorization was successful.
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
      Executing endpoint 'test-route'
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[9]
      Proxying to http://protected.debug.website.com/lab/requests/ HTTP/2 RequestVersionOrLower 
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[56]
      Received HTTP/1.1 response 200.

NOTE THAT IN THE Proxying to .../lab/requests/ HTTP/2 ... LINE ABOVE, THERE is now a TRAILING / AFTER requests

A workaround that works is to insert something like "base-route" below, when I do that, YARP starts including the backslash again:

     "Routes": {
       "base-route" : {
    "ClusteriD" : "TEST",
        "AuthorizationPolicy": "Default",
        "Match": {
          "Path": "/REQUESTS/"
        }
      },
      "test-route": {

Log for this is:

      Request starting HTTP/2 GET https://outwardfacing.debugsite.com/lab/requests/ - - -
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1001]
      2 candidate(s) found for the request path '/requests/'
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1005]
      Endpoint 'base-route' with route pattern '/REQUESTS/' is valid for the request path '/requests/'
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1005]
      Endpoint 'psyche-route' with route pattern '/REQUESTS/{**path}' is valid for the request path '/requests/'
dbug: Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware[1]
      Request matched endpoint 'base-route'
dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[15]
      Static files was skipped as the request already matched an endpoint.
dbug: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[8]
      AuthenticationScheme: Identity.Application was successfully authenticated.
dbug: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[1]
      Authorization was successful.
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
      Executing endpoint 'base-route'
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[9]
      Proxying to http://protected.debug.website.com/lab/requests/ HTTP/2 RequestVersionOrLower 
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[56]
      Received HTTP/1.1 response 200.

Yarp successfully routes the base ../requests/ packets with "base-route" as shown above, and it routes packets to pathes underneath the Requests/ folder using "test-route".

Another worksaround would be to change the "http://protected.debug.website.com/lab/" destination base folder structure to mimic the origination structure, or maybe to manually send a 301 redirection back to the browser for any packets missing the backslash to use the origin https://outwardfacing.debugsite.com host in the 301 (perhaps https://github.com/microsoft/reverse-proxy/discussions/1633).

I did try enabling AutoRedirect in HTTPClient (https://www.reddit.com/r/csharp/comments/r0gu1a/yarp_redirecting_instead_of_proxying/) did not completely work, some packets went through OK but some did not. 

Microsoft Visual Studio Community 2022
Version 17.10.3
VisualStudio.17.Release/17.10.3+35013.160
Microsoft .NET Framework
Version 4.8.09037

Installed Version: Community

ASP.NET and Web Tools   17.10.341.11210
ASP.NET and Web Tools

Azure App Service Tools v3.0.0   17.10.341.11210
Azure App Service Tools v3.0.0

Azure Functions and Web Jobs Tools   17.10.341.11210
Azure Functions and Web Jobs Tools

C# Tools   4.10.0-3.24312.19+771f269b3abcbbd991f05becf8fe5e991d24b0c1
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

NuGet Package Manager   6.10.1
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

Razor (ASP.NET Core)   17.10.3.2427201+4f57d1de251e654812adde201c0265a8ca7ca31d
Provides languages services for ASP.NET Core Razor.

SQL Server Data Tools   17.10.172.0
Microsoft SQL Server Data Tools

TypeScript Tools   17.0.30327.2001
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools   4.10.0-3.24312.19+771f269b3abcbbd991f05becf8fe5e991d24b0c1
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools   17.10.0-beta.24228.1+dd749058c91585e9b5dae62b0f8df892429ee28f
Microsoft Visual F# Tools

Visual Studio IntelliCode   2.2
AI-assisted development for Visual Studio.
benjaminpetit commented 2 months ago

It seems that it's the behavior expected by TemplateBinder, if we look at tests when the route looks like "Test/{val1}" and val1 is empty, then the expected result is Test (without trailing slash).

I can't say I really agree with that...

Maybe we should fix that in yarp instead?

gplindauer commented 2 months ago

I vote for forwarding it to the YARP developers for analysis-- definitely don't want to break something if that is how it has been operating for a while. My ultimately wish would be to have a flexibility to choose that the forwarding destination to be "Test/" if that is what the implementor needs, and a different way if they need "Test". Right now I'm not sure there is any way to get "Test/" and IIS isn't playing well without the trailing "/".

karelz commented 2 months ago

Triage:

Tratcher commented 2 months ago

I agree this is buggy, the config clearly calls for the trailing slash.

I don't think any of those services are using these transforms so the risk should be minimal.

karelz commented 1 month ago

Triage: Per @benjaminpetit it is best to fix it in ASP.NET Core -- class TemplateBinder. @benjaminpetit will move this bug there.