dreamstar-enterprises commented 6 days ago

This was closed, but I really do think this is a Spring Boot issue, rather than external library issue.

Re-opening for further consideration:

https://github.com/spring-projects/spring-boot/issues/42305

dreamstar-enterprises commented 6 days ago

I don't undersand why this was closed w/o explanation? It is not really a duplicate as the other issue was closed, unless that was re-opened?

wilkinsona commented 6 days ago

As Brian already explained, we want to allow the Springdoc team some time to evaluate https://github.com/springdoc/springdoc-openapi/issues/2708. Please respect the team's wishes. Opening duplicate issues across multiple projects just wastes the time of the OSS community.

dreamstar-enterprises commented 6 days ago

Yes, no problem I will wait, as requested. I do still think this is a Spring Boot framework thing, since these settings are for Spring Boot (not Spring Docs) - so whichever library tries to generate an internal redirect URL, the code to add the Gateway prefix should NOT be in their library code. It should be managed by Spring Boot.

Resource Server Settings:

# default server settings
server:
  address: ${LOCALHOST}
  port: ${RESOURCE_SERVER_PORT}
  ssl:
    enabled: false
  forward-headers-strategy: native (here framework doesn't work for me - I keep getting 403 Forbidden)

Headers in Spring BFF when forwarding request: I can see my BFF forwarding the right headers in the request to the Resource Server:

Forwarded: proto=http;host="localhost:7080";for="127.0.0.1:51801"
X-Forwarded-For: 127.0.0.1
X-Forwarded-Proto: http
X-Forwarded-Prefix: /bff
X-Forwarded-Port: 7080
X-Forwarded-Host: localhost:7080
host: localhost:9090
content-length: 0
Authorization: Bearer eyJhb...

wilkinsona commented 5 days ago

If you believe it's a Spring Boot problem, then you should provide a complete, yet minimal sample that demonstrates that's the case. If indeed it is a Spring Boot problem, such a sample should not depend on Spring Docs, or any other third-party code. Instead it should only depend on Spring Boot and contain the minimal amount of application code that's necessary to reproduce the problem.

dreamstar-enterprises commented 5 days ago

Hi Wilinsona,

Thanks for offering to consider this further..

My Resource Server YAML settings are below. It has one protected endpoint called /secret-message It has a security chain too. Apart from that, there isn't much to it (I can put it on github on request, if needed)

Resource Server YAML Config

#**********************************************************************************************************************#
#***************************************************** VARIABLES ******************************************************#
#**********************************************************************************************************************#

dse-servers:
  # server settings
  scheme: ${SCHEME}
  hostname: ${HOST}

  # reverse proxy server
  reverse-proxy-host: ${REVERSE_PROXY_HOST}
  reverse-proxy-port: ${REVERSE_PROXY_PORT}

  # bff server
  bff-server-prefix: ${BFF_SERVER_PREFIX}

  # resource server
  resource-server-port: ${RESOURCE_SERVER_PORT}
  resource-server-prefix: ${RESOURCE_SERVER_PREFIX}

  # auth-0 authorization server
  auth0-auth-registration-id: ${AUTH0_SERVER_REG_ID}
  auth0-issuer-uri: ${AUTH0_SERVER_ISSUER_URI}
  auth0-dreamstar-frontiers-url: ${AUTH0_DREAMSTAR_FRONTIERS_URL}

#**********************************************************************************************************************#
#************************************************** SPRING SETTINGS ***************************************************#
#**********************************************************************************************************************#

# default spring settings
spring:
  # application settings
  application:
    name: Timesheets-RESTApiApplication
  # profile settings
  profiles:
    active: dev
  # lifecycle settings
  lifecycle:
    timeout-per-shutdown-phase: ${TIMEOUT_SHUTDOWN}
  # main settings
  main:
    allow-bean-definition-overriding: true
  # webflux settings
  webflux:
    base-path: ${RESOURCE_SERVER_PREFIX}
  # postgreSQL configurations
  r2dbc:
    url: ${R2DBC_POSTGRES_URL}
    username: ${R2DBC_POSTGRES_USERNAME}
    password: ${R2DBC_POSTGRES_PASSWORD}
    pool:
      max-size: 10
      max-acquire-time: 10s
  # flyway configurations
  flyway:
    url: ${JDBC_POSTGRES_URL}
    user: ${R2DBC_POSTGRES_USERNAME}
    password: ${R2DBC_POSTGRES_PASSWORD}
    schemas: public
    table: flyway_schema_history
    locations: classpath:db/migration
    clean-on-validation-error: false
    clean-disabled: false
    baseline-on-migrate: true
    out-of-order: true
    enabled: true
    baseline-description: "init"
    baseline-version: 1
  # security configurations
  security:
    oauth2:
      client:
        registration:
          # oauth2.0 client registrations - (for auth0 auth server)
          auth0:
            client-id: ${AUTH0_SERVER_CLIENT_ID}
            client-secret: ${AUTH0_SERVER_CLIENT_SECRET}
          # oauth2.0 client registrations - (for in-house auth server)

#**********************************************************************************************************************#
#************************************************** SERVER SETTINGS ***************************************************#
#**********************************************************************************************************************#

# default server settings
server:
  address: ${LOCALHOST}
  port: ${RESOURCE_SERVER_PORT}
  ssl:
    enabled: false
  forward-headers-strategy: native

#**********************************************************************************************************************#
#************************************************ SPRING DOC SETTINGS *************************************************#
#**********************************************************************************************************************#

# spring doc settings
springdoc:
  api-docs:
    enabled: true
    version: openapi_3_1
    path: /v3/api-docs
  swagger-ui:
    enabled: true
    path: /v1/swagger-ui.html
    url: /v3/api-docs
    operations-sorter: method
  show-actuator: false
  enable-kotlin: true
  enable-spring-security: true
  enable-default-api-docs: true
  default-produces-media-type: application/json

#**********************************************************************************************************************#
#************************************************** END OF YAML *******************************************************#
#**********************************************************************************************************************#

Env Variables My local.env file settings are:

SCHEME=http HOST=localhost LOCALHOST=127.0.0.1

TIMEOUT_SHUTDOWN=30s TIMEOUT_SESSION=2100

REVERSE_PROXY_HOST=localhost REVERSE_PROXY_PORT=7080

ANGULAR_SERVER_HOST=localhost ANGULAR_SERVER_PORT=4200 ANGULAR_SERVER_PREFIX=/angular-ui

BFF_SERVER_HOST=localhost BFF_SERVER_PORT=9090 BFF_SERVER_PREFIX=/bff

RESOURCE_SERVER_PORT=8080 RESOURCE_SERVER_PREFIX=/api/v1/resource

Resource Server Security Chain

@Configuration
@EnableWebFluxSecurity
@EnableReactiveMethodSecurity(useAuthorizationManager = true)
internal class ResourceSecurityConfig {

    @Bean
    /* security filter chain for authentication & authorization (reactive) */
    /* this should be webSession stateless */
    fun resourceServerSecurityFilterChain(
        http: ServerHttpSecurity,

        authenticationEntryPoint: AuthenticationEntryPoint,
        accessDeniedHandler: AccessDeniedHandler,

        authenticationManagerResolver: ReactiveAuthenticationManagerResolver<ServerWebExchange>,

        ipWhiteListFilter: IPWhiteListFilter,
        validEndPointFilter: ValidEndPointFilter,

        reactiveRequestCache: ReactiveRequestCache,
        statelessSecurityContextRepository: StatelessSecurityContextRepository,

        ): SecurityWebFilterChain {

            /* enable csrf */
            http.csrf { csrf ->
                csrf.disable()
            }

            /* configure request cache */
            http.requestCache { cache ->
                cache.requestCache(reactiveRequestCache)
            }

            /* configure security context */
            http.securityContextRepository(statelessSecurityContextRepository)

            /* configure session management */
            // there is no explicit createSessionPolicy.NEVER for Spring Webflux like there is for Spring Servlet

            /* oauth2.0 resource server */
            http.oauth2ResourceServer { oauth2 ->
                oauth2.authenticationManagerResolver(authenticationManagerResolver)
                oauth2.authenticationEntryPoint(authenticationEntryPoint)
                oauth2.accessDeniedHandler(accessDeniedHandler)
            }

            /* configure authorization  */
            http.authorizeExchange { authorize ->
                authorize
                    .pathMatchers("/hello").permitAll()
                    .pathMatchers(
                        "/v3/api-docs",
                        "/v1/swagger-ui.html",
                        "/v1/webjars/swagger-ui/**").permitAll()
                    .anyExchange().authenticated()
            }

            /* other filters */
            // apply ip-whitelist filter before http basic authentication
            http.addFilterBefore(ipWhiteListFilter, SecurityWebFiltersOrder.HTTP_BASIC)
            // apply valid end-point filter before http basic authentication
            http.addFilterBefore(validEndPointFilter, SecurityWebFiltersOrder.HTTP_BASIC)

            /* exception handling */
            // handlers for any exceptions not handled elsewhere
            http.exceptionHandling { exceptionHandling ->
                exceptionHandling.authenticationEntryPoint(authenticationEntryPoint)
                exceptionHandling.accessDeniedHandler(accessDeniedHandler)
            }

        return http.build()
    }

}

Accssing this works:

But I notice that in the response header, here is no /bff prefix, next to the "Location:" field

Hence, in the next browser re-direct I get this:

Spring BFF Config

I did check my Spring BFF (Spring Cloud Gateway) to see if the relevant headers were being forwarded, and I do see this: So, all the relevant headers (including an access token)

Forwarded: proto=http;host="localhost:7080";for="127.0.0.1:51801"
X-Forwarded-For: 127.0.0.1
X-Forwarded-Proto: http
X-Forwarded-Prefix: /bff
X-Forwarded-Port: 7080
X-Forwarded-Host: localhost:7080
host: localhost:9090
content-length: 0
Authorization: Bearer eyJhb...

The full settings to my BFF can be found here:

https://github.com/dreamstar-enterprises/docs/blob/master/Spring%20BFF/BFF/src/main/resources/application.yaml

So, I'm not sure why the Resource Server is not adding the /bff Gateway prefix, even though the request should have X-Forwarded-Prefix: /bff present?

Grateful for any help, on where you think I may have gone wrong, or if this is a bug.

wilkinsona commented 5 days ago

Unfortunately, that's really not what we're looking for as there seem to be a significant number of moving parts that should not be relevant here.

We're looking for something that's complete yet minimal. That means that it should contain only what's absolutely necessary to reproduce the problem and nothing more. What you've shared above does not appear to be minimal – if there's a problem purely with the handing of proxy headers, there should be no need to involve Spring Security. It also is not complete – without doing additional work, there's no way for us to reproduce the problem.

You should start with an empty application generated by https://start.spring.io and add the bare minimum of dependencies and application code to reproduce your problem. You should then share that application with us (zip it up and attach it here or push it to a separate GitHub repository) and provide precise instructions on the steps that are necessary to reproduce it.

dreamstar-enterprises commented 5 days ago

Thanks Andy,

Ok, I will try to get an minimal viable example up and running. I think the issue has to do with 'native' vs 'framework' (spring boot setting), and me using 'native', since with 'framework', I keep getting a 403 Forbidden, error, and so I'm forced to use 'native', and that, for some reason, does not add the Gateway prefix (as shown in the posts above)

Please see, https://stackoverflow.com/questions/78990375/spring-boot-forward-headers-strategy-framework-keeps-giving-403-forbidden

dreamstar-enterprises commented 5 days ago

demo.zip

Hi there,

As requested please see attached minimum reproducable example (it's a zip that is 140kb in size)

When I have:

default server settings

server: address: localhost port: 8080 ssl: enabled: false forward-headers-strategy: native

I get following behaviour (the Location URL still misses the /bff Gateway prefix)

With this, I get:

default server settings

server: address: localhost port: 8080 ssl: enabled: false forward-headers-strategy: framework

I get not 404 Not Found (with my original server, that had Spring Security I got 403 Forbidden instead)

This took me less than 5 minutes to reproduce. Hopefully you can do the same.

Please note my requests are going through a reverse proxy at port 7080, that strips the /bff prefix. It then goes through the Spring BFF, on port 9090, that forwards it to the resource server at port 8080

Hopefully, this is enough to warrant some investigation.

wilkinsona commented 5 days ago

I'm afraid it doesn't. The sample isn't minimal as it still depends on Spring Docs. It's also using Actuator which, as far as we know, isn't related to the problem. If this is a Spring Boot problem, it should be possible to reproduce it without Spring Docs. The sample also includes a controller with a request mapping for /test/hello but none of your screenshots above are using that path. This falls short of being precise instructions for reproducing the problem.

Unfortunately, having wasted quite a bit of time on this already, I can't justify spending any more time on it without concrete evidence that there's a bug in Spring Boot related to proxy header handling.

dreamstar-enterprises commented 5 days ago

Hi Andy,

I appreciate the very stringent requirements.

To meet them:

how does one create a re-direct URL manually? The one I'm using is specifically the one created by Spring Docs, when the /swagger-ui.html is accessed, as that is the root place of where I am seeing the issue
The endpoint /test/hello returns a simple message 'Hello World!' - just to give the router at least one end point - to make sure people can test the server works
The actuator can be simply turned off by commenting the line out.
One simply has to access the /swagger-ui.html endpoint to reproduce the issue

I'm not sure how much more specific I can be. I've tried to be as crystal clear as possible with how I'm seeing the problem, with screen shot backups.

Happy to accommodate further requests.

I think if I can generate a redirect URL manually, I should be able to narrow it down to being a Spring Boot issue, or Spring Docs issue. I'm not sure how to do that though.

Thanks for your patience and understanding.

philwebb commented 5 days ago

The swagger code that does the redirect is here.

When you have forward-headers-strategy: framework, the org.springframework.web.server.adapter.ForwardedHeaderTransformer class is used which supports X-Forwarded-Prefix headers.

When you use forward-headers-strategy: native the reactor.netty.http.server.DefaultHttpForwardedHeaderHandler class is used, which does not support X-Forwarded-Prefix headers.

I've opened https://github.com/reactor/reactor-netty/issues/3432 to see if the reactor-netty team is interested in adding support.

dreamstar-enterprises commented 5 days ago

Thank you very much Phil!

I do wonder why when I used forward-headers-strategy: framework, I kept getting 404 Not Found, ( https://github.com/user-attachments/files/17015495/demo.zip) or 403 Forbidden (and why I had to use forward-headers-strategy: native, that didn't add the X-Forwarded-Prefix).

Is that because forward-headers-strategy: framework is not supported for Webflux, and why you raised the issue here, https://github.com/reactor/reactor-netty/issues/3432 ?

I did think about adding the gateway prefix to the context path here:

  # webflux settings
  webflux:
    base-path: "/bff${CONTEXT_PATH}"

But I'm not sure that would work, as I'd need to send requests to http:localhost:7080/bff/bff/context-path/ (the first /bff gets stripped out by the reverse proxy, before it sends it on to the resource server)

Someone suggested trying to add the gateway prefix to the context-path here https://github.com/reactor/reactor-netty/issues/259#issuecomment-679873342 - but I didn't quite understand it. I'll try to look into it further.

Thanks for the help anyway, again.

spring-projects / spring-boot

Spring Boot (Webflux) - Swagger UI - redirect URI does not include Gateway Prefix #42317

default server settings

default server settings