Regression RequestRejectedException: The request was rejected because the URL was not normalized [SPR-16740]

[spring-projects-issues]

Caleb Cushing opened SPR-16740 and commented

it's possible this regression belongs to a more specific component of spring, but I'm not sure where the best spot to report this is at this time

in Brussels-SR6, this code worked, now updating to SR9 and it's broken

@Bean( name = "viewResolver" )
public InternalResourceViewResolver viewResolver() {
    InternalResourceViewResolver viewResolver = new InternalResourceViewResolver();
    viewResolver.setViewClass( JstlView.class );
    viewResolver.setPrefix( "/jsp/" );
    viewResolver.setSuffix( ".jsp" );
    return viewResolver;
}

this configuration was adapted from an older XML configuration a while ago.

Now when trying to visit a jsp? or at least some I get this.

Apr 17, 2018 4:25:56 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [jsp] in context with path [] threw exception
org.springframework.security.web.firewall.RequestRejectedException: The request was rejected because the URL was not normalized.
    at org.springframework.security.web.firewall.StrictHttpFirewall.getFirewalledRequest(StrictHttpFirewall.java:248)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:193)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

i'm guessing this has something to do with one of the recent security vulns. internally when I debug the servelet context is //jsp/.... note: I think we were using /jsp/ instead of /jsp because we didn't want to match /jspFoo.jsp, only the nested path.

Update That doesn't just fix it... we have a failing test with SAML. Forwarded URL expected:</jsp/testSamlResponse.jsp> but was:</jsptestSamlResponse.jsp>

not sure yet if it' simple to fix, at a glance it calls this piece of controller

    private static final String TEST_SAML_RESPONSE_VIEW = "testSamlResponse";

    @RequestMapping( value = "/generateSamlResponse", method = RequestMethod.POST )
    public String doPost( Model model, HttpServletRequest request ) {

        final String serverName = request.getParameter( "redirectHost" );
        final String propertyFileName = request.getParameter( "propertiesFileName" );
        final SamlAssertionProperties properties = samlTestService.loadConfigration( "classpath:/" + propertyFileName );

        // Override properties from file with user selections and generate assertion
        updateProperties( request, properties );
        final String assertion = samlTestService.buildAssertion( properties );
        final String url = StringUtilsDex.isEmpty( serverName ) ? "/saml/SSO" : serverName;
        model.addAttribute( "assertion", assertion );
        model.addAttribute( "url", url );
        return TEST_SAML_RESPONSE_VIEW;
    }

Update 2 This became an issue as of Brussels SR7

 

Update 3 this looks like how we actually need to fix this, worth stating that login in this case is the name of the view. I don't know that removing the /jsp/ mappings are necessary, but neither do they seem to be relevant. Not sure how it ends up being //jsp/..

--- a/dex-ui/src/main/webapp/WEB-INF/web.xml
+++ b/dex-ui/src/main/webapp/WEB-INF/web.xml
@@ -30,41 +30,6 @@
                <url-pattern>*.jsp</url-pattern>
        </servlet-mapping>
-       <servlet-mapping>
-               <servlet-name>jsp</servlet-name>
-               <url-pattern>/jsp/login.jsp</url-pattern>
-       </servlet-mapping>
-
-       <servlet-mapping>
-               <servlet-name>jsp</servlet-name>
-               <url-pattern>/jsp/error.jsp</url-pattern>
-       </servlet-mapping>
-
-       <servlet-mapping>
-               <servlet-name>jsp</servlet-name>
-               <url-pattern>/jsp/version.jsp</url-pattern>
-       </servlet-mapping>
-
-       <servlet-mapping>
-               <servlet-name>jsp</servlet-name>
-               <url-pattern>/accessdenied.jsp</url-pattern>
-       </servlet-mapping>
-
-       <servlet-mapping>
-               <servlet-name>jsp</servlet-name>
-               <url-pattern>/jsp/register.jsp</url-pattern>
-       </servlet-mapping>
-
-       <servlet-mapping>
-               <servlet-name>jsp</servlet-name>
-               <url-pattern>/forgotPassword.jsp</url-pattern>
-       </servlet-mapping>
-
-       <servlet-mapping>
-               <servlet-name>jsp</servlet-name>
-               <url-pattern>/verifyUser.jsp</url-pattern>
-       </servlet-mapping>
-
        <error-page>
                <error-code>400</error-code>
                <location>/public/error/400</location>
@@ -86,7 +51,7 @@
        </error-page>        <welcome-file-list>
-               <welcome-file>/jsp/login.jsp</welcome-file>
+               <welcome-file>login</welcome-file>
        </welcome-file-list> 

---

Affects: 4.3.16

Attachments:

• image-2018-04-20-12-08-57-772.png (122.58 kB)
• image-2018-04-20-13-15-25-857.png (149.34 kB)
• jsp-regression.zip (81.94 kB)

[spring-projects-issues]

Rossen Stoyanchev commented

So the controller returns "testSamlResponse" which should then become a forward to "/jsp/testSamlResponse.jsp" as far as I can see, but you're saying the forward is to "//jsp/..." ? How does that come about? Without any sample code I can't check.

Indeed the Spring Security docs says the following about the firewall:

Some containers normalize these out before performing the servlet mapping,
but others don’t. To protect against issues like these, FilterChainProxy uses
an HttpFirewall strategy to check and wrap the request. Un-normalized
requests are automatically rejected by default, and path parameters and
duplicate slashes are removed for matching purposes.

/cc Rob Winch  

[spring-projects-issues]

Caleb Cushing commented

yes, if it's helpful, we're using Tomcat.

[spring-projects-issues]

Rossen Stoyanchev commented

You could maybe debug in InternalResourceView to see what URL it dispatches to? Need to understand where the "//" comes from..

[spring-projects-issues]

Caleb Cushing commented

it doesn't appear to ever get to InternalResourceView, screenshot shows it's already ``//jsp/login.jsp by the time it hits StandardContextValve.invoke trying to figure out what's happening before that.

 

!image-2018-04-20-12-08-57-772.png!

[spring-projects-issues]

Caleb Cushing commented

Mapper:931 is where the extra / is added.

!image-2018-04-20-13-15-25-857.png!

[spring-projects-issues]

Rossen Stoyanchev commented

Please provide the details of the incoming request, and explain the processing sequence. Does it get to a Spring MVC controller which then forwards to a JSP, or does it fail before it even gets to Spring MVC? And when does the failure occur relative to that? I don't have a sample to run, and I can't guess what actually happens.

[spring-projects-issues]

Rossen Stoyanchev commented

I know this is most likely due to a change in Spring Security, but to provide a more helpful answer, I need the info requested in my last comment.

[spring-projects-issues]

Caleb Cushing commented

[^jsp-regression.zip] isn't a working sample :/ I'm stuck trying to figure out how I can get it to use a web.xml, or replicate a web.xml with a welcome-file-list.

this should probably be removed

@Configuration
static class MvcConfig extends WebMvcConfigurerAdapter {
    @Override
    public void addViewControllers( final ViewControllerRegistry registry ) {
        registry.addViewController( "/" ).setViewName( "welcome" );
    }
} 

tried adding this

@Bean
public EmbeddedServletContainerFactory servletContainer() {

    TomcatEmbeddedServletContainerFactory factory = new TomcatEmbeddedServletContainerFactory();

    TomcatContextCustomizer contextCustomizer = new TomcatContextCustomizer() {
        @Override
        public void customize(Context context) {
            context.addWelcomeFile("/jsp/welcome.jsp");
        }
    };
    factory.addContextCustomizers(contextCustomizer);

    return factory;
} 

but it doesn't seem to recognize the welcomefile, and/or I can't access the jsps like /jsp/welcome.jsp, thought this might fix it, but it didn't

@Configuration
static class MvcConfig extends WebMvcConfigurerAdapter {
    @Override
    public void addResourceHandlers( final ResourceHandlerRegistry registry ) {
        registry.addResourceHandler("/jsp/*.jsp").addResourceLocations("/WEB-INF/jsp/");
    }
} 

this is the web.xml I'm trying to replicate, I think if I could get this recognized or an identical configuration, I can reproduce it.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">
    <display-name>MyApp</display-name>
    <listener>
        <listener-class>org.springframework.boot.legacy.context.web.SpringBootContextLoaderListener</listener-class>
    </listener>
    <servlet>
        <servlet-name>jsp</servlet-name>
        <servlet-class>org.apache.jasper.servlet.JspServlet</servlet-class>
        <init-param>
            <param-name>logVerbosityLevel</param-name>
            <param-value>WARNING</param-value>
        </init-param>
        <init-param>
            <param-name>compilerSourceVM</param-name>
            <param-value>1.8</param-value>
        </init-param>
        <init-param>
            <param-name>compilerTargetVM</param-name>
            <param-value>1.8</param-value>
        </init-param>
        <load-on-startup>3</load-on-startup>
    </servlet>    <servlet-mapping>
        <servlet-name>jsp</servlet-name>
        <url-pattern>*.jsp</url-pattern>
    </servlet-mapping>    <welcome-file-list>
        <welcome-file>/jsp/welcome.jsp</welcome-file>
    </welcome-file-list></web-app> 

[spring-projects-issues]

Caleb Cushing commented

the demo currently works, like the fixes, but maybe if you can help point out a way to get the welcome-file stuff working... or maybe it's got all the crazy needed and I just don't know how (without spending many more hours) to get it stood up in tomcat. I think it would work at that point. Feedback appreciated.

[spring-projects-issues]

Rossen Stoyanchev commented

Isn't this what you're looking for?

https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-spring-mvc-welcome-page 

I think SpringBootServletInitializer is for when you deploy as a WAR.

At any rate, at this point I'm not sure what this has to do with the original issue, so I'm resolving for now.

 

[spring-projects-issues]

Caleb Cushing commented

it has to do with the original issue because I'm trying to replicate the issue for you... in a demo app (because we have 400k lines of code, and a lot of it is crap). and no that wouldn't replicate the issue correctly, it has to be set as a welcome-file.

[spring-projects-issues]

Rossen Stoyanchev commented

I'm resolving as I suspect strongly the change is related to Spring Security and not to the Spring Framework. If you do come up with some sample, feel free to comment, and I would be happy to take a look to confirm or provide guidance.