Email HTML - request URL path segment contains illegal characters

[blissjoe]

Describe the bug Email providers handle the password reset HTML in unpredictable ways. Gmail converts the spaces to c return (%0d) while another provider in Evolution removes them.

To Reproduce Steps to reproduce the behavior:

1. Go to 'Forgotten Password'
2. Enter Username
3. Click 'Search'
4. Click 'Email'
5. Open email account and 'click here' to continue

Expected behavior I expect the Token URL to be be valid and allow the user to reset their password.

Additional context I've excluded most of the token but it gets broken up with a space at, what appears to be, every 59 characters.

• Gmail - Firefox browser

https://removed.url/pwm/public/forgottenpassword/H4sIAAAAAAAAAAEqAtX9UFdNLkdDTTEQcNGQP7Gyw-D-vUqcYYgeGXmG1sw%0DvH1jXrYakoW4K_mhCBuFEZEklqbAP7a7C4Q-BabOyQSZucVX2p2CL-5g_Ht......

H4sIAAAAAAAAAAEqAtX9UFdNLkdDTTEQcNGQP7Gyw-D-vUqcYYgeGXmG1sw vH1jXrYakoW4K_mhCBuFEZEklqbAP7a7C4Q-BabOyQSZucVX2p2CL-5g_Ht 

The first space is between sw and vH

This link causes the error. I do see this note after the security violation which removes the %0d but it appears to happen out of order - util.Validator, stripped potentially harmful chars from value

2023-03-06T08:20:41Z, ERROR, http.PwmResponse, {hufya} 5063 ERROR_SECURITY_VIOLATION (request URL path segment contains illegal characters) [removed]
2023-03-06T08:20:41Z, WARN , http.PwmHttpResponseWrapper, attempt to write cookie 'ID' after response is committed
2023-03-06T08:20:41Z, WARN , http.PwmHttpResponseWrapper, attempt to write cookie 'SESSION' after response is committed
2023-03-06T08:40:03Z, WARN , util.Validator, stripped potentially harmful chars from value: input=https://removed.url/pwm/public/forgottenpassword/H4sIAAAAAAAAAAEqAtX9UFdNLkdDTTEQcNGQP7Gyw-D-vUqcYYgeGXmG1sw%0DvH1jXrYakoW4K_mhCBuFEZEklqbAP7a7C4Q-BabOyQSZucVX2p2CL-5g_Ht...... strippedOutput=https://removed.url/pwm/public/forgottenpassword/H4sIAAAAAAAAAAEqAtX9UFdNLkdDTTEQcNGQP7Gyw-D-vUqcYYgeGXmG1swvH1jXrYakoW4K_mhCBuFEZEklqbAP7a7C4Q-BabOyQSZucVX2p2CL-5g_Ht......

• Namecheap email - Evolution Client

https://removed.url/pwm/public/forgottenpassword/H4sIAAAAAAAAAAEeAuH9UFdNLkdDTTEQcNGQP7Gyw-D-vUqcYYgVAtHGtMJosD6NsgwaMDIXNzEBjQwaUJ-ViJquvrQjALf68BJ1D26l4wvAIlGjojKD3P.....

H4sIAAAAAAAAAAEeAuH9UFdNLkdDTTEQcNGQP7Gyw-D-vUqcYYgVAtHGtMJ osD6NsgwaMDIXNzEBjQwaUJ-ViJquvrQjALf68BJ1D26l4wvAIlGjojKD3P 

The first space on this one is between MJ and os

When clicking on this link it works as expected.

---

It's similar to this other issue - https://github.com/pwm-project/pwm/issues/694

I'm not sure I understand the purpose of the util.Validator or if it's possible to remove the c return between each section of the token.

[jrivard]

I don't see these issues with the default templates/macros in v2.0.6 in Gmail. There shouldn't be any spaces in the URL. Un-escaped spaces in URLs are not valid. Perhaps your SMTP server or proxy is modifying the URLs? Can you verify this issue with the default email templates?

[blissjoe]

Thanks Jason,

I upgraded to 2.0.6 last week. I just did some more testing and tried to eliminate some variables. I sent one password reset to my Gmail account using the same PWM instance, email relay, etc. Everything is the same except the client accessing the emails.

Here is the new example when I right click and copy the link using the Firefox on Linux. I cut off part of the code for security.

I am still seeing the character returns in the URL - %0D

https://removed/pwm/public/forgottenpassword/H4sIAAAAAAAAAAEqAtX9UFdNLkdDTTEQISAxDsuuCOD13HHskFRNQXLOlEh%0D_nbdVmMeH2ZfDswMiiXj6hOASdqdkTRBE5Q4iy_8EiZkyU64o9G2Zrvs_7T%0DbdryGHNSzx66zPZuDkLwjqyx5_g4slsSJPwfPD1XOVeB8bDXPJ754vyvSYt%0DBC0eiIdJXMFvZ3R5nETIPWYXxwDj4FXWFmEMI-kaQfG4kBZhtg0-cw9TwG8%0D34xRYrn3-eTpGNIoNOGbRgSrX3SgmPzElxXFK84WPSzA-_WaLhwp0m85rUb%0Ds1ejr3V9-vByk4oHin3Fm9Q-M2BnMNPihYoQlrQkXrxMx0oQcXCbD4TJCm1%0Dr1ESekZ7GqcacOhmwwNmsZWhA-0yI6xS14WK51JFxA1exK2ktv5LgBkM8dq%0DIzok7hWo4Jaw9lx2UQY4Whsyj3bG-U-bTJ77UvZSeeqkTdBpazPrDF8u1vc%0D5CkrEWeJFKB9MVpRxyozeoxK_00euxtUKfeItBvX645dzWgxVj_r53EiD8E%0DO......

I do not see the same issue when I pull up the email using the Gmail app on my phone or accessing it with Evolution.

However, I see the character returns when accessing the email with Chromium on Linux and Firefox on Windows. If I pull up the email in Firefox, then choose Show Original the text for the link doesn't include character returns in the HTML.

In my case, it appears the web browsers handle the URL differently than other email clients. I will continue to do more testing and see if I can give you any more information.

Edit - I forgot to mention that we're using the default email template outside of changing the Subject. Before testing I clicked Return this setting to default value and saved the config.

[nate1032]

I have the same issue. Currently running v2.0.6. The following error only occurs for some users when the click the link from the Forgotten Password email. Copy and pasting the key still works...

PWM 5063 - A security violation has occurred. Please try again later. { 5063 ERROR_SECURITY_VIOLATION (request URL path segment contains illegal characters) }

[gskornowicz]

This affects me too. v2.0.6 after upgrade from v2.0.0. No other changes than upgrade. 2023-09-15T10:18:08Z, ERROR, filter.RequestInitializationFilter, {a2xln} 5063 ERROR_SECURITY_VIOLATION (request URL path segment contains illegal characters) [82.145.216.228]

[steffen-moser]

We see the same problem in version 2.0.6 here.

[jrivard]

Logs at trace level showing the request URL would be helpful.

[steffen-moser]

I don't have traces, yet, but setting

        <setting key="pwm.appProperty.overrides" modifyTime="2023-10-24T00:19:49Z" syntax="STRING_ARRAY" syntaxVersion="0">
            <label>Settings ⇨ Application ⇨ Application ⇨ App Property Overrides</label>
            <value>security.http.permittedUrlPathCharacters=^[a-zA-Z0-9-_=]*$</value>
        </setting>

seems to solve it for us. Essentially, I added the equals sign ("=") to the URL-allowed characters. The default declares the regex "^[a-zA-Z0-9-_]*" as allowed. I'll do further testing...

[steffen-moser]

Further testing revealed that adding the equals symbol to the regex seems to solve it for clicking the hyperlink in Mozilla Thunderbird.

For Google Mail's webmail system, the problem persists. Why? Google Mail seems to substitute carriage return symbols in the URL by "%0D". At first, I thought that adding the "%" symbol to the regex list of allowed symbols might help, but it does not. Most probably, somewhere between Browser, Apache, Tomcat, or PWM, the "%0D" is unsubstituted to carriage return.

However, setting the value to

            <value>security.http.permittedUrlPathCharacters=^[a-zA-Z0-9-_=\r\n]*$</value>

seems to help. "\r" and "\n" should allow the line breaks.

[gskornowicz]

@steffen-moser it worked for us. Browser: Opera Client: Gmail Thank you!