search

virtualmin / virtualmin-gpl

Virtualmin web hosting control panel for Webmin
https://www.virtualmin.com
GNU General Public License v3.0
317 stars 99 forks source link

BUG found 3x `RedirectMatch` `.well-known` lines, causes redirect loop. #443

Open chris001 opened 2 years ago

chris001 commented 2 years ago

Virtual server got into an infinite redirect loop to itself. Found these lines in the host's config for the https host on 443:

RemoveHandler .php
RemoveHandler .php8.1
RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1

When you comment all 3 RedirectMatch lines save and apply, the site works normally, no infinite redirect loop.

jcameron commented 2 years ago

Can you post the entire config block containing these RedirectMatch lines?

chris001 commented 2 years ago 
#SuexecUserGroup "#1000" "#1000"
ServerName virtual.server.tld
ServerAlias www.virtual.server.tld
ServerAlias mail.virtual.server.tld
DocumentRoot /home/ns2/public_html
ErrorLog /var/log/virtualmin/virtual.server.tld_error_log
CustomLog /var/log/virtualmin/virtual.server.tld_access_log combined
ScriptAlias /cgi-bin/ /home/ns2/cgi-bin/
DirectoryIndex index.php index.php4 index.php5 index.htm index.html
<Directory /home/ns2/public_html>
    Options -Indexes +IncludesNOEXEC +SymLinksIfOwnerMatch 
    allow from all
    AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
    Require all granted
</Directory>
<Directory /home/ns2/cgi-bin>
    allow from all
    AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
    Require all granted
</Directory>
RemoveHandler .php
RemoveHandler .php8.1
#RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
#RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
#RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
<FilesMatch \.php$>
    SetHandler proxy:unix:/var/php-fpm/1650918378286180.sock|fcgi://localhost
</FilesMatch>
SSLEngine on
SSLCertificateFile /etc/ssl/virtualmin/1650918378286180/ssl.combined
SSLCertificateKeyFile /etc/ssl/virtualmin/1650918378286180/ssl.key
SSLCACertificateFile /etc/ssl/virtualmin/1650918378286180/ssl.ca
SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1

Edit: The well-known seems to be leftover from Let's Encrypt cert request process.

iliajie commented 2 years ago

Chris, what makes you think that the problem is in .well-know and not in redirect or in redirect + HSTS itself?

Do you still have too many redirects error if you change redirect to, for example:

Redirect permanent / https://virtual.server.tld

or 

RedirectMatch 301 "^\/((?!\.well\-known\/).*)$" "https://virtual.server.tld/$1"

Also, does it still perform redirect loop if you use instead:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{REQUEST_URI} !^/\.well\-known/acme\-challenge/
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</IfModule>

Jamie, in this issue https://github.com/virtualmin/virtualmin-gpl/issues/444 where I complain about request host not being preserved upon redirect, but it seems that the only solution is to use mod_rewrite (as in example above) to solve that problem. I presume this intentional that you use a simple RedirectMatch but do you think we could switch to using mod_rewrite instead?

iliajie commented 2 years ago

Although, if DNS is hosted on CloudFlare, then the only solution for this problem is to use CloudFlare SSL/TLS encryption mode set to Full or Full Strict.

I was thinking that possibly using RedirectMatch 302 ^/(?!.well-known)(.*)$ https://domain.com/$1 could solve too many redirects issue, as temporary redirects are not cached on the browser side (unlike permanent default (301)) but it doesn't work with CloudFlare (however could in some other cases).

Moreover, ideally, Virtualmin should avoid to setup Apache redirects in case of CloudFlare and use CloudFlare API to enable Always Use HTTPS option (settings/always_use_https).

chris001 commented 2 years ago

Chris, what makes you think that the problem is in .well-know and not in redirect or in redirect + HSTS itself?

I believe the problem is somewhere in the line "RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1" because, when I uncomment one of those 3 lines, then browse to virtual.server.tld, apache will redirect the page to itself, in an infinite loop... (the browser stops after about 20 times and says there was a problem with server, infinite redirect, try to clear cookies)

Do you still have too many redirects error if you change redirect to, for example:

Redirect permanent / https://virtual.server.tld

Yes this one also gives infinite redirect loop... should add / on the end of the https://virtual.server.tld/ because without it, a new problem is coming, apache will eat the / separating the server name and the page name.. https://virtual.server.tldpagename.php "domain not found, please check."

RedirectMatch 301 "^\/((?!\.well\-known\/).*)$" "https://virtual.server.tld/$1"

Yes this one is giving an infinite redirect loop..

Also, does it still perform redirect loop if you use instead:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{REQUEST_URI} !^/\.well\-known/acme\-challenge/
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</IfModule>

This one, rewrite version, is working, loads pages fine, no infinite redirect loop.

iliajie commented 2 years ago

This one, rewrite version, is working, loads pages fine, no infinite redirect loop.

Chris, is DNS hosted for this domain on CloudFlare?

chris001 commented 2 years ago

Chris, is DNS hosted for this domain on CloudFlare?

DNS is hosted for this domain on 2 virtualmin servers, on BIND9.

However, I just realize, to check if the rewrite module was enabled. It was disabled! Had assumed it was enabled. After enabled, and restart apache, this rewrite version also is a redirect infinite loop!

iliajie commented 2 years ago

Also, does it still perform redirect loop if you use instead:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{REQUEST_URI} !^/\.well\-known/acme\-challenge/
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</IfModule>

This one, rewrite version, is working, loads pages fine, no infinite redirect loop.

Are you really sure then that this redirect works as expected?

chris001 commented 2 years ago

Also, does it still perform redirect loop if you use instead:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{REQUEST_URI} !^/\.well\-known/acme\-challenge/
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</IfModule>

This one, rewrite version, is working, loads pages fine, no infinite redirect loop.

Are you really sure then that this redirect works as expected?

When mod_rewrite is enabled, that mod_rewrite version of the redirect is failing, it's infinite redirect loop.

iliajie commented 2 years ago

This one, rewrite version, is working, loads pages fine, no infinite redirect loop.

You said here that it works?

chris001 commented 2 years ago

This one, rewrite version, is working, loads pages fine, no infinite redirect loop.

You said here that it works?

I said it without realizing mod_rewrite was disabled, so that code wrapped in the block <IfModule mod_rewrite.c> did not run! My mistake! Page only loaded fine because that code didn't run! When I enabled mod_rewrite, that code caused infinite redirect loop!

iliajie commented 2 years ago

Alright then, as I expected it to fail at first and was surprised to hear that it didn't.

What do you think is causing redirect loop? The code that performs redirect is certainly correct.

chris001 commented 2 years ago

What is causing the redirect loop? Great question! I guess, first, to check the apache access log and error log for this virtual host..? Second, maybe there is a virtualmin setting to treat subdomains equal to domains, it's maybe in a the domain level setting or config? Third, maybe it's related to well-known which is a special thing, so try a different redirect, without that well-known, perfect syntax, supposed to work, see if it really works, that means bug is related to well-known? Last, paste the redirect into an online apache mod rewrite syntax checker, to verify if it really is perfect, and it does not have a small syntax bug in there somewhere?

iliajie commented 2 years ago

Chris, can you tell more about your setup? I cannot reproduce redirect loop issue on regular install (without cloud DNS).

Also, why your initial issue request has three lines of RedirectMatch directives on the config?

...
#RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
#RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
#RedirectMatch ^/(?!.well-known)(.*)$ https://virtual.server.tld/$1
...

Which cloud provider are you using?

chris001 commented 2 years ago

Not sure why there was 3 directives in the apache config. I believe the Virtualmin Lets Encrypt certificate request placed 1 of those 3 lines there, and Virtualmin failed to remove it? Next month, same. Next month, same... 3 lines left behind there. Next month, probably 4 lines there. What cloud provider? Ordinary generic Linux KVM VPS servers with IPv4, IPv6 thru tunnelbroker. DNS is self hosted on 2 virtualmin systems BIND9. For more details, can you private message, or email? I could also create an account for you on the virtualmin system to login and browse if you want..

iliajie commented 2 years ago

Not sure why there was 3 directives in the apache config. I believe the Virtualmin Lets Encrypt certificate request placed 1 of those 3 lines there, and Virtualmin failed to remove it? Next month, same. Next month, same... 3 lines left behind there. Next month, probably 4 lines there.

Well, no, as we don't do this. Besides it would make no sense to add RedirectMatch of such kind as Let's Encrypt doesn't need it. This redirect is added by a separate feature, to make sure that http version of the website is redirected to https.

Ordinary generic Linux KVM VPS servers with IPv4, IPv6 thru tunnelbroker.

Possibly network tunneler does some redirects internally?

I wonder, if you put that RidirectMatch directive inside of <Directory /home/domain/public_html> - does it change anything?

Also, is there anything prominent in response headers?

chris001 commented 2 years ago

Response headers when RedirectMatch inside <Directory /home/domain/public_html>

HTTP/1.1 302 Found =>
Date => Sat, 10 Sep 2022 15:27:22 GMT
Server => Apache/2.4.52 (Ubuntu)
Strict-Transport-Security => max-age=63072000; includeSubDomains; preload
X-Frame-Options => DENY
X-Content-Type-Options => nosniff
Location => https://virtual.server.tld/
Content-Length => 304
Connection => close
Content-Type => text/html; charset=iso-8859-1

EDIT: Response headers when comment out that RedirectMatch:

HTTP/1.1 200 OK =>
Date => Sat, 10 Sep 2022 15:31:22 GMT
Server => Apache/2.4.52 (Ubuntu)
Strict-Transport-Security => max-age=63072000; includeSubDomains; preload
X-Frame-Options => DENY
X-Content-Type-Options => nosniff
Upgrade => h2,h2c
Connection => Upgrade, close
Vary => Accept-Encoding
Content-Type => text/html; charset=UTF-8