Using apache as reverse proxy to nginx

[RobertHosking]

I'm trying to use Apache as a reverse proxy. But I can't seem to figure out how without the nginx complaining. Here is my Apache config:

<VirtualHost *:80>
    ServerName btcpay.mydomain.com
    ServerAdmin webmaster@localhost
    ProxyPreserveHost On
    ProxyPass / http://localhost:9000/
    ProxyPassReverse / http://localhost:9000/
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
        RewriteEngine on
        RewriteCond %{SERVER_NAME} =btcpay.mydomain.com
        RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

As you can see, Apache should be listening for requests on btcpay.mydomain.com and proxying to port 9000 on the host machine.

So here is my first attempt at generating docker-compose.yml (I omitted the blank ones):

-------SETUP-----------
Parameters passed:
BTCPAY_PROTOCOL:https
BTCPAY_HOST:btcpay.mydomain.com
BTCPAY_ADDITIONAL_HOSTS:
REVERSEPROXY_HTTP_PORT:8999
REVERSEPROXY_HTTPS_PORT:9000
REVERSEPROXY_DEFAULT_HOST:none:
NBITCOIN_NETWORK:mainnet
BTCPAYGEN_CRYPTO1:btc
BTCPAYGEN_REVERSEPROXY:nginx
BTCPAYGEN_LIGHTNING:lnd::
ACME_CA_URI:https://acme-v01.api.letsencrypt.org/directory
----------------------
Additional exported variables:
BTCPAY_DOCKER_COMPOSE=/root/btcpayserver-docker/Generated/docker-compose.generated.yml
BTCPAY_BASE_DIRECTORY=/root
BTCPAY_ENV_FILE=/root/.env
BTCPAYGEN_OLD_PREGEN=false
BTCPAY_SSHTRUSTEDFINGERPRINTS:
BTCPAY_CRYPTOS:btc
BTCPAY_ANNOUNCEABLE_HOST:btcpay.mydomain.com

The result: Nginx 400 error: The plain HTTP request was sent to HTTPS port

Ok, so this makes me think that maybe I should configure BTCPay to run on HTTP instead since Apache is handling SSL and redirects to HTTPS anyways.

Here is my second attempt (With comments on what has changed)

-------SETUP-----------
Parameters passed:
BTCPAY_PROTOCOL:http    # change to http
BTCPAY_HOST:btcpay.mydomain.com
BTCPAY_ADDITIONAL_HOSTS:
REVERSEPROXY_HTTP_PORT:9000    # set port 9000 to http instead of https
REVERSEPROXY_HTTPS_PORT:8999   #set port 8999 to https instead of http
REVERSEPROXY_DEFAULT_HOST:none:
NBITCOIN_NETWORK:mainnet
BTCPAYGEN_CRYPTO1:btc
BTCPAYGEN_REVERSEPROXY:nginx
BTCPAYGEN_LIGHTNING:lnd::
ACME_CA_URI:https://acme-v01.api.letsencrypt.org/directory
----------------------
Additional exported variables:
BTCPAY_DOCKER_COMPOSE=/root/btcpayserver-docker/Generated/docker-compose.generated.yml
BTCPAY_BASE_DIRECTORY=/root
BTCPAY_ENV_FILE=/root/.env
BTCPAYGEN_OLD_PREGEN=false
BTCPAY_SSHTRUSTEDFINGERPRINTS:
BTCPAY_CRYPTOS:btc
BTCPAY_ANNOUNCEABLE_HOST:btcpay.mydomain.com

The result: Page not available: server redirected too many times

I have also tried about a dozen other things but they all result in one of those two errors.

Any idea what's going on here? Thanks!

[NicolasDorier]

Have you checked https://docs.btcpayserver.org/faq-and-common-issues/faq-deployment#btcpay-is-expecting-you-to-access-this-website-from you need to properly forward the real host.

[RobertHosking]

@NicolasDorier Thanks! I have tried that, but I'm still not able to access over HTTPS. See the config below.

I have made some progress. I can access the site over HTTP but not over HTTPS.

I was able to get this far by:

• Adding nginx-https to BTCPAY_EXCLUDE_FRAGMENTS to stop nginx from generating SSL certs since I am offloading that to Apache/Certbot on the host machine
• Creating two virtual hosts (one for HTTP :80 and one for HTTPS :443)
• Proxying traffic to the REVERSEPROXY_HTTP port instead of REVERSEPROXY_HTTPS port
• Disabling automatic redirect to HTTPS that was put there by Certbot

With this setup I can access the site at http://btcpay.mydomain.com with a notice that says:

"Your access to BTCPay Server is over an unsecured network. If you are using the docker deployment method with NGINX and HTTPS is not available, you probably did not configure your DNS settings correctly."

When I visit https://btcpay.mydomain.com, I get Nginx 400 error: The plain HTTP request was sent to HTTPS port.

Here is my current Apache config

<VirtualHost *:80>
    ServerName btcpay.mydomain.com
    ServerAdmin webmaster@localhost
    ProxyPreserveHost On
    ProxyPass / http://localhost:8999/
    ProxyPassReverse / http://localhost:8999/
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
#RewriteEngine on
#RewriteCond %{SERVER_NAME} =btcpay.mydomain.com
#RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:433>
    ServerName btcpay.mydomain.com
    ServerAdmin webmaster@localhost
    RequestHeader set X-Forwarded-Proto "https"
    ProxyPreserveHost On
    ProxyPass / http://localhost:8999/
    ProxyPassReverse / http://localhost:8999/
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Here are my ENV variables (null values excluded):

-------SETUP-----------
Parameters passed:
BTCPAY_PROTOCOL:http
BTCPAY_HOST:btcpay.mydomain.com
REVERSEPROXY_HTTP_PORT:8999
REVERSEPROXY_HTTPS_PORT:9000
REVERSEPROXY_DEFAULT_HOST:none
NBITCOIN_NETWORK:mainnet
BTCPAYGEN_CRYPTO1:btc
BTCPAYGEN_REVERSEPROXY:nginx
BTCPAYGEN_LIGHTNING:none
BTCPAYGEN_ADDITIONAL_FRAGMENTS:
BTCPAYGEN_EXCLUDE_FRAGMENTS:nginx-https
ACME_CA_URI:https://acme-v01.api.letsencrypt.org/directory
----------------------
Additional exported variables:
BTCPAY_DOCKER_COMPOSE=/root/btcpayserver-docker/Generated/docker-compose.generated.yml
BTCPAY_BASE_DIRECTORY=/root
BTCPAY_ENV_FILE=/root/.env
BTCPAYGEN_OLD_PREGEN=false
BTCPAY_CRYPTOS:btc
BTCPAY_ANNOUNCEABLE_HOST:btcpay.mydomain.com
----------------------

[NicolasDorier]

@RobertHosking have you checked this link: https://docs.btcpayserver.org/faq-and-common-issues/faq-deployment#can-i-use-an-existing-nginx-server-as-a-reverse-proxy-with-ssl-termination I think you are missing X-Forwarded-Proto

[RobertHosking]

@NicolasDorier I am using Apache on host machine, not Nginx. I included the X-Forwarded-Proro in Apache with no effect.

My reverse proxy should look like this:

1. Client sends HTTP request to Apache
2. Apache sees that client has sent HTTP request.
3. Apache sends back 3xx redirect response telling the Client to connect to https://
4. Client sends HTTPS request to Apache
5. Apache decrypts the HTTPS traffic and sets the X-Forwarded-Proto: https
6. Apache sends the HTTP request to the BTCPay Nginx container
7. Nginx sees that the URL is http:// but also sees that X-Forwarded-Proto is https and trusts that the request is HTTPS
8. Nginx sends back the requested web page or data

I have made some changes. Maybe I'm getting closer.

This is my Apache config now:

<VirtualHost *:80>
        ServerName btcpay.mydomain.com
        ServerAdmin webmaster@localhost
        RequestHeader set X-Forwarded-Proto "https"
        ProxyPreserveHost On
        ProxyPass / http://localhost:8999/
        ProxyPassReverse / http://localhost:8999/
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
#RewriteEngine on
#RewriteCond %{SERVER_NAME} =btcpay.mydomain.com
#RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

I just added the X-Forwarded-Proto header to the <VirtualHost *:80> block and removed the <VirtualHost *:443> block all together.

With this setup, I can still access over HTTP. The error from BTCPay now reads:

BTCPay is expecting you to access this website from https://btcpay.mydomain.com/. If you use a reverse proxy, please set the X-Forwarded-Proto header to http (More information)

This is expected. It means that the X-Forwarded-Proto header is working. Now, I just have to terminate SSL in apache... I think..

When I add the rewrite rules that redirect to HTTPS, I get Apache Proxy error 502: Error reading from remote server

I'm still stuck. Cannot get HTTPS to work.

[NicolasDorier]

Invoking @woutersamaey I know he managed to make it works.

[NicolasDorier]

I don't know about apache, but it does not seem correct. The virtual host should be 443, because if I understand, your apache handle the HTTPS part.

[NicolasDorier]

Can you also check in the sources of the home btcpay page? At the bottom you should see something like.

<script type="text/javascript">
--
  | var expectedDomain = "mainnet.demo.btcpayserver.org";
  | var expectedProtocol = "https";
  | if (window.location.host != expectedDomain \|\| window.location.protocol != expectedProtocol + ":") {
  | document.getElementById("badUrl").style.display = "block";
  | document.getElementById("browserScheme").innerText = window.location.protocol.substr(0, window.location.protocol.length -1);
  | }
  | </script>

[NicolasDorier]

Oh now you want to make Apache handle HTTPS sorry I have not read correctly. I can't help you on that, as I don't relaly know how to configure apache SSL. :(

For information why are you using Apache in front?

[woutersamaey]

This should not be difficult. Let me look it up and get back to you.

[woutersamaey]

1. In Apache you must have 2 virtual hosts and have the RequestHeader set X-Forwarded-Proto "https" line in the :443 vhost, as this is the one doing the offloading. You have put this in the :80 vhost which is wrong.
2. The :80 vhost should be a simple redirect to the :443 vhost, so you enforce users to connect safely. Nothing more is needed there.

Besides this you setup looks good...

You may still run into issues with PSBT later on. For Nginx we needed to increase some buffers to allow for very long URLs. Not sure if Apache can handle very long URLs without modification.

[NicolasDorier]

You may still run into issues with PSBT later on. For Nginx we needed to increase some buffers to allow for very long URLs. Not sure if Apache can handle very long URLs without modification.

I really need to find better way of passing PSBT around...

[woutersamaey]

I haven't tried the PSBT feature so I'm not sure what the current implementation is. Basically if you need to move more data around with web requests it's best to put the data in POST requests. Alternatives may be storing them in the user session or in a cookie, but that has it's own drawbacks. If it's just a couple of forms, like in a wizard, I would put everything in POSTs. If you don't want the pages to look like forms, you can use Javascript on a text link (example) to trigger the submit of a hidden form on the page and still end up with a POST for something that would normally be a GET. The downside of using POST is that when the user hits refresh, the browser will ask to "resubmit the data". If the wizard is long, you should consider storing the data in the user session and have a "reset link" which resets the process to step 1.

If you want, we can discuss further on a more proper channel.

[NicolasDorier]

It just hurt my feeling to make a POST requests when really my call is idempotent. Not mentioning that the route already have POST defined for other actions that are actually not idempotent :(

[NicolasDorier]

Also, at one point I redirect from javascript :(

[woutersamaey]

I'll check this out when I get more play time and make a proper suggestion.

[RobertHosking]

@woutersamaey I'm glad to hear someone got this to work with apache :)

@NicolasDorier I'm using apache because I have many docker services running on this server. Apache is the reverse proxy to those services. If I were to use Nginx for BTCPay, I would have to rewrite the reverse proxy configurations in Nginx for all my other services too.

With this setup, I am getting an Apache Proxy Error on HTTPS

@woutersamaey can you confirm that my apache and BTCPay env variables are correct?

Apache

<VirtualHost *:80>
    ServerName btcpay.mydomain.com
    ServerAdmin webmaster@localhost

    ProxyPreserveHost On
    ProxyPass / http://localhost:8999/
    ProxyPassReverse / http://localhost:8999/

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

        RewriteEngine on
        RewriteCond %{SERVER_NAME} =btcpay.mydomain.com
        RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:433>
    ServerName btcpay.mydomain.com
    ServerAdmin webmaster@localhost

    RequestHeader set X-Forwarded-Proto "https"
    ProxyPreserveHost On
    ProxyPass / http://localhost:8999/
    ProxyPassReverse / http://localhost:8999/

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

BTCPay Params

BTCPAY_PROTOCOL:http
BTCPAY_HOST:btcpay.mydomain.com
REVERSEPROXY_HTTP_PORT:8999
REVERSEPROXY_HTTPS_PORT:9000
REVERSEPROXY_DEFAULT_HOST:none
NBITCOIN_NETWORK:mainnet
BTCPAYGEN_CRYPTO1:btc
BTCPAYGEN_REVERSEPROXY:nginx
BTCPAYGEN_EXCLUDE_FRAGMENTS:nginx-https

[RobertHosking]

I noticed that the apache error logs show this

Connection reset by peer: [client :] AH01102: error reading status line from remote server localhost:9000, referer: https://btcpay.mydomain.com/

It seems that traffic is going to the REVERSEPROXY_HTTPS_PORT 9000 instead of port 8999 like the Apache config says. There's something else happening here.

Shouldn't the REVERSEPROXY_HTTPS_PORT be ignored since BTCPAY_PROTOCOL is http? I know I can't leave it empty or else it will try to use 443 which is already taken by apache.

[NicolasDorier]

BTCPAY_PROTOCOL is actually almost unused. It is just use so some services can point back to btcpay. (Like if you log out of RTL, it will redirect to http://btcpay.mydomain.com if you set BTCPAY_PROTOCOL to http)

[NicolasDorier]

It seems that traffic is going to the REVERSEPROXY_HTTPS_PORT 9000 instead of port 8999 like the Apache config says. There's something else happening here.

How is it possible? your apache config does not have any mention of port 9000

[RobertHosking]

My issue has been resolved! I just did a git pull and tried again and it worked!

Here's what I did:

export BTCPAY_HOST="btcpay.example.com"
export NBITCOIN_NETWORK="mainnet"
export BTCPAYGEN_CRYPTO1="btc"
export BTCPAYGEN_EXCLUDE_FRAGMENTS="nginx-https" # offload SSL termination to apache
 export REVERSEPROXY_HTTP_PORT=9001
export BTCPAYGEN_REVERSEPROXY="" # set empty since using apache as reverse proxy
. ./btcpay-setup.sh -i

create file /etc/apache2/sites-availible/btcpay.example.com.conf

file contents:

<VirtualHost *:80>
    ServerName btcpay.example.com
        RequestHeader set X-Forwarded-Proto https # add to offload SSL termination

    ServerAdmin webmaster@localhost
    ProxyPreserveHost On
    ProxyPass / http://localhost:9001/
    ProxyPassReverse / http://localhost:9001/

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
RewriteEngine on
RewriteCond %{SERVER_NAME} =btcpay.example.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

enable virtual host sudo a2ensite btcpay.example.com

sudo service apache2 restart

generate SSL cert sudo certbot --apache -d btcpay.example.com

[NicolasDorier]

awesome!