Production-NoReverseProxy - Using docker image & apache as reverse proxy for rtl & lnd REST possible solution?

[Shinbar]

Have to say what a great piece of work this is! I did run into a problem which I think I have solved, just putting it down here so it might help others (sorry if its not the correct place). I am successfully running production btcpayserver-docker with NoReverseProxy config, behind an apache server configured as a reverse proxy.

The apache server is handling a letsencrypt ssl and is correctly forwarding traffic through to server running the btcpayserver docker containers.

Apache config required to get this working:

<VirtualHost *:443>

ServerName btcpay.example.com
RequestHeader set X-Forward-Proto "http"
ProxyPreserveHost On

ProxyPass /.well-known/acme-challenge # let apache server handle lets encrypt certs config

ProxyPass / http://192.168.8.1
ProxyPassReverse / http://192.168.8.1/

LimitRequestLine 500000
LimitRequestFieldSize 500000

DocumentRoot /var/www/html/letsencrypt/btcpayserver
<Directory /var/www/html/letsencrypt/btcpayserver>
     Options +Indexes +FollowSymLinks +MultiViews +ExecCGI
     AddHandler cgi-script .pl .cgi
     AllowOverride All
     Require all granted
</Directory>
SSLCertificateFile /etc/letsencrypt/live/btcpay.example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/btcpay.example.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf

</VirtualHost>

The Problem The problem came when I try and access the lnd REST apis and rtl UI, this results in a 404 page not found.

Possible Solution: The docker containers running those services do not expose them outside the containers to the host machine, as I suspect they are expected to be made available to a nginx container instead (which I am not running).

To circumvent this I have edited the Generated/docker-compose.generated.yml and edited lnd_bitcoin so that port 8080 is exposed to the host, and under bitcoin_rtl port 3000 is exposed to the host as follows

lnd_bitcoin:
.....
....
ports:
- "8080:8080"

and

bitcoin_rtl:
....
....
ports:
- "3000:3000"

Not sure if this is the correct place to do this, does Generated/docker-compose.generated.yml get overwritten (perhaps on an update)? Is there a better different place to put these changes so they are no in danger of being overwritten?

Added the following to my Apache Config:

ProxyPass /rtl http://192.168.8.1:3000/rtl
ProxyPassMatch "^/lnd-rest/btc/(.*)" "http://192.168.8.1:8080/$1"

Now I can from my btcpay server UI get to RTL UI and the lnd rest api is now accessable, thought this might be helpful to other.

[NicolasDorier]

Editing the generated.xml is not a good idea, as it will be overwritten when you update.

https://docs.btcpayserver.org/faq-and-common-issues/faq-deployment#cause-3-btcpay-is-expecting-you-to-access-this-website-from

Also check

https://docs.btcpayserver.org/faq-and-common-issues/faq-deployment#can-i-use-an-existing-nginx-server-as-a-reverse-proxy-with-ssl-termination

You can still use nginx inside the docker-compose, while delegating things to apache.

[Shinbar]

Thanks @NicolasDorier for your pointers, didn't not occur to me that you could leave the nginx docker container and still have an apache2 server running as a reverse proxy. I have re-instated the nginx server by doing:

export BTCPAYGEN_REVERSEPROXY="none"

and made sure that the nginx container was not handling LetsEncrypt SSL as that is still managed by my externally facing Apache2 server, by doing the following:

BTCPAYGEN_EXCLUDE_FRAGMENTS="$BTCPAYGEN_EXCLUDE_FRAGMENTS;nginx-https"

I then ran into an Error 503 Service Temporarily Unavailable could see this was issued by the nginx server (signature shown). I found #247 and resolved by:

export REVERSEPROXY_DEFAULT_HOST="$BTCPAY_HOST"

which allows access to the internal BTCPayServer without maching doman names.

All good with this configuration, no changes were needed to the Apache2 server.

New Problem

Removed the Apache reverse proxy configuration for the Ride The Lighting container, as I thought it should route to this now through the nginx server, however it was reporting back a 502 Bad Gateway.

docker ps shows that the container running RTL has an internal port of 3000/tcp however I cannot see how that is exposed through the nginx? As that only appear to have port 80/tcp exposed externally?

533f2146893b shahanafarooqui/rtl:0.6.8 "/sbin/tini -g -- no…" 36 minutes ago Up 36 minutes 3000/tcp generated_bitcoin_rtl_1

Again I am forced back to modifying the Generated/docker-compose.generated.yml adding

bitcoin_rtl:
....
....
ports:
- "3000:3000"

and then doing a docker-compose -f "Generated/docker-compose.generated.yml" up --remove-orphans -d and then pointing my reverse proxy back to this port 3000 which then allows this to work.

As you quite rightly said very much want to avoid editing the Generated/docker-compose.generated.yml any ideas as to what is wrong or I am doing wrong here?

[NicolasDorier]

export BTCPAYGEN_REVERSEPROXY="none"

If you did that, there is no nginx.

[NicolasDorier]

however I cannot see how that is exposed through the nginx?

Connect to btcpay site as admin, server settings / services, then you can just click there.

[Shinbar]

Thanks @NicolasDorier for getting back so quickly. My mistake I did have the following set export BTCPAYGEN_REVERSEPROXY="nginx" so I do have the docker containers as follows and nginx is working well as I can log on to the server etc.

$ docker ps
CONTAINER ID        IMAGE                                 COMMAND                  CREATED             STATUS              PORTS                                                                                    NAMES
5bbcc69dff42        shahanafarooqui/rtl:0.6.8             "/sbin/tini -g -- no…"   3 minutes ago       Up 3 minutes        3000/tcp                                                                                 generated_bitcoin_rtl_1
7fa82abd5f3d        btcpayserver/docker-gen:0.7.7         "/usr/local/bin/dock…"   14 hours ago        Up 14 hours                                                                                                  nginx-gen
ce40285f3c3b        nginx:1.16.0                          "nginx -g 'daemon of…"   14 hours ago        Up 14 hours         0.0.0.0:80->80/tcp                                                                       nginx
df8bf1e8a55c        btcpayserver/btcpayserver:1.0.3.164   "/app/docker-entrypo…"   14 hours ago        Up 14 hours         49392/tcp                                                                                generated_btcpayserver_1
92b7c751e664        btcpayserver/lnd:v0.8.2-beta          "/sbin/tini -g -- /d…"   14 hours ago        Up 14 hours         8080/tcp, 0.0.0.0:9735->9735/tcp                                                         btcpayserver_lnd_bitcoin
538eeef7e56d        btcpayserver/docker-gen:0.7.7         "/usr/local/bin/dock…"   14 hours ago        Up 14 hours                                                                                                  tor-gen
cf6942625843        btcpayserver/bitcoin:0.19.0.1         "/entrypoint.sh bitc…"   14 hours ago        Up 14 hours         8332-8333/tcp, 18332-18333/tcp, 18443-18444/tcp, 28332-28333/tcp, 39388/tcp, 43782/tcp   btcpayserver_bitcoind
0225f2982927        postgres:9.6.5                        "docker-entrypoint.s…"   14 hours ago        Up 14 hours         5432/tcp                                                                                 generated_postgres_1
088d09d62592        nicolasdorier/nbxplorer:2.1.14        "dotnet NBXplorer.dll"   14 hours ago        Up 14 hours         32838/tcp                                                                                generated_nbxplorer_1
bf09aaac6da6        btcpayserver/tor:0.4.1.5              "./entrypoint.sh tor"    14 hours ago        Up 14 hours         9050-9051/tcp                                                                            tor
r

I had also discovered I had left in the apache2 reverse proxy config to route through to the RTL 3000 port container. As you said nginx should be handling this, after removing this config from apache2, and removing my mods of the docker-compose.generated.yml it all works well and as you designed.

I can now get to RTL via server settings / services :) Keep up the great work.