alexjustesen / speedtest-tracker

Speedtest Tracker is a self-hosted internet performance tracking application that runs speedtest checks against Ookla's Speedtest service.
https://speedtest-tracker.dev/
MIT License
2.76k stars 106 forks source link

Docker with Traefik : HTTPS Mixed Content #280

Closed guigui42 closed 1 year ago

guigui42 commented 1 year ago

I am using the following Docker compose template :

version: '3.4'
services:
  speedtesttracker:
    image: ghcr.io/alexjustesen/speedtest-tracker:latest
    container_name: speedtesttracker
    networks:
      backend:
    ports:
      - 8766:80
    volumes:
      - speedtest2:/config
    environment:
      - PUID=272
      - PGID=276
    labels:
      - traefik.enable=true
      - traefik.http.routers.speedtest2.rule=Host(`speedtest2.XXXXXXXX.com`)
      - traefik.http.services.speedtest2.loadbalancer.server.port=80
      - traefik.docker.network=web
      - traefik.http.routers.speedtest2.tls=true
      - traefik.http.routers.speedtest2.tls.certresolver=myresolver

volumes:
  speedtest2:
    name: speedtest2
  backend:
    external: true

when trying to access it using my domain, I get the mixed content errors : https://speedtest2.XXXXXXXX.com/admin/login

image image

And looking at the source code, it is indeed trying to load HTTP content into my HTTPS page : image

Trying to access the same page using the HTTP page will work fine : http://MYIP:8766

Is there a way to pass the Base URL as en Docker Env variable to avoid this ?

alexdelprete commented 1 year ago

This topic has been discussed many times, see discussions in #34 #54 #95.

And looking at the source code, it is indeed trying to load HTTP content into my HTTPS page :

You configured Traefik to access ST via http/80:

      - traefik.http.routers.speedtest2.rule=Host(`speedtest2.XXXXXXXX.com`)
      - traefik.http.services.speedtest2.loadbalancer.server.port=80

ST now uses mixed mode, so it accepts both http/80 and https/443 connections. It's up to you to configure Traefik accordingly (https/443) to avoide mixed-content. Otherwise you access ST via traefik via https/443 and traefik pulls content via http/80, that creates the issue.

guigui42 commented 1 year ago

Thanks, with your help and previous discussions you mentioned, the solution was to change the port to 443, but also to add the following label :

- traefik.http.services.speedtest2.loadbalancer.server.scheme=https

and inside the Traefik commands : - --serversTransport.insecureSkipVerify=true

And Happy new year ;)

alexdelprete commented 1 year ago

That's one of the reasons why I don't like using labels: too many things to change. In the dynamic file I only change the URL to the backend service. :)

Happy new year.

P.S.: can you close the issue please? Thanks.

pbek commented 1 year ago
  • --serversTransport.insecureSkipVerify=true

Changing a setting inside traefik that affects the whole server doesn't seem to be a very good option imho. 😅

alexdelprete commented 1 year ago

Changing a setting inside traefik that affects the whole server doesn't seem to be a very good option imho. 😅

Is there a better way to tell Traefik to ignore certificate errors for backend services using self-signed certificates? I'm all ears. ;)

pbek commented 1 year ago

No, I haven't so far. 😅 But the main issue here for me is that speedtest-tracker seems to ignore my APP_URL setting in the .env, where I'm using the https://mydomain URL. It still makes requests to http://mydomain.

alexdelprete commented 1 year ago

No, I haven't so far.

So why did you say it's not a good option? If you want to use backend with SS certificates, it's the ONLY option. Better than http (for the backend) if you ask me. :)

But the main issue here for me is that speedtest-tracker seems to ignore my APP_URL setting in the .env, where I'm using the https://mydomain URL. It still makes requests to http://mydomain.

Why do you think APP_URL should have effect on ST? Did you read it somehwere in the docs?

If you read the discussions I pointed to @guigui42, you'll find out that ST uses SSL_MODE to govern the schema, and now it defaults to SSL_MODE=mixed, that means that it will accept connections both on http/80 and https/443 (through the SSL cert.).

If you use Traefik accepting only https client connections, you must use https schema to SpeedTest-Tracker (that implies using insecureSkipVerify setting), otherwise you have mixed-mode errors in the browser. It's Traefik that governs the schema (http/https) used when calling ST urls.

alexdelprete commented 1 year ago

In this post I detailed quita a few things regarding this issue.

And in this one I summarized in a couple of lines how to make Traefik work correctly with ST.

@alexjustesen: in the past you asked me to revise the docs regarding Traefik integration configuration, has that been merged in the docs? Can't find it.

pbek commented 1 year ago

I don't wanted to sound disrespectful! Thank you for the links to the posts with more explanation.

Why do you think APP_URL should have effect on ST? Did you read it somehwere in the docs?

I didn't found it. I'm sorry. I just was confused by the setting in the .env.

If someone wants to use the container standalone having TLS termination inside it surely practical. But I guess most of the times someone would want to "separate concerns" and use a reverse proxy (like traefik, caddy or even apache or ngnix directly) and do TLS termination there. It then wouldn't make a lot of sense to do TLS termination twice.

I myself probably would try to use the X-Forwarded-* server variables to check if some reverse proxy is in place (X-Forwarded-Host, X-Forwarded-Port, X-Forwarded-Proto) and use that as URL, but I don't know how practical that would be in this case. 🤷🏻

Anyway, setting insecureSkipVerify to true does work, so thank you very much for maintaining this project!

alexdelprete commented 1 year ago

I don't wanted to sound disrespectful! Thank you for the links to the posts with more explanation.

You weren't disrespectful, but it sounded like you thought this was not working because it was not properly designed/thought, that's all. No problem at all Patrizio, and sorry if I sounded rude, it was not my intention, I was only trying to understand what you meant.

it then wouldn't make a lot of sense to do TLS termination twice.

When I tell people that use reverse proxies that it's not mandatory to do TLS termination also on the backend once you let the proxy handle it they say it's not secure because they read "somewhere" that it's better to do it both on the proxy AND on the backend service. Makes no sense to me but they get caught by this "SSL EVERYWHERE" / "SSL END-TO-END" magic thing...(pure marketing stuff). :)

In this specific case though, you need to do TLS on both because of the mixed-content issue. It's a workaround, as I wrote in the issues I linked you before, but it's pretty solid and there are not a lot of cons. I don't know how other PHP frameworks/setups manage these things without having this mixed-mode issue, but I'm sure it can be solved in some way. Until then, this is the recommended configuration.

Anyway, setting insecureSkipVerify to true does work, so thank you very much for maintaining this project!

Glad it's working for you, I had no doubts it would. And last thing: I'm not the maintainer, @alexjustesen is the maintainer and dev. I'm just one of the first guys who embraced the project since day 1, and unfortunately we have the same name. :D

Ciao,

Alessandro

pbek commented 1 year ago

"SSL EVERYWHERE" / "SSL END-TO-END"

I only do that when I use an extra external reverse proxy, like Cloudflare. That makes a lot of sense to me. But I don't tend to also secure the network connection from the local traefik container to the local web-application container. 😁

In this specific case though, you need to do TLS on both because of the mixed-content issue.

I didn't grasp yet why ST needs to run in "mixed-mode" (doing http and https requests and not "only-http" or "only-https"). I always thought it was a settings/... issue that ST doesn't know to which URL the requests should be made.

Do you think it's a Laravel thing? In my projects I usually use one env var with the URL to make the requests to (if an absolute URL is needed and a relative URL isn't enough).

And last thing: I'm not the maintainer, @alexjustesen is the maintainer and dev. I'm just one of the first guys who embraced the project since day 1, and unfortunately we have the same name.

Haha, thank you! It seems it was really early for me when I answered the message. 😆 😅

alexdelprete commented 1 year ago

I only do that when I use an extra external reverse proxy, like Cloudflare.

I use cloudflare as DNS provider, and I use cloudflared as the main entry point so I don't need to open anything on OPNsense. Cloudlfared then routes everything to Traefik that allows me to do much more stuff. It's a secure and flexible architecture, IMHO.

I didn't grasp yet why ST needs to run in "mixed-mode" (doing http and https requests and not "only-http" or "only-https")

It's not ST: Traefik manages the connection with the client, it then manages the connection to the backend service, and it lets the 2 communicate "transparently". Problem is that if you configure the backend access with http/80, while the client uses https/443 with Traefik, the browser complains for mixed-content because it sees the client (Traefik) using http schema URLs inside an https connection.

ST is just a bunch of scripts/code/etc. :)

The problem is how Traefik is managing these scenarios, it seems it's not setting some headers correctly, and we're not the only ones complaining about this, and using the forced TLS on the backend workaround, read here: https://community.traefik.io/t/force-https-on-traefik-with-docker-compose-mixed-content/13529/5

pbek commented 1 year ago

Cloudlfared

Yes, for keeping all external ports closed it's great! I didn't wanted to rely on it too much tho, you'll never know when they start charging money for cloudflared or when your account gets locked or something.

It's not ST: Traefik manages the connection with the client, it then manages the connection to the backend service, and it lets the 2 communicate "transparently". Problem is that if you configure the backend access with http/80, while the client uses https/443 with Traefik, the browser complains for mixed-content because it sees the client (Traefik) using http schema URLs inside an https connection.

And I ask why it's necessary to let traefik access ST via https (and not http). The ST frontend is making those mixed requests to the ST backend. If it would detect how the request was made (https) or just make all requests to the APP_URL env var it creates in .env then there wouldn't be any mixed content issues, wouldn't there? 😁

alexdelprete commented 1 year ago

Yes, for keeping all external ports closed it's great! I didn't wanted to rely on it too much tho, you'll never know when they start charging money for cloudflared or when your account gets locked or something.

Well, until free, I'll use it, when it won't be free, I'll open up OPNsense. :) But for now it's the best solution. Actually Cloudflared is one of the only services/apps that I really consider "iconoclastic", it's simple and changes the scenario completely, I love it.

And I ask why it's necessary to let traefik access ST via https (and not http). The ST frontend is making those mixed requests to the ST backend.

Because it doesn't know: Traefik seems not to set the headers properly. Read that thread I linked you on Traefik forum. If you use NGINX or Caddy as proxies, the mixed-content problem doesn't show up, because they set headers properly. I'm now experimenting with a redirect middleware that uses regexp to convert all http urls to https, to be used for ST and other Laravel apps.

If it would detect how the request was made (https) or just make all requests to the APP_URL env var it creates in .env then there wouldn't be any mixed content issues, wouldn't there? 😁

The request are made in whatever schema (http/https) you configured Traefik to use for the backend, that's the issue. Traefik should override that based on the user (client) schema, not the backend schema. That is the root cause.

I already told Alex in the past that APP_URL could be used to enforce the schema, like many PHP apps do. But if you ask me, that's another workaround, the reverse proxy should really be "transparent".

alexdelprete commented 1 year ago

@alexjustesen I found an enforce_ssl workaround for Laravel: https://stackoverflow.com/a/70926207

Could be useful in some cases to have a config option to force this.

image

pbek commented 1 year ago

Because it doesn't know: Traefik seems not to set the headers properly.

Hm, if it's about headers those could also be overwritten for the container with Traefik. But the X-Forwarded-* headers are set just fine for me with Traefik by default.

alexdelprete commented 1 year ago

if it's about headers those could also be overwritten for the container with Traefik. But the X-Forwarded-* headers are set just fine for me with Traefik by default.

I remembered I was using another service (LinkAce) based on Laravel, that has a pretty good docs site, and look what I found: https://www.linkace.org/docs/v1/setup/setup-with-docker/advanced-configuration/

image

alexjustesen commented 1 year ago

Holy crap... for different reasons I need this.

alexdelprete commented 1 year ago

Holy crap... for different reasons I need this.

Need what? The info regarding proxies? Well, take a look around there, you might take several things, also the non-docker installation is interesting...you had it in the roadmap.

alexjustesen commented 1 year ago

No no the bookmarks thing

alexdelprete commented 1 year ago

LinkAce is fantastic. I'm only missing an official plugin for Chrome/Edge. Dev said he will work on it...using a bookmarklet for now.

pbek commented 1 year ago

Hehe, I manage bookmarks locally in Markdown files and the browser extension for it via QOwnNotes. 😁

alexdelprete commented 1 year ago

LinkAce has a bit more functionalities than a simple bookmark manager. ;)

pbek commented 1 year ago

The link checker is nice...

alexdelprete commented 1 year ago

Archive.org integration too...

KaKi87 commented 8 months ago

I remembered I was using another service (LinkAce) based on Laravel, that has a pretty good docs site, and look what I found: linkace.org/docs/v1/setup/setup-with-docker/advanced-configuration

That was the solution for Apache, thanks !

RequestHeader set x-forwarded-port 443
RequestHeader set x-forwarded-proto https
ProxyPass / http://localhost:<http_port>/
ProxyPassReverse / http://localhost:<http_port>/

(The ProxyPreserveHost line happened not to be necessary).