alexjustesen / speedtest-tracker

Speedtest Tracker is a self-hosted internet performance tracking application that runs speedtest checks against Ookla's Speedtest service.
https://speedtest-tracker.dev/
MIT License
2.77k stars 106 forks source link

`The page isn’t redirecting properly` / `bad request` #54

Closed filcuk closed 1 year ago

filcuk commented 1 year ago

I've just updated to the latest release. I'm getting the above error in browser, this is via Traefik. No info in the console. Container log follows:

[21-Oct-2022 09:32:37] NOTICE: fpm is running, pid 100
                                                            |_|
Brought to you by serversideup.net
--------------------------------------------------------------------
To support Server Side Up projects visit:
https://serversideup.net/sponsor
GID/UID
User uid:    1044
User gid:    65539
-------------------------------------
πŸ” SSL_MODE has set to FULL, setting the web server to work in HTTPS only...
πŸƒβ€β™‚οΈ Checking for Laravel automations...
πŸ” Linking the storage...
   ERROR  The [public/storage] link already exists.  
βœ…  Database exists
πŸ‡  Configuring Speedtest Tracker...
πŸ”—  Creating database symlink
βœ…  Environment file exists
πŸ”—  Creating symlinks to config and log files
βœ…  App key exists
πŸ”’  Fixing app path file permissions
πŸ’°  Building the cache...
127.0.0.1 - - [21/Oct/2022:09:32:38 +0000] "GET /ping HTTP/1.1" 301 162 "-" "curl/7.81.0"
[21-Oct-2022 09:32:37] NOTICE: ready to handle connections
[21-Oct-2022 09:32:37] NOTICE: systemd monitor interval set to 10000ms
πŸš›  Migrating the database...
βœ…  All set, starting Speedtest Tracker container...
πŸ’ͺ  Starting the queue worker...
xxx.xxx.xxx.xxx - - [21/Oct/2022:09:33:20 +0000] "GET / HTTP/1.1" 301 162 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0"

I've tested it through a localhost too, updating the .env accordingly in each case, and got a 404:

❌ There seems to be a failure in checking the web server + PHP-FPM. Here's the response:
192.168.90.1 - - [21/Oct/2022:09:39:12 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03Gk\x19mm\x18\x1F\x7F\xFD\xF2\xAB \xF8L\xA6\xD0\xFD\xD9/\xE6\xAB\xEF\xEA&X\x18sfl\x8D\xFF\x12 \xA1\x0C%\x92\xFF\x176\xB5\xB2\xA4Oog\xB3Y)&\xFF.\x14\xBF\xAE\x98\xBE\x16.\xA1\x07s\x00\xFF\x11\x00\x22\x13\x01\x13\x03\x13\x02\xC0+\xC0/\xCC\xA9\xCC\xA8\xC0,\xC00\xC0" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:21 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\x168\x0B\x021!\x92\xDA\x86m\xAF48s2\x1EC\x85+<\x01\x80+\x94X\x15\x00z\xED'\xD6\x00 V\xF5\xF5\x89:\xE4\x99\x8C\xA1\x09\xA8\x9D\x8D\xC7|V\x96\x16\xAC\xFC\x85" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:21 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xB9,\x93\xFB\xE9\x1C[.\x84\xC9\x5C\x07\x8E\x99\x93" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:21 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xA4\xF2\xA7j\x10\xEB\x8DdsEi\xEEa\x88\xC0\xCD\x06PUm\x00\x91:U%\xAB\xDB\xAD\x0E\x80\x1E\xBF 8\xA7\x87v=\x97" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:21 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xFBZ's\xA3q\x96&\xE7:\x8E\xDF\xFD#\xA5\x0Ef\x22\x08\xD2\xD1\x00^\x8A\xB1\xAB\x10;q\x22T\xEC \xD7\x19\x82}\x17\x1A<bN@\xDE\xC3:\x99,h\xB2\x8F\x9A\xE8\xAB\xF3n^\x05\x0F+\xAA\xA6\xCE\xE2\xB7\x00\x22ZZ\x13\x01\x13\x02\x13\x02\x13\x03\xC0+\xC0/\xC0,\xC00\xCC\xA9\xCC\xA8\xC0\x13\xC0\x14\x00\x9C\x00\x9D\x00/\x005\x01\x00\x01\x91\xDA\xDA\x00\x00\x00\x17\x00\x00\xFF\x01\x00\x01\x00\x00" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:22 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\x19\x9COx\x18\x8F \xFD\xB9\xB6\xDE\xC8oC\x158\x84\x0F\x83\xE8k\xDC\xA0\xCF&\x93" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:22 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03~\xC03!\xE5" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:22 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\x10\xB7\xE1b\x8E\xA0\xBE\x8C\x95\xC1\x0E\x84\x84#\xCC\x9C\x92\xF5\x7F\xEB\x11\xDB\xB2\xB2~'\xB84V\xCDb/ \xE4\xB1t$k\xD7\x1A4\xC5D\x90^\x86\x09\x1C\x88E\x8A\x13- \xAA\x7F\xFB\x96\x0E\xA8\x1B\x8C\xDF\x98_\x00\x22\x1A\x1A\x13\x01\x13\x02\x13\x02\x13\x03\xC0+\xC0/\xC0,\xC00\xCC\xA9\xCC\xA8\xC0\x13\xC0\x14\x00\x9C\x00\x9D\x00/\x005\x01\x00\x01\x91\xAA\xAA\x00\x00\x00\x17\x00\x00\xFF\x01\x00\x01\x00\x00" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:22 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xB7\x83\x98\x04\xD8X\x1D\x16\xDF<\xA4\xC3\xC5\x90" 400 150 "-" "-"
192.168.90.1 - - [21/Oct/2022:09:39:26 +0000] "GET / HTTP/1.1" 301 162 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.47"
192.168.90.1 - - [21/Oct/2022:09:39:46 +0000] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03=M\xBD<\x88\xBD\xA75\xBB\x14\xC5\x15e*O\xD4\xFB&\xA2ftB\x86haT4" 400 150 "-" "-"
alexjustesen commented 1 year ago

Just double checking did you restart the container after updating the .env file? Also try running php artisan optimize in the docker CLI window. Laravel caches the .env file to improve performance.

alexjustesen commented 1 year ago

Also in the cli you can run php artisan about to confirm your environment variables

filcuk commented 1 year ago

I run php artisan optimize:

   INFO  Caching the framework bootstrap files.

  config ........................................................... 35ms DONE
  routes .......................................................... 123ms DONE

Tested > Restarted > Tested again

Still the same issue unfortunately

* Just to clarify, I did restart between .env changes previously

alexjustesen commented 1 year ago

If you run php artisan about do you see your new APP_URL?

alexdelprete commented 1 year ago

Confirmed. Not working also for me.

My docker-compose.yml:

version: '3.3'
services:
  speedtest-tracker:
    image: ghcr.io/alexjustesen/speedtest-tracker:v0.1.0-alpha7
    container_name: speedtest-tracker
    restart: unless-stopped
    ports:
      - 8008:80
    environment:
      TZ: Europe/Rome
      PUID: 1000
      PGID: 1000
      DB_CONNECTION: mysql
      DB_HOST: mariadb.axel.dom
      DB_PORT: 3306
      DB_DATABASE: speedtest_tracker
      DB_USERNAME: xxxxxxx
      DB_PASSWORD: xxxxxxx
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - $PWD/config:/config

Log at first start of the container:

Brought to you by serversideup.net
--------------------------------------------------------------------
To support Server Side Up projects visit:
https://serversideup.net/sponsor
-------------------------------------
GID/UID
-------------------------------------
User uid:    1000
User gid:    1000
-------------------------------------
πŸ” SSL_MODE has set to FULL, setting the web server to work in HTTPS only...
πŸƒβ€β™‚οΈ Checking for Laravel automations...
πŸƒβ€β™‚οΈ An SSL key was not detected, so I'll generate a self-signed SSL certificate pair for you...
πŸ” Linking the storage...
   INFO  The [public/storage] link has been connected to [storage/app/public].
πŸ‡  Configuring Speedtest Tracker...
βœ…  Environment file exists
βœ…  Environment file exists
πŸ”—  Creating symlinks to config and log files
βœ…  App key exists
πŸ”’  Fixing app path file permissions
πŸ’°  Building the cache...
.+....+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+.+........+.+.....+.+.....+.........+.+..+.+..+.......+...........+.+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+......+...+......+......+..+...+....+...+...+............+..+....+..+.........+...+......+.+.........+..+....+........+.+............+..+......+......+.........+.+..................+..+...............................+...+..+.......+......+......+.........+..+.......+...........+.+...+...+...+..+............+...+.......+..+...+......+.+........+.+..+.........+.+..+.+.....+.......+.........+........+...+...+..........+.....+............+....+........+......+.......+..+.+.........+............+....................+.+.....+...+....+......+..+.......+..+......+.......+..+......+......................+......+.....+.+.....+......+...+....+...+..+..........+..................+..+.+..+.........+......+....+........+...+....+.....+....+............+.....+...+......+...+.......+...+...+.....+............+................+.....+.+......+..+.+.....+.......+..+......+......+...+......+.......+...............+.....+...+......+...+.+..+....+...+...............+........+...+....+............+...........+....+...+...........+.+...+..+....+.....+....+..+.+..+.......+.....+....+.....+.......+...............+.................+......+.+......+..+.......+...+..+......+.......+.....+.......+...+......+............+...+......+.....+...+.+.....+.....................+.+..................+..+............+.+.........+.....+.......+...+.....+...+....+...+.....+......+.[21-Oct-2022 11:56:09] NOTICE: fpm is running, pid 110
[21-Oct-2022 11:56:09] NOTICE: ready to handle connections
..[21-Oct-2022 11:56:09] NOTICE: systemd monitor interval set to 10000ms
......+.+.....+.+........+...+.........+....+..+...+....+...+.....+............................+......+.....+...............+....+..+............+.......+...+..................+..+...+....+........+.+.........+.....................+.....+.......+..+.+...........+.+..+......+....+...+........+.......+.........+...+........+...+.+...+.....+...+..........+...+.....+......+...+...+...+...............+....+..+.............+......+..+....+..............+.......+..+.+.....+.+......+...............+..............+.+.....+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
πŸš›  Migrating the database...
...+.........+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+...+.....+....+.....+...+.......+......+.....+...+....+.....+...+...+....+...+...+.....+......+...+..........+.....+.+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*............+...............................+..+.......+..............+.+...........+.........+.........+.+......+...+..+...+.+.....+...+...+.......+......+.........+........+.............+..+....+.........+.........+...........+.+.....+....+...........+.............+...+...+.....+.......+..+..........+...............+........+...+...............+.......+...+..+...+.........+...+....+.....+....+.....+................+...+........+....+...........+..........+.........+......+...+.....+......+...................+.........+............+...........+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
βœ…  All set, starting Speedtest Tracker container...
πŸ’ͺ  Starting the queue worker...
127.0.0.1 - - [21/Oct/2022:11:56:10 +0200] "GET /ping HTTP/1.1" 301 162 "-" "curl/7.81.0"

Tried connecting to https://docker2.axel.dom:8008 (used https because from what I understand now it's forced in the container), browser response: ERR_SSL_PROTOCOL_ERROR.

Tried https://docker2.axel.dom:8008 response: ERR_CONNECTION_REFUSED

Log of the container after the two connection tests:

10.1.10.45 - - [21/Oct/2022:11:58:34 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03,~\x82@\xF7\xCB\xBB\xB0\xC4\x02F7\xC7\xBB@\x94\x1B\xD6\xAC\xC0G\xE6\xA5\x11\xA9u[Vh0\x91\xC7 \x9Eb\x87\xF3\xA1\x8D\xD6\x0F\x1E\x06\x00\xA6\xE0\x83"400 150 "-" "-"
10.1.10.45 - - [21/Oct/2022:11:58:34 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xEDB\x01\xAB\x82htu\xE6\xB4P\xEAa\xAA\xADH.N\x1FD3!\xA3]\x16\x07\x0C\xB5\xCB\x0E>} w\xD7\xF8\x5C\xEEtqw\xE7u\x03\xE3p\x9D\xA8C\xD3\x19\xC0b\xA9\xF4\xE0\xFF\xA7\xE9\xF9\x97.\xB6\xA1\xBA\x00\x22" 400 150 "-" "-"
10.1.10.45 - - [21/Oct/2022:11:58:34 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xAC\xC3\xA9E5\xC6\xCC,\xE2n\xDA,\x1BP\xBFqlW1\xDFF\xFFni\xC1m\xC4j\x9Em\xE3\xC3 \x81\x1D\x90D\x98C\xC5\x9Cf\xFD\x1C\xB4\xBA\xBBe\xD0j\xD6\xD6\xFE\x9DI\xDA\x85\x89\xAE\xB6K\xFF&\xC1E\x00\x22\xFA\xFA\x13\x01\x13\x02\x13\x02\x13\x03\xC0+\xC0/\xC0,\xC00\xCC\xA9\xCC\xA8\xC0\x13\xC0\x14\x00\x9C\x00\x9D\x00/\x005\x01\x00\x01\x91JJ\x00\x00\x00\x00\x00\x15\x00\x13\x00\x00\x10docker2.axel.dom\x00\x17\x00\x00\xFF\x01\x00\x01\x00\x00" 400 150 "-" "-"
10.1.10.45 - - [21/Oct/2022:11:58:34 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03+\xB0\x04\x11\x5C\xAC\xE9^\xEC\xD7\xF5)=^x\xC3\xA8\xAF\xC0[U\xA9gi\xDA4 !*P\x97M \x00,L\x16\x89I\xBC\xE2\x86\xE3L\xC0f|rV\xB8\x8BP[\xC0T^\x82't\xFA_X\xD0[\x80\x00\x22::\x13\x01\x13\x02\x13\x02\x13\x03\xC0+\xC0/\xC0,\xC00\xCC\xA9\xCC\xA8\xC0\x13\xC0\x14\x00\x9C\x00\x9D\x00/\x005\x01\x00\x01\x91zz\x00\x00\x00\x00\x00\x15\x00\x13\x00\x00\x10docker2.axel.dom\x00\x17\x00\x00\xFF\x01\x00\x01\x00\x00" 400 150 "-" "-"
10.1.10.45 - - [21/Oct/2022:11:58:36 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03H\xC5\x1D\x01K\x8DvI\x1A\xC6\xAE\xB8\xDB\xD1\xEC],\xE85~\x1F9\xD64\x08\xFC\x18\xDC\xDE\xA1\x18\xD2 i\x915y\xE9\x9E\x00Y\xAB\xA7\xA2\xEE\xF1\xE2c]\x03\xF2\xF2\xE1\xF5\x13Cc\xF6\xDF\xF6+\x7F\x90\xD4\xDA\x00\x22ZZ\x13\x01\x13\x02\x13\x02\x13\x03\xC0+\xC0/\xC0,\xC00\xCC\xA9\xCC\xA8\xC0\x13\xC0\x14\x00\x9C\x00\x9D\x00/\x005\x01\x00\x01\x91JJ\x00\x00\x00\x00\x00\x15\x00\x13\x00\x00\x10docker2.axel.dom\x00\x17\x00\x00\xFF\x01\x00\x01\x00\x00" 400 150 "-" "-"
10.1.10.45 - - [21/Oct/2022:11:58:36 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\x1D{N\x5CD\xB0ol\xE8[j\xD2\x87u\x9B\xC9\x0C\x81\xB2f\xE3z5\xE8\xE2\xF9_\xF3\xBB\xE1\xD6\x01 \xE4\x16:hV\xB2\x0Bl\xD3w\x83K\x89\xCAK\xD7\x15\x81x\x93R\x01\xD1}Zn" 400 150 "-" "-"
10.1.10.45 - - [21/Oct/2022:11:58:36 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03w\x1A'\xD1\x1E\xF5|\x09x\xC48\x5C\x8E\xBB\x13\xEB\x8C2\x07" 400 150 "-" "-"
10.1.10.45 - - [21/Oct/2022:11:58:36 +0200] "\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\x0C\xAD\xD4\xABj\xB3Mr\xB2q\xDD\x03\x9A\xB4\xCD\xDD:i(=\x82\x14v" 400 150 "-" "-"
filcuk commented 1 year ago

In browser network log, I'm getting 301 - moved permanently if that helps I have tried to change subdomain to make sure it's not traefik. image

My compose:

  speedtracker:
    image: ghcr.io/alexjustesen/speedtest-tracker:latest
    container_name: speedtracker
    networks:
      - t2_proxy
    volumes:
      - $DOCKERDIR/appdata/speedtracker:/config
    environment:
      - PUID=$PUID
      - PGID=$PGID
    labels:
      # Traefik
      - "traefik.enable=true"
      - "traefik.http.routers.speedtracker-rtr.entrypoints=https"
      - "traefik.http.routers.speedtracker-rtr.rule=Host(`speedtracker.$DOMAINNAME0`)"
      - "traefik.http.routers.speedtracker-rtr.middlewares=chain-authelia@file"
      - "traefik.http.routers.speedtracker-rtr.service=speedtracker-svc"
      - "traefik.http.services.speedtracker-svc.loadbalancer.server.port=80"
alexjustesen commented 1 year ago

@alexdelprete change your port mapping to use the ssl internal port 443 instead of 80

filcuk commented 1 year ago

If you run php artisan about do you see your new APP_URL?

yes

filcuk commented 1 year ago

Using port 443 through Traefik, I'm landing on:

<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx</center>
</body>
</html>

Container logs don't show anything new:

πŸ’ͺ  Starting the queue worker...
127.0.0.1 - - [21/Oct/2022:10:06:56 +0000] "GET /ping HTTP/1.1" 301 162 "-" "curl/7.81.0"
xxx.xxx.xxx.xxx- - [21/Oct/2022:10:07:02 +0000] "GET / HTTP/1.1" 400 248 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0"
xxx.xxx.xxx.xxx- - [21/Oct/2022:10:07:03 +0000] "GET / HTTP/1.1" 400 248 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0"

404 through localhost.

Another thing of note (maybe) is that I'm getting failure in checking the web server intermittently:

πŸ’°  Building the cache...
[21-Oct-2022 10:09:01] NOTICE: fpm is running, pid 108
[21-Oct-2022 10:09:01] NOTICE: ready to handle connections
[21-Oct-2022 10:09:01] NOTICE: systemd monitor interval set to 10000ms
πŸš›  Migrating the database...
❌ There seems to be a failure in checking the web server + PHP-FPM. Here's the response:
βœ…  All set, starting Speedtest Tracker container...
πŸ’ͺ  Starting the queue worker...
127.0.0.1 - - [21/Oct/2022:10:09:02 +0000] "GET /ping HTTP/1.1" 301 162 "-" "curl/7.81.0"

It sometimes happens and sometimes not with the same config.

alexdelprete commented 1 year ago

@alexdelprete change your port mapping to use the ssl internal port 443 instead of 80

Damn...that was so easy to fix, leftover from previous config. :)

Works through traefik, finally:

image

Question: in .env what should APP_URL be? I have APP_URL=http://localhost now. Should I change it?

alexdelprete commented 1 year ago

Using port 443 through Traefik, I'm landing on

I'm using this for Traefik, and it's working:

    speedtest:
      loadBalancer:
        servers:
          - url: "https://docker2.axel.dom:8008"
        passHostHeader: true
alexjustesen commented 1 year ago

@alexdelprete you mind taking a look at @filcuk Traefik issue above and seeing if anything stands out?

nvm lol

alexdelprete commented 1 year ago

@alexdelprete you mind taking a look at @filcuk Traefik issue above and seeing if anything stands out?

Sure, I already answered...;)

Question: how do I start manually the first test?

alexdelprete commented 1 year ago

We need a discord channel for the project...:)

alexjustesen commented 1 year ago

I'm not running Traefik in my homelab so if you have a config you both think is worth sharing I'll add it to the docs.

alexjustesen commented 1 year ago

We need a discord channel for the project...:)

We get past 200 stars, I'll make one.

alexdelprete commented 1 year ago

I'm not running Traefik in my homelab so if you have a config you both think is worth sharing I'll add it to the docs.

It's not worth it, it's a very basic config like any standard http/https service.

I'd add a full docker-compose.yml example to the docs.

alexdelprete commented 1 year ago

@alexjustesen Alex, users could get confused about .env and config.yml: I would clear out the fact that .env variables pertains to the container and can be managed at docker level through environment config, and that config.yml pertains to the app configuration (runtime).

Personally, I prefer to have all env variables in the compose file and not have an .env, but it's subjective obviously. Would be good to explain users that you can configure them in both places. :)

alexjustesen commented 1 year ago

.env is going to remain for all environmental configuration, config.yml is getting removed in an upcoming release and moved to a settings page so everything can be managed from the UI.

filcuk commented 1 year ago

I still can't get it to work, getting bad request no matter what I try. I'm running over 60 services via Traefik and I haven't seen this before, but @alexdelprete is running, so I'm flummoxed. I'll get back to this issue when I figure it out, but thanks for your help so far

alexdelprete commented 1 year ago

still can't get it to work, getting bad request no matter what I try.

Sometimes it's the simple things (like my port in the compose file above).

If you post the compose file and the .env / config.yml file I can try to help.

UPDATE: I saw you posted it, and this looks wrong, shouldn't it be 443?

- "traefik.http.services.speedtracker-svc.loadbalancer.server.port=80"

filcuk commented 1 year ago

Thanks @alexdelprete, though I have updated the port since.

My compose:

  speedtracker:
    image: ghcr.io/alexjustesen/speedtest-tracker:latest
    container_name: speedtracker
    networks:
      - t2_proxy
    volumes:
      - $DOCKERDIR/appdata/speedtracker:/config
    environment:
      - PUID=$PUID
      - PGID=$PGID
    labels:
      # Traefik
      - "traefik.enable=true"
      - "traefik.http.routers.speedtracker-rtr.entrypoints=https"
      - "traefik.http.routers.speedtracker-rtr.rule=Host(`speedtracker.$DOMAINNAME0`)"
      - "traefik.http.routers.speedtracker-rtr.middlewares=chain-authelia@file"
      - "traefik.http.routers.speedtracker-rtr.service=speedtracker-svc"
      - "traefik.http.services.speedtracker-svc.loadbalancer.server.port=443"

The .env is unchanged, though I've tested http://localhost, https://localhost as well as the actual host with rebooting and re-caching in between. I've also removed all files in /config, the container, and re-spun fresh.

I've set up a local-only container with the same result:

  speedtracker2:
    image: ghcr.io/alexjustesen/speedtest-tracker:latest
    container_name: speedtracker2
    restart: unless-stopped
    networks:
      - default
    ports:
      - "4430:443"
    volumes:
      - $DOCKERDIR/appdata/speedtracker-tmp:/config
    environment:
      - PUID=$PUID
      - PGID=$PGID

The only change was from port 80 getting permanent redirect to port 443 getting bad request.

php artisan about:

  Environment ................................................................
  Application Name ......................................... Speedtest Tracker
  Laravel Version ..................................................... 9.36.4
  PHP Version ......................................................... 8.1.11
  Composer Version ..................................................... 2.4.2
  Environment ..................................................... production
  Debug Mode ............................................................. OFF
  URL .............................................................. localhost
  Maintenance Mode ....................................................... OFF

  Cache ......................................................................
  Config .............................................................. CACHED
  Events .......................................................... NOT CACHED
  Routes .............................................................. CACHED
  Views ........................................................... NOT CACHED

  Drivers ....................................................................
  Broadcasting ........................................................... log
  Cache ................................................................. file
  Database ............................................................ sqlite
  Logs ................................................................ stderr
  Mail .................................................................. smtp
  Queue ............................................................. database
  Session ........................................................... database

  Filament ...................................................................
  Packages ................... filament, forms, notifications, support, tables
  Version ........................................................... v2.16.35
  Views ........................................................ NOT PUBLISHED

I'm at a loss

alexdelprete commented 1 year ago

I'm at a loss

First thing I do in these cases, is making sure the docker container is working, bypassing traefik.

So in your case: in your compose file you are missing the ports section. Then in the local container you included it. I assume 4430 is the local port in the compose file, mapped to 443 in the container.

With the browser, if you point at https://docker.domain.dom:4430 what happens? This has to work, because it's not using traefik, you're goind direct to the container.

If this doesn't work, it's a container config issue, if it works, it's traefik config issue.

Let me know...

filcuk commented 1 year ago

I've mentioned above I get the same result when bypassing Traefik, which is the container web server returning 400:

I've set up a local-only container with the same result:

<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx</center>
</body>
</html>

Which would eliminate Traefik as the cause.

Could this from my previous comment be relevant though:

❌ There seems to be a failure in checking the web server + PHP-FPM. Here's the response:

With that said, I'm just testing the new release. We can let this stew and see if more people turn up with the same issue.

alexdelprete commented 1 year ago

Sorry, I didn't understand you made the same test I described. :)

Well, that error means the internal nginx is not redirecting internally to HTTPS. I don't have the error you noticed (check failure).

The local-container must work, traefik has nothing to do with the issue.

Are you sure the local-only variables you used in the compose file are ok? Try with explicit values first, just to debug...

alexjustesen commented 1 year ago

Not a clue if this would be helpful but in the base image they reference traefik and allowing it to direct traffic to a self-signed cert: https://github.com/serversideup/docker-php/tree/dev#the-easiest-way-to-get-a-trusted-certificate

filcuk commented 1 year ago

Sorry, I didn't understand you made the same test I described. :)

No worries!

Not a clue if this would be helpful but in the base image they reference traefik and allowing it to direct traffic to a self-signed cert: https://github.com/serversideup/docker-php/tree/dev#the-easiest-way-to-get-a-trusted-certificate

Great tip, I've included env SSL_MODE=off to let Traefik handle the TLS and changed port back to 80, now it works. May be worth adding to the documentation issue?

alexdelprete commented 1 year ago

Great tip, I've included env SSL_MODE=off to let Traefik handle the TLS and changed port back to 80, now it works. May be worth adding to the documentation issue?

Could you also try with SSL_MODE=mixed please?

This doesn't explain why it's working for me with SSL_MODE=full. Except for one thing: does your traefik config allow SSL with self-signed certificates? Do you have this in the static config?

serversTransport:
  insecureSkipVerify: true
alexdelprete commented 1 year ago

May be worth adding to the documentation issue?

I think this is a specific issue with your setup, because you can't even access the container bypassing traefik.

If Traefik is used, it has to be simply configured to accept self-signed certificates.

filcuk commented 1 year ago

Could you also try with SSL_MODE=mixed please?

Can confirm mixed mode works fine too. I've switched back to default full again to confirm the previous issue and it is present. So mixed and off work fine, full does not.

If Traefik is used, it has to be simply configured to accept self-signed certificates.

I have this insecureSkipVerify currently disabled, I thought it may reduce security, but from what I'm just now reading, that doesn't seem to be the case. Not really sure if there is any reason not to enable it.

alexdelprete commented 1 year ago

So we found your specific problem: basically that setting regulates how Traefik should manage certificates on the backend side. If you set it to true, it will ignore the fact they're self-signed. On the public/frontend side, it will always use the public certificate. If you have it disabled, you won't be able to access backend services with self-signed certs.

The last mistery to solve is that you couldn't access the container directly with the browser with SSL_MODE=full. :)

filcuk commented 1 year ago

The last mistery to solve is that you couldn't access the container directly with the browser with SSL_MODE=full. :)

I just re-spun a local container to test - that was user error, I was trying to access it on http://localip:4430, whereas it needed https

alexdelprete commented 1 year ago

Finally, we solved all the mysteries. :)

So I would suggest to use SSL_MODE=mixed, and users can use 80 or 443 based on their specific setups. If behind a reverse-proxy, 443 must be used because of #34, and the reverse-proxy has to be configured to accept self-signed certificates.

Case closed. :)

Daniel-Kalus commented 1 year ago

I'm going to add to this because I just spent hours debugging this issue, trying everything posted here but nothing worked. What finally worked was adding the label - traefik.http.services.speedtest.loadbalancer.server.scheme=https to the speedtest container in addition to - --serversTransport.insecureSkipVerify=true in the traefik container as found here and now it finally works! This worked for me with SSL_MODE off, mixed or full (or commented out aka the default .env):

traefik:
  command:
    - --log.level=INFO # log levels are DEBUG, PANIC, FATAL, ERROR, WARN, and INFO.
    - --api.insecure=true
    - --providers.docker=true
    - --providers.docker.exposedbydefault=false
    - --entrypoints.web.address=:80
    - --entrypoints.websecure.address=:443
    - --entrypoints.web.http.redirections.entrypoint.to=websecure
    - --certificatesresolvers.letsencrypt.acme.tlschallenge=true
    - --certificatesresolvers.letsencrypt.acme.email=***
    - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
    - --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web
    - --serversTransport.insecureSkipVerify=true

speetest:
  labels:
    - traefik.enable=true
    - traefik.http.routers.speedtest.rule=Host(`speedtest.${DOMAINNAME}`)
    - traefik.http.routers.speedtest.tls=true
    - traefik.http.routers.speedtest.tls.certresolver=letsencrypt
    - traefik.http.services.speedtest.loadbalancer.server.port=443
    - traefik.http.services.speedtest.loadbalancer.server.scheme=https
alexdelprete commented 1 year ago

What finally worked was adding the label - traefik.http.services.speedtest.loadbalancer.server.scheme=https to the speedtest container

This tells Traefik to access the container using https schema. So that implies that if you configure SSL_MODE=off in speedtest-tracker, it wouldn't work, because you have turned off HTTPS access in the container, and Traefik wouldn't be able to access it.

in addition to - --serversTransport.insecureSkipVerify=true in the traefik container as found here

This was recommended (in static config format, not labels, but it's the same setting) six posts above yours: https://github.com/alexjustesen/speedtest-tracker/issues/54#issuecomment-1287112077 and this is true for every service that uses self-signed certificates, not specific to ST.

What that person says in that post of 2019 is obvious (and confusing to traefik newbies):

I have this in my static config:

serversTransport:
  # Accept self-signed certificates for backend services
  insecureSkipVerify: true

And my service definition for speedtest-tracker (configured with SSL_MODE=full) is this:

speedtest:
  loadBalancer:
    servers:
      - url: "https://docker2.domain.name:8008"
    passHostHeader: true

If I set SSL_MODE=off in ST, Traefik can't access the service anymore, so when you say that your config works for any SSL_MODE that can't be correct.

Since you use traefik labels at the service container level, you need to add this:

- traefik.http.services.speedtest.loadbalancer.server.scheme=https

only when you want Traefik to access that specific service via https, and that depends on how you configure the service, in this case Speedtest-Tracker.

If I set SSL_MODE=off and so I configure Traefik to use httpinstead of httpsfor the service definition:

speedtest:
  loadBalancer:
    servers:
      - url: "http://docker2.domain.name:8008"
    passHostHeader: true

Traefik can still access ST, but when browsing ST, the browser complains because it's receiving mixed-content (http urls mixed with https urls). So I advised @alexjustesen to use as a default SSL_MODE configuration MIXED, this way ST by default accepts both HTTP and HTTPS connections, and the user can adapt things based on his specific setup.

Daniel-Kalus commented 1 year ago

If I set SSL_MODE=off in ST, Traefik can't access the service anymore, so when you say that your config works for any SSL_MODE that can't be correct.

I swear to you that it works with SSL_MODE=off: Screenshot 2022-12-03 042908

alexdelprete commented 1 year ago

Daniel,

In your config below you're telling Traefik to access the container via port 443, with https. (btw, you have a spelling error in the name of the container, you missed a D).

speetest:
  labels:
    - traefik.enable=true
    - traefik.http.routers.speedtest.rule=Host(`speedtest.${DOMAINNAME}`)
    - traefik.http.routers.speedtest.tls=true
    - traefik.http.routers.speedtest.tls.certresolver=letsencrypt
    - traefik.http.services.speedtest.loadbalancer.server.port=443
    - traefik.http.services.speedtest.loadbalancer.server.scheme=https

with SSL_MODE=off the nginx configuration of speedtest-tracker does not even map/open port 443, so tell me: how does Traefik connect to a non-existing port? A miracle...:)

Try to access the container directly, bypassing Traefik, using this url (replace the domain): https://speedtest.yourlocaldomain.dom

If this local test works, it means SSL_MODE=off is not in effect, that's why Traefik accesses it.

Another test: restart speedtest-tracker and then check docker log of the container, in the startup phase it tells you the SSL_MODE setting, use this command: docker logs speedtest-tracker

This is my log, the first line after the userid tells you how SSL_MODE is configured.

--------------------------------------------------------------------
 ____                             ____  _     _        _   _
/ ___|  ___ _ ____   _____ _ __  / ___|(_) __| | ___  | | | |_ __
\___ \ / _ \  __\ \ / / _ \  __| \___ \| |/ _` |/ _ \ | | | |  _ \
 ___) |  __/ |   \ V /  __/ |     ___) | | (_| |  __/ | |_| | |_) |
|____/ \___|_|    \_/ \___|_|    |____/|_|\__,_|\___|  \___/| .__/
                                                            |_|
Brought to you by serversideup.net
--------------------------------------------------------------------
To support Server Side Up projects visit:
https://serversideup.net/sponsor
-------------------------------------
GID/UID
-------------------------------------
User uid:    1000
User gid:    1000
-------------------------------------
πŸ” **SSL_MODE has set to FULL, setting the web server to work in HTTPS only...**
πŸƒβ€β™‚οΈ Checking for Laravel automations...
πŸ” Linking the storage...
   INFO  The [public/storage] link has been connected to [storage/app/public].
πŸ‡  Configuring Speedtest Tracker...
βœ…  Environment file exists
πŸ”—  Creating symlinks to config and log files
βœ…  App key exists
πŸ”’  Fixing app path file permissions
πŸ’°  Building the cache...
[03-Dec-2022 04:43:07] NOTICE: fpm is running, pid 105
[03-Dec-2022 04:43:07] NOTICE: ready to handle connections
[03-Dec-2022 04:43:07] NOTICE: systemd monitor interval set to 10000ms
πŸš›  Migrating the database...
βœ…  All set, starting Speedtest Tracker container...
⏰  Starting the cron service...
πŸ’ͺ  Starting the queue worker...
127.0.0.1 - - [03/Dec/2022:04:43:08 +0100] "GET /ping HTTP/1.1" 301 162 "-" "curl/7.81.0"
Daniel-Kalus commented 1 year ago

You're right, the SSL_MODE was still set to FULL, in fact none of the variables from the .env under /config were set inside the container (checked with echo). I copied the .env to my project root so it gets read by docker-compose and now the variables are being set. Now my configuration works only with SSL_MODE full or mixed. With SSL_MODE=off I get "Bad Gateway". I still need - traefik.http.services.speedtest.loadbalancer.server.scheme=https otherwise I get the error 400 "plain HTTP send to HTPPS", I guess that's because I set up traefik to redirect all http request to the https entrypoint.

Is the /config/.env supposed to be read on container startup or do we have to copy it to the host so it gets read by docker-compose?

alexjustesen commented 1 year ago

Heads up, v0.5.0 changed the SSL_MODE to "mixed" so you can now reference ports 80 and 443 when mapping to the container.

alexdelprete commented 1 year ago

Is the /config/.env supposed to be read on container startup or do we have to copy it to the host so it gets read by docker-compose?

In general, you can use env variables in the docker compose environment section directly.

But in this case, there's something not clear on your setup: .env file is read by the app in the /config folder, so you need to map a docker volume to that. That's where the .env file is created and then you can edit it.

Did you use the example compose file from the docs? Show me your full docker-compose file, I think you have some issues there.

alexdelprete commented 1 year ago

Heads up, v0.5.0 changed the SSL_MODE to "mixed" so you can now reference ports 80 and 443 when mapping to the container.

Wise choice...;)

Daniel-Kalus commented 1 year ago

This is my compose file, none of the variables in the .env file under ${DATADIR}/speedtest/app are set inside the container.

version: '3.3'

services:
  speedtest:
    image: 'ghcr.io/alexjustesen/speedtest-tracker:latest'
    container_name: speedtest
    restart: unless-stopped
    networks: ["traefik"]
    depends_on: ["speedtest-db"]
    ports:
      - ${SPEEDTEST_PORT}:443
    environment:
      TZ: "${TIMEZONE}"
      PUID: "1000"
      PGID: "1000"
      DB_CONNECTION: "mysql"
      DB_HOST: "speedtest-db"
      DB_PORT: "3306"
      DB_DATABASE: "speedtest_tracker"
      DB_USERNAME: "speedy"
      DB_PASSWORD: "password"
    volumes:
      - ${DATADIR}/speedtest/app:/config
    labels:
      traefik.enable: "true"
      traefik.http.routers.speedtest.rule: "Host(`speedtest.${DOMAINNAME}`)"
      traefik.http.routers.speedtest.tls: "true"
      traefik.http.routers.speedtest.tls.certresolver: "letsencrypt"
      traefik.http.services.speedtest.loadbalancer.server.port: "443"
      traefik.http.services.speedtest.loadbalancer.server.scheme: "https"

  speedtest-db:
    image: mariadb:10
    container_name: speedtest-db
    restart: unless-stopped
    networks: ["traefik"]
    environment:
      MARIADB_DATABASE: "speedtest_tracker"
      MARIADB_USER: "speedy"
      MARIADB_PASSWORD: "password"
      MARIADB_RANDOM_ROOT_PASSWORD: "true"
    volumes:
      - ${DATADIR}/speedtest/db:/var/lib/mysql

networks:
  traefik:
    name: traefik
    external: true
alexdelprete commented 1 year ago

${DATADIR}/speedtest/app

You have the .env file in that folder, on the docker host side? If you edit it and restart ST, it doesn't read the changes?

Anyway, I'm usng env variables in the environment section of the compose file, SSL_MODE included, and it works perfectly, actually I prefer it too, without editing another file. I prefer having as much as possible in docker-compose, in general.

alexdelprete commented 1 year ago

@alexjustesen I confirm what Daniel says: is the .env file under /config read by the app at startup or not? I tried modifying some variables and restarted the container but it didn't pick-up the changes.

Variables set in docker-compose work fine, and actually I do prefer to use that to modify the config.

alexjustesen commented 1 year ago

It is but it's cached at the moment, if you change the env vars I suggest running php artisan optimize to reset the cached config vars

alexdelprete commented 1 year ago

It is but it's cached at the moment

so on restart it doesn't read it because cache has priority? does the cache expire?

luckily I prefer docker-compose env vars...