[Solved] Problem when using Docker and Nginx as reverse proxy

[mehrab-wj]

Hi, I've set up my project based on docker and docker-compose, I've exposed the port 6001, which is the port for running the WebSocket as default, and configured Nginx based on the documentation, but when I try to connect to my WS server, I get this error:

upstream prematurely closed connection while reading response header from upstream, client: *, server: site.cpm, request: "GET /app/7f0cb504a1426efd6854a03779a1c8a9?protocol=7&client=js&version=7.0.6&flash=false HTTP/1.1", upstream: "http://172.19.0.5:3030/app/7f0cb504a1426efd6854a03779a1c8a9?protocol=7&client=js&version=7.0.6&flash=false", host: "site.com"

Also, another weird problem is now I can't even connect to my WS server using direct IP and port.

Nginx config file:

map $http_upgrade $type {
  default "web";
  websocket "ws";
}

upstream websocket {
    server php:6001;
}

server {
    listen 80;
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    ssl_certificate /app/docker/nginx/certs/public-selfsigned.crt;
    ssl_certificate_key /app/docker/nginx/certs/private-selfsigned.key;

    ssl_protocols TLSv1.2 TLSv1.1 TLSv1;

    server_name panel.melodigram.app;

    client_max_body_size 2048M;

    index index.php index.html;
    error_log  /var/log/nginx/fpm.log;
    access_log /var/log/nginx/access.log;
    root /app/public_html;

    location / {
        try_files /nonexistent @$type;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass php:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
    location @web {
        try_files $uri $uri/ /index.php?$query_string;
        gzip_static on;
    }

    location @ws {
        proxy_pass http://websocket;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_read_timeout     60;
        proxy_connect_timeout  60;
        proxy_redirect         off;
    }
}

Broadcasting config: broadcasting.php

<?php

return [
    'connections' => [

        'pusher' => [
            'driver' => 'pusher',
            'key' => env('PUSHER_APP_KEY'),
            'secret' => env('PUSHER_APP_SECRET'),
            'app_id' => env('PUSHER_APP_ID'),
            'options' => [
                'cluster' => env('PUSHER_APP_CLUSTER'),
                'encrypted' => true,
                'host' => '0.0.0.0',
                'port' => 6001,
                'scheme' => 'http',
                'curl_options' => [
                    'CURLOPT_SSL_VERIFYHOST' => 0,
                    'CURLOPT_SSL_VERIFYPEER' => 0,
                ],
            ],
        ],
        // ...
    ],
];

Websocet config: websockets.php

<?php

use BeyondCode\LaravelWebSockets\Dashboard\Http\Middleware\Authorize;

return [

    /*
     * Set a custom dashboard configuration
     */
    'dashboard' => [
        'port' => env('LARAVEL_WEBSOCKETS_PORT', 6001),
    ],

    /*
     * This package comes with multi tenancy out of the box. Here you can
     * configure the different apps that can use the webSockets server.
     *
     * Optionally you specify capacity so you can limit the maximum
     * concurrent connections for a specific app.
     *
     * Optionally you can disable client events so clients cannot send
     * messages to each other via the webSockets.
     */
    'apps' => [
        [
            'id' => env('PUSHER_APP_ID'),
            'name' => env('APP_NAME'),
            'key' => env('PUSHER_APP_KEY'),
            'secret' => env('PUSHER_APP_SECRET'),
            'path' => env('PUSHER_APP_PATH'),
            'capacity' => null,
            'enable_client_messages' => false,
            'enable_statistics' => true,
        ],
    ],

    /*
     * This class is responsible for finding the apps. The default provider
     * will use the apps defined in this config file.
     *
     * You can create a custom provider by implementing the
     * `AppProvider` interface.
     */
    'app_provider' => BeyondCode\LaravelWebSockets\Apps\ConfigAppProvider::class,

    /*
     * This array contains the hosts of which you want to allow incoming requests.
     * Leave this empty if you want to accept requests from all hosts.
     */
    'allowed_origins' => [
        //
    ],

    /*
     * The maximum request size in kilobytes that is allowed for an incoming WebSocket request.
     */
    'max_request_size_in_kb' => 250,

    /*
     * This path will be used to register the necessary routes for the package.
     */
    'path' => 'laravel-websockets',

    /*
     * Dashboard Routes Middleware
     *
     * These middleware will be assigned to every dashboard route, giving you
     * the chance to add your own middleware to this list or change any of
     * the existing middleware. Or, you can simply stick with this list.
     */
    'middleware' => [
        'web',
        Authorize::class,
    ],

    'statistics' => [
        /*
         * This model will be used to store the statistics of the WebSocketsServer.
         * The only requirement is that the model should extend
         * `WebSocketsStatisticsEntry` provided by this package.
         */
        'model' => \BeyondCode\LaravelWebSockets\Statistics\Models\WebSocketsStatisticsEntry::class,

        /**
         * The Statistics Logger will, by default, handle the incoming statistics, store them
         * and then release them into the database on each interval defined below.
         */
        'logger' => BeyondCode\LaravelWebSockets\Statistics\Logger\HttpStatisticsLogger::class,

        /*
         * Here you can specify the interval in seconds at which statistics should be logged.
         */
        'interval_in_seconds' => 60,

        /*
         * When the clean-command is executed, all recorded statistics older than
         * the number of days specified here will be deleted.
         */
        'delete_statistics_older_than_days' => 60,

        /*
         * Use an DNS resolver to make the requests to the statistics logger
         * default is to resolve everything to 127.0.0.1.
         */
        'perform_dns_lookup' => false,
    ],

    /*
     * Define the optional SSL context for your WebSocket connections.
     * You can see all available options at: http://php.net/manual/en/context.ssl.php
     */
    'ssl' => [
        /*
         * Path to local certificate file on filesystem. It must be a PEM encoded file which
         * contains your certificate and private key. It can optionally contain the
         * certificate chain of issuers. The private key also may be contained
         * in a separate file specified by local_pk.
         */
        'local_cert' => env('LARAVEL_WEBSOCKETS_SSL_LOCAL_CERT', null),

        /*
         * Path to local private key file on filesystem in case of separate files for
         * certificate (local_cert) and private key.
         */
        'local_pk' => env('LARAVEL_WEBSOCKETS_SSL_LOCAL_PK', null),

        /*
         * Passphrase for your local_cert file.
         */
        'passphrase' => env('LARAVEL_WEBSOCKETS_SSL_PASSPHRASE', null),
    ],

    /*
     * Channel Manager
     * This class handles how channel persistence is handled.
     * By default, persistence is stored in an array by the running webserver.
     * The only requirement is that the class should implement
     * `ChannelManager` interface provided by this package.
     */
    'channel_manager' => \BeyondCode\LaravelWebSockets\WebSockets\Channels\ChannelManagers\ArrayChannelManager::class,
];

Docker compose file: docker-compose.yml

version: '3'

services:
  nginx:
    build: ./docker/nginx
    restart: always
    command: ['nginx-debug', '-g', 'daemon off;']
    volumes:
      - ./:/app/
      - ./docker/nginx/nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - 80:80
      - 443:443
    depends_on:
      - php
    networks:
      - internal
  php:
    build: ./
    image: zagreus/melodi
    container_name: zagreus-melodi
    restart: always
    working_dir: /app
    command: [ "bash", "/app/docker/initialize.sh" ]
    ports:
      - 6001:6001
    volumes:
      - ./:/app/
      - ./docker/php/supervisord.conf:/etc/supervisor/conf.d/supervisord.conf
    depends_on:
      - database
      - redis
    networks:
      - internal
  database:
    image: mariadb
    container_name: melodi-database
    restart: always
    environment:
      - MARIADB_RANDOM_ROOT_PASSWORD=yes
      - MARIADB_DATABASE=${DB_DATABASE:?DB_DATABASE not entered}
      - MARIADB_USER=${DB_USERNAME:?DB_USERNAME not entered}
      - MARIADB_PASSWORD=${DB_PASSWORD?DB_PASSWORD not entered}
    volumes:
      - ${DB_PERSIST_PATH:?DB_PERSIST_PATH not entered}:/var/lib/mysql
    networks:
      - internal
    # ports:
    #   - 3306:3306
  phpmyadmin:
    image: phpmyadmin
    container_name: melodi-phpmyadmin
    restart: always
    depends_on:
      - database
    ports:
      - 8080:80
    networks:
      - internal
    environment:
      - PMA_HOST=database
      - MYSQL_ROOT_PASSWORD=${DB_PASSWORD:-password}
      - PMA_ARBITRARY=1
      - UPLOAD_LIMIT=2G
  redis:
    image: redis:7.0.0-bullseye
    container_name: melodi-redis
    restart: always
    networks:
      - internal
networks:
  internal:
      driver: bridge

[mehrab-wj]

Ok, so I found out that the problem was even though I was using Nginx as a reverse proxy and my Nginx was handling SSL by itself, I've also filled LARAVEL_WEBSOCKETS_SSL_LOCAL_CERT and LARAVEL_WEBSOCKETS_SSL_LOCAL_PK in my environment file that causes the problem.

So if you are using Nginx as a reverse proxy, after making sure that the port is open on your container, just let Nginx handle the SSL and don't configure it again on this package.

[ssheduardo]

Ok, so I found out that the problem was even though I was using Nginx as a reverse proxy and my Nginx was handling SSL by itself, I've also filled LARAVEL_WEBSOCKETS_SSL_LOCAL_CERT and LARAVEL_WEBSOCKETS_SSL_LOCAL_PK in my environment file that causes the problem.

So if you are using Nginx as a reverse proxy, after making sure that the port is open on your container, just let Nginx handle the SSL and don't configure it again on this package.

Hi, can you add the part where you have solved it, what file you touched or where you placed the certificate, I have had the same problem using nginx-proxy (https://hub.docker.com/r/jwilder/nginx-proxy). I also use docker compose. Thanks

[mehrab-wj]

Hi @ssheduardo, I don't know what is your problem exactly, but what I've done was use the Nginx image and configure the reverse proxy by myself. Then I used CDN so the CDN would handle the SSL and I just have to enable SSL in my Nginx by just creating a self-signed certificate. when you do configure your Nginx correctly, no changes are needed in the package and you could connect to it through your reverse proxy.

[ssheduardo]

Hi @ssheduardo, I don't know what is your problem exactly, but what I've done was use the Nginx image and configure the reverse proxy by myself. Then I used CDN so the CDN would handle the SSL and I just have to enable SSL in my Nginx by just creating a self-signed certificate. when you do configure your Nginx correctly, no changes are needed in the package and you could connect to it through your reverse proxy.

This is my issues: https://github.com/beyondcode/laravel-websockets/issues/986

[RyanSacks]

Hi @ssheduardo, I don't know what is your problem exactly, but what I've done was use the Nginx image and configure the reverse proxy by myself. Then I used CDN so the CDN would handle the SSL and I just have to enable SSL in my Nginx by just creating a self-signed certificate. when you do configure your Nginx correctly, no changes are needed in the package and you could connect to it through your reverse proxy.

You didn't create a Docker container for websockets? I don't see it in your docker-compose.yml file.