search

dunglas / mercure

🪽 An open, easy, fast, reliable and battery-efficient solution for real-time communications
https://mercure.rocks
GNU Affero General Public License v3.0
3.98k stars 297 forks source link

Help for configuring Mercure behind nginx as proxy-server #854

Open Astro-Otter-Space opened 10 months ago

Astro-Otter-Space commented 10 months ago

Hello,

I'm deploying Mercure hub on my server (VPS hosted by OVH) where an API REST (Symfony+FOSRestBundle) and a front app (VueJS 3, vue-cli) are installed yet, served with Nginx. API and FO have their own domains (https://api.exemple.space/ and https://www.exemple.space/). I've created a domain for Mercure (https://mercure.exemple.com/). Mercure (version 0.15) is installed with binary, not with docker.

I added an nginx host configuration for working as reverse-proxy (https://mercure.rocks/docs/hub/nginx):

# should i keep these lines commented or not ?
#server {
#  listen      80 http2;
#  server_name mercure.exemple.space;
#  return 301 https://mercure.exemple.space;
#}

server {
  listen      443 ssl http2;
  listen [::]:443 ssl http2;
  server_name mercure.exemple.space;

  ssl_certificate /etc/letsencrypt/live/mercure.exemple.space/fullchain.pem; 
  ssl_certificate_key /etc/letsencrypt/live/mercure.exemple.space/privkey.pem;

  location / {
    proxy_pass http://127.0.0.1:3000; # <-- is correct ?
    proxy_read_timeout 24h;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_connect_timeout 300s;

    #proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
  }

  # Configuration des logs
  access_log  /var/log/nginx/mercure_access.log;
  error_log   /var/log/nginx/mercure_error.log;
}

i set public and private keys and set them as env variables in /etc/profile.d/mercure.sh

export MERCURE_PUBLISHER_JWT_KEY=$(cat /home/me/mercure/publisher.key.pub)
export MERCURE_PUBLISHER_JWT_ALG=RS256
export MERCURE_SUBSCRIBER_JWT_KEY=$(cat /home/me/mercure/subscriber.key.pub)
export MERCURE_PUBLISHER_JWT_ALG=RS256
export SERVER_NAME=localhost:3000 # <--  mercure.exemple.space or mercure.exemple.space:3000, which one is correct ?

I run with command MERCURE_PUBLISHER_JWT_KEY=$MERCURE_PUBLISHER_JWT_KEY MERCURE_SUBSCRIBER_JWT_KEY=$MERCURE_SUBSCRIBER_JWT_KEY /usr/bin/mercure run --config /home/stephane/mercure/Caddyfile

My Caddyfile : 

{
    {$DEBUG:debug}

    {$CADDY_GLOBAL_OPTIONS}
    order mercure after encode

    # Ports
    http_port 3001 # Should i keep this line ?
    https_port 3000 # Should i keep this line ?
    {$GLOBAL_OPTIONS}
}

{$CADDY_EXTRA_CONFIG}

{$SERVER_NAME:localhost} {
        # tls line is commented because Mercure cant read keys, change rights/owner ?
    # tls /etc/letsencrypt/live/mercure.exemple.space/fullchain.pem /etc/letsencrypt/live/mercure.exemple.space/privkey.pem

    log {
        format filter {
            wrap console
            fields {
                uri query {
                    replace authorization REDACTED
                }
            }
        }
    }

    encode zstd gzip

    mercure {
        # Transport to use (default to Bolt)
        transport_url {$MERCURE_TRANSPORT_URL:bolt://mercure.db}
        # Publisher JWT key
        publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
        # Subscriber JWT key
        subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
        # Extra directives
        # CORS
        cors_origins https://www.exemple.space https://exemple.local:8080 https://localhost:8080 <-- maybe some error here ?
        publish_origins *
        anonymous
        subscriptions
        {$MERCURE_EXTRA_DIRECTIVES}
    }

    {$CADDY_SERVER_EXTRA_DIRECTIVES}

    respond /healthz 200
    respond "Not Found" 404
}

In log i have : 

2024/01/03 11:07:15.053 ERROR   tls.obtain  could not get certificate from issuer   {"identifier": "mercure.exemple.space", "issuer": "acme.zerossl.com-v2-DV90", "error": "[mercure.astro-otter.space] creating new order: attempt 1: https://acme.zerossl.com/v2/DV90/newOrder: performing request: Post \"https://acme.zerossl.com/v2/DV90/newOrder\": context deadline exceeded (Client.Timeout exceeded while awaiting headers) (ca=https://acme.zerossl.com/v2/DV90)"}
2024/01/03 11:07:15.053 DEBUG   events  event   {"name": "cert_failed", "id": "533a237d-c07a-4cd4-a12d-dfe931a9c5aa", "origin": "tls", "data": {"error":{},"identifier":"mercure.exemple.space","issuers":["acme-v02.api.letsencrypt.org-directory","acme.zerossl.com-v2-DV90"],"renewal":false}}

2024/01/03 11:07:15.053 ERROR   tls.obtain  will retry  {"error": "[mercure.exemple.space] Obtain: [mercure.exemple.space] creating new order: attempt 1: https://acme.zerossl.com/v2/DV90/newOrder: performing request: Post \"https://acme.zerossl.com/v2/DV90/newOrder\": context deadline exceeded (Client.Timeout exceeded while awaiting headers) (ca=https://acme.zerossl.com/v2/DV90)", "attempt": 1, "retrying_in": 60, "elapsed": 127.939511407, "max_duration": 2592000}

If i run curl from my local env : 

$ curl -X GET https://mercure.exemple.space/.well-known/mercure
Client sent an HTTP request to an HTTPS server.
$ curl -X GET https://mercure.exemple.space:3000/.well-known/mercure
curl: (35) error:0A000438:SSL routines::tlsv1 alert internal error

In my JS app (from local or prod env), i have CORS error with this code : 

  const hubUrl = new URL('https://mercure.exemple.space');
  const domain = 'https://api.exemple.space'
  const topic = 'notifications/all';
  hubUrl.searchParams.append('topic', `${domain}/${topic}`);
  const eventSource = new EventSource(hubUrl.toString(), {withCredentials: true});
  eventSource.onmessage = (event) => {
    console.log(event.data);
  }

I need help for wrtting good configuration for Nginx and Caddyfile. I'll fix CORS errors later. Thank you for help :)

dunglas commented 10 months ago

Try to disable TLS on Mercure. For instance, set SERVER_NAME to http://localhost (notice the http:// prefix).

Astro-Otter-Space commented 10 months ago

ok I'll do that. I changed SERVER_NAME value in /etc/environment

$ echo $SERVER_NAME 
http://localhost

and changed Caddyfile like this : 

{
    {$DEBUG:debug}

    {$CADDY_GLOBAL_OPTIONS}
    order mercure after encode

    # Ports
    http_port 3000
    auto_https off
    {$GLOBAL_OPTIONS}
}

{$CADDY_EXTRA_CONFIG}

{$SERVER_NAME:localhost} {
    # tls /etc/letsencrypt/live/mercure.exemple.space/fullchain.pem /etc/letsencrypt/live/mercure.exemple.space/privkey.pem

line tls ... commented and disabled like i saw here or here Is it enough ?

I have no more error in console (a good point ^^).

$ MERCURE_PUBLISHER_JWT_KEY=$MERCURE_PUBLISHER_JWT_KEY \
MERCURE_SUBSCRIBER_JWT_KEY=$MERCURE_SUBSCRIBER_JWT_KEY \
SERVER_NAME=$SERVER_NAME \
DEBUG=debug \
/usr/bin/mercure run --config Caddyfile
2024/01/04 08:32:28.347 INFO    using provided configuration    {"config_file": "Caddyfile", "config_adapter": ""}
2024/01/04 08:32:28.350 WARN    Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies    {"adapter": "caddyfile", "file": "Caddyfile", "line": 2}
2024/01/04 08:32:28.351 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2024/01/04 08:32:28.352 WARN    http.auto_https automatic HTTPS is completely disabled for server   {"server_name": "srv0"}
2024/01/04 08:32:28.352 DEBUG   http.auto_https adjusted config {"tls": {"automation":{"policies":[{}]}}, "http": {"http_port":3000,"servers":{"srv0":{"listen":[":3000"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"headers","response":{"set":{"Content-Type":["text/html; charset=utf-8"]}}}],"match":[{"path":["/"]}]},{"handle":[{"encodings":{"gzip":{},"zstd":{}},"handler":"encode","prefer":["zstd","gzip"]},{"anonymous":true,"cors_origins":["https://news.exemple.space","https://exemple.local:8080","https://localhost:8080"],"handler":"mercure","publish_origins":["*"],"publisher_jwt":{"alg":"{env.MERCURE_PUBLISHER_JWT_ALG}","key":"{env.MERCURE_PUBLISHER_JWT_KEY}"},"subscriber_jwt":{"alg":"{env.MERCURE_SUBSCRIBER_JWT_ALG}","key":"{env.MERCURE_SUBSCRIBER_JWT_KEY}"},"subscriptions":true,"transport_url":"bolt://mercure.db"}]},{"handle":[{"handler":"static_response","status_code":200}],"match":[{"path":["/healthz"]}]},{"handle":[{"body":"\u003c!DOCTYPE html\u003e\n\t\u003chtml lang=en\u003e\n\t\u003cmeta charset=\"utf-8\"\u003e\n\t\u003cmeta name=\"robots\" content=\"noindex\"\u003e\n\t\u003ctitle\u003eWelcome to Mercure\u003c/title\u003e\n\t\u003ch1\u003eWelcome to Mercure\u003c/h1\u003e\n\t\u003cp\u003eThe URL of your hub is \u003ccode\u003e/.well-known/mercure\u003c/code\u003e.\n\tRead the documentation on \u003ca href=\"https://mercure.rocks\"\u003eMercure.rocks, real-time apps made easy\u003c/a\u003e.","handler":"static_response"}],"match":[{"path":["/"]}]},{"handle":[{"body":"Not Found","handler":"static_response","status_code":404}]}]}],"terminal":true}],"automatic_https":{"disable":true},"logs":{"logger_names":{"localhost":"log0"}}}}}}
2024/01/04 08:32:28.354 DEBUG   http    starting server loop    {"address": "[::]:3000", "tls": false, "http3": false}
2024/01/04 08:32:28.355 INFO    http.log    server running  {"name": "srv0", "protocols": ["h1", "h2", "h3"]}
2024/01/04 08:32:28.355 INFO    autosaved config (load with --resume flag)  {"file": "/home/stephane/.config/caddy/autosave.json"}
2024/01/04 08:32:28.355 INFO    serving initial configuration
2024/01/04 08:32:28.357 INFO    tls.cache.maintenance   started background certificate maintenance  {"cache": "0xc0004c0000"}
2024/01/04 08:32:28.359 WARN    tls storage cleaning happened too recently; skipping for now    {"storage": "FileStorage:/home/stephane/.local/share/caddy", "instance": "359fed6e-f64a-4f98-a8cb-e2b7bb24d40c", "try_again": "2024/01/05 08:32:28.359", "try_again_in": 86399.999999583}
2024/01/04 08:32:28.359 INFO    tls finished cleaning storage units

And i saw my request in log : 

2024/01/04 08:37:37.796 INFO    http.log.access handled request {"request": {"remote_ip": "127.0.0.1", "remote_port": "36484", "client_ip": "127.0.0.1", "proto": "HTTP/1.1", "method": "GET", "host": "127.0.0.1:3000", "uri": "/.well-known/mercure?topic=https%3A%2F%2Fapi.exemple.space%2Fnotifications%2Fdso", "headers": {"Sec-Fetch-Site": ["cross-site"], "Sec-Fetch-Mode": ["cors"], "Referer": ["https://localhost:8080/"], "Accept-Encoding": ["gzip, deflate, br"], "X-Forwarded-For": ["185.101.209.57"], "Cache-Control": ["no-cache"], "Sec-Ch-Ua-Mobile": ["?0"], "User-Agent": ["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"], "Accept-Language": ["fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7"], "Sec-Ch-Ua-Platform": ["\"Linux\""], "Sec-Fetch-Dest": ["empty"], "X-Forwarded-Host": ["mercure.exemple.space"], "X-Forwarded-Proto": ["https"], "Sec-Ch-Ua": ["\"Not_A Brand\";v=\"8\", \"Chromium\";v=\"120\", \"Google Chrome\";v=\"120\""], "Accept": ["text/event-stream"], "Origin": ["https://localhost:8080"]}}, "bytes_read": 0, "user_id": "", "duration": 0.000004338, "size": 0, "status": 0, "resp_headers": {"Server": ["Caddy"]}}

If i curl from local : 

curl -k -I -X GET https://mercure.exemple.space/.well-known/mercure
HTTP/2 200 
server: nginx
date: Thu, 04 Jan 2024 08:33:39 GMT
content-length: 0

JS side, i got 200 too but CORS error

Access to resource at 'https://mercure.exemple.space/.well-known/mercure?topic=https%3A%2F%2Fapi.astro-otter.space%2Fnotifications%2Fall' from origin 'https://localhost:8080' has been blocked by CORS policy: The value of the 'Access-Control-Allow-Credentials' header in the response is '' which must be 'true' when the request's credentials mode is 'include'.
mercure:1 

       GET https://mercure.exemple.space/.well-known/mercure?topic=https%3A%2F%2Fapi.exemple.space%2Fnotifications%2Fall net::ERR_FAILED 200 (OK)
home:1 EventSource's response has a MIME type ("text/plain") that is not "text/event-stream". Aborting the connection.

It seems it's on a good way ^^

Astro-Otter-Space commented 10 months ago

I added in my nginx vhost these lines : 

  location / {
    if ($http_origin ~ '^https?://(localhost(:[0-9]+)?|[^/]*\.exemple\.space)') {
        set $cors "true";
    }    

    if ($cors = "true") {
      add_header 'Access-Control-Allow-Origin' '$http_origin' always;    
      add_header 'Access-Control-Allow-Credentials' 'true';
      add_header 'Access-Control-Allow-Headers' 'Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
      add_header 'Access-Control-Allow-Methods' 'GET,POST,OPTIONS';
      add_header "Content-Type" "text/event-stream";
    }       

    proxy_pass http://127.0.0.1:3000;
    proxy_read_timeout 24h;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_connect_timeout 300s;

    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
  }

And no more CORS errors.

dunglas commented 10 months ago

This shouldn't be necessary but thanks for the workaround! I'll take a look.

Astro-Otter-Space commented 10 months ago

Without Access-Control-Allow-Origin, Access-Control-Allow-Credentials and Content-Type, i have CORS error JS-side.

With this nginx configuration, what should be values of cors_origins and publish_origins in my Caddyfile ? You told me to set http://localhost for env variable SERVER_NAME. Should i set proxy_pass http://localhost:3000; instead of proxy_pass http://127.0.0.1:3000; ?

UPDATE 09/01/2024 : i've changed proxy_pass http://127.0.0.1:3000; into proxy_pass http://localhost:3000; in nginx. Now when i do curl http://localhost:3000 (on server) and curl https://mercure.astro-otter.space in local, i have the response from Caddy same as defined in Caddyfile,good point (youpi)

If i'm publishing from my Symfony controller or POST curl request:

Astro-Otter-Space commented 10 months ago

Finally i found a way to make it working correctly. I resume : First, i'm working with binary, not docker image. Env file /etc/environment : 

MERCURE_PUBLISHER_JWT_KEY="mySecretKey"
MERCURE_SUBSCRIBER_JWT_KEY="mySecretKey"
# Mercure URL
SERVER_NAME=http://localhost
MERCURE_PUBLIC_URL=https://mercure.exemple.space/.well-known/mercure

My nginx vhost : 

server {
  listen      443 ssl http2;
  listen [::]:443 ssl http2;
  server_name mercure.exemple.space;

  ssl_certificate /etc/letsencrypt/live/mercure.exemple.space/fullchain.pem; # managed by Certbot
  ssl_certificate_key /etc/letsencrypt/live/mercure.exemple.space/privkey.pem; # managed by Certbot

  location / {
    if ($http_origin ~ '^https?://(localhost(:[0-9]+)?|127.0.0.1(:[0-9]+)?|[^/]*\.exemple\.space)') {
        set $cors "true";
    }    

    if ($cors = "true") {
      add_header 'Access-Control-Allow-Origin' '$http_origin' always;    
      add_header 'Access-Control-Allow-Credentials' 'true';
      add_header 'Access-Control-Allow-Headers' 'Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
      add_header 'Access-Control-Allow-Methods' 'GET,POST,OPTIONS';
      add_header 'Content-Type' 'text/event-stream';
    }       

    proxy_pass http://localhost:3000;
    proxy_read_timeout 24h;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_connect_timeout 300s;

    #proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
  }

  # Configuration des logs
  access_log  /var/log/nginx/mercure_access.log;
  error_log   /var/log/nginx/mercure_error.log;

I create a systemd file for running mercure as service /etc/systemd/system/mercure.service : 

[Unit]
Description=Mercure.Rocks service
After=network.target
StartLimitBurst=5
StartLimitIntervalSec=33

[Service]
Type=simple
WorkingDirectory=/tmp
EnvironmentFile=-/etc/environment
ExecStart=/usr/bin/bash -c "MERCURE_PUBLISHER_JWT_KEY=$MERCURE_PUBLISHER_JWT_KEY MERCURE_SUBSCRIBER_JWT_KEY=$MERCURE_SUBSCRIBER_JWT_KEY SERVER_NAME=$SERVER_NAME /usr/bin/mercure run --config /var/www/mercure/Caddyfile"
StandardOutput=file:/var/log/nginx/mercure.log
StandardError=file:/var/log/nginx/mercure.log
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

In /var/www/mercure, here's my caddyfile : 

{
    {$DEBUG:debug}

    {$CADDY_GLOBAL_OPTIONS}
    order mercure after encode

    # Ports
    http_port 3000
    auto_https off
    {$GLOBAL_OPTIONS}
}

{$CADDY_EXTRA_CONFIG}

{$SERVER_NAME:localhost} {
    log {
        output file /var/log/caddy/mercure.log {
            roll true
            roll_size_mb 10
                roll_keep 5
        }
        format filter {
            wrap console
            fields {
                uri query {
                    replace authorization REDACTED
                }
            }
        }
        level INFO
    }

    encode zstd gzip

    mercure {
        # Transport to use (default to Bolt)
        transport_url {$MERCURE_TRANSPORT_URL:bolt://mercure.db}
        # Publisher JWT key
        publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
        # Subscriber JWT key
        subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
        # Extra directives
        # CORS
        cors_origins /** add here URL **/
        publish_origins *
        anonymous
        subscriptions
        {$MERCURE_EXTRA_DIRECTIVES}
    }

    {$CADDY_SERVER_EXTRA_DIRECTIVES}

    header / Content-Type "text/html; charset=utf-8"
    respond / `<!DOCTYPE html>
    <html lang=en>
    <meta charset="utf-8">
    <meta name="robots" content="noindex">
    <title>Welcome to Mercure</title>
    <h1>Welcome to Mercure</h1>
    <p>The URL of your hub is <code>/.well-known/mercure</code>.
    Read the documentation on <a href="https://mercure.rocks">Mercure.rocks, real-time apps made easy</a>.`

    respond /healthz 200

Code-side

My backend is a symfony (upgrade to API Platform ne day ^^). /path/to/backend/c/.env

MERCURE_URL=http://localhost:3000/.well-known/mercure
MERCURE_PUBLIC_URL=https://mercure.exemple.space/.well-known/mercure # use variable from /etc/environment ?
MERCURE_JWT_SECRET="PutYoutJWTHere"

/path/to/backend/config/packages/mercure.yaml

mercure:
  hubs:
    default:
      url: '%env(MERCURE_URL)%'
      public_url: '%env(MERCURE_PUBLIC_URL)%'
      jwt: 
        value: '%env(MERCURE_JWT_SECRET)%'
        publish: '*'

Front-side is a VueJS application, using EventSource, nothing exotic.

I think i need some adjustments but with all these it's working :).