v1.26.2 breaks "/webcam/?action=stream" popup when using HTTPS

[masterblaster79]

Fluidd Version

v1.26.2 and higher

Browser

Firefox, Other (Please tell us below)

Device

Desktop PC, Other (Please tell us below)

Operating System

Linux

What happened

I am serving fluidd via nginx with https enabled. The /webcam/?action=stream URI is working only once

What did you expect to happen

/webcam/?action=stream URI should work reliably on subsequent requests also

How to reproduce

1. Install Klipper/Fluidd/crowsnest/nginx using kiauh.sh on a server with webcam connected (Debian Bookworm in my case)
2. Configure nginx to serve fluid via HTTPS
3. Connect to fluidd and configure the webcam (I am using crowsnest/ustreamer)
4. Click the camera fullscreen and then the camera popout, it'll open a new Window with an URL like this: https://fluidd-server/webcam/?action=stream&cacheBust=1702474264593 URL
5. The first time this works just fine
6. Close the popup and try to open it again, this time the browser will show a blank page only
7. The only way to get /webcam/?action=stream back to work, is to purge the browser cache
8. Using plain unencrypted HTTP is working properly

Additional information

Klipper/Fluidd is running on Vanilla Debian Bookworm on an Banana PI SBC Installation has been done with kiauh.sh I reconfigured fluidd nginx config to offer https only (http requests are being redirected to https) Client side I used Linux/Firefox, Linux/Chromium and Android/Firefox - all show the same behaviour

[pedrolamas]

Hi @masterblaster79, thanks for opening this ticket.

Can I ask that you share your nginx configuration?

[masterblaster79]

Yes, I will share this in a moment. One thing I also noticed, in the nginx logfile configuration I can see these requests: "GET /webcam/assets/index-252af6e9.js "GET /webcam/assets/index-93bc11ba.css Which should not go to the /webcam/ namespace usually (and hence ustreamer answers this with 404).

Maybe related to this: https://github.com/fluidd-core/fluidd/commit/9df4addb12eee47ecf98369dc2727906ee995576

[masterblaster79]

klipper@3dp:~/fluidd$ cat /etc/nginx/sites-enabled/fluidd 
# /etc/nginx/sites-available/fluidd

server {
    listen 80;
    return 301 https://$host$request_uri;
    access_log /var/log/nginx/fluidd-access.log;
    error_log /var/log/nginx/fluidd-error.log warn;
}

server {
    #listen 80;
    listen 443 ssl;
    ssl_certificate     /etc/ssl/certs/3dp.pem;
    ssl_certificate_key /etc/ssl/private/3dp.key;

    access_log /var/log/nginx/fluidd-access.log;
    error_log /var/log/nginx/fluidd-error.log;

    # disable this section on smaller hardware like a pi zero
    gzip off;
    #gzip_vary on;
    #gzip_proxied any;
    #gzip_proxied expired no-cache no-store private auth;
    #gzip_comp_level 4;
    #gzip_buffers 16 8k;
    #gzip_http_version 1.1;
    #gzip_types text/plain text/css text/xml text/javascript application/javascript application/x-javascript application/json application/xml;

    # web_path from fluidd static files
    root /home/klipper/fluidd;

    index index.html;
    server_name _;

    # disable max upload size checks
    client_max_body_size 0;

    # disable proxy request buffering
    proxy_request_buffering on;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location = /index.html {
        add_header Cache-Control "no-store, no-cache, must-revalidate";
    }

    location /websocket {
        proxy_pass http://apiserver/websocket;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_read_timeout 86400;
    }

    location ~ ^/(printer|api|access|machine|server)/ {
        proxy_set_header Connection "";
        proxy_pass http://apiserver$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Scheme $scheme;
        proxy_read_timeout 600;
    }

    location /webcam/ {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        postpone_output 0;
        proxy_buffering on;
        proxy_ignore_headers X-Accel-Buffering;
        #access_log off;
        #error_log /dev/null emerg;
        proxy_pass http://mjpgstreamer1/;
    }

    location /webcam2/ {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        postpone_output 0;
        proxy_buffering on;
        proxy_ignore_headers X-Accel-Buffering;
        access_log off;
        #error_log /dev/null emerg;
        proxy_pass http://mjpgstreamer2/;
    }

    location /webcam3/ {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        postpone_output 0;
        proxy_buffering on;
        proxy_ignore_headers X-Accel-Buffering;
        access_log off;
        #error_log /dev/null emerg;
        proxy_pass http://mjpgstreamer3/;
    }

    location /webcam4/ {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        postpone_output 0;
        proxy_buffering on;
        proxy_ignore_headers X-Accel-Buffering;
        access_log off;
        #error_log /dev/null emerg;
        proxy_pass http://mjpgstreamer4/;
    }
}

klipper@3dp:~/fluidd$ cat /etc/nginx/conf.d/upstreams.conf 
# /etc/nginx/conf.d/upstreams.conf
upstream apiserver {
    #ip_hash;
    keepalive 5;
    server 127.0.0.1:7125;
}

upstream mjpgstreamer1 {
    #ip_hash;
    keepalive 5;
    server 127.0.0.1:8080;
}

upstream mjpgstreamer2 {
    #ip_hash;
    keepalive 5;
    server 127.0.0.1:8081;
}

upstream mjpgstreamer3 {
    #ip_hash;
    keepalive 5;
    server 127.0.0.1:8082;
}

upstream mjpgstreamer4 {
    #ip_hash;
    keepalive 5;
    server 127.0.0.1:8083;
}

[masterblaster79]

It's basically the plain one from kiauh with some minor (unrelated, I tested) modifications and ssl enabled

[pedrolamas]

Thanks for providing that, I will now check your configuration.

In the meantime, you might want to check my own configuration as I use a similar SSL setup.

I am certain this is not a bug but some sort of configuration issue, so I will now change the labels of the ticket.

[pedrolamas]

So a couple of things you might want to try to change on your location /webcam/ block:

1. not sure you actually need the proxy_http_version 1.1; and proxy_set_header Connection ""; lines, so try to remove those
2. change proxy_buffering on; to proxy_buffering off;
3. add access_log off; and error_log off;

Point 2 above is quite important, and I would definitely do that one!

[masterblaster79]

Yes, maybe.

Two points make me think it's a bug:

1. It works just fine when using HTTP without SSL offload.
2. v1.26.1 an earlier are fine

Your configuration is a bit different, as it's a frontend proxy sending everything to upstream servers. In my case I am serving fluidd from the same hosts file system (root /home/klipper/fluidd;)

[masterblaster79]

Regarding your suggestions, they are unrelated. I already tried that. These are changes I did to the configuration provided by kiauh, but without those it's the same. https://www.nginx.com/blog/avoiding-top-10-nginx-configuration-mistakes/

[pedrolamas]

That is fair, I am by no means an expert on nginx configuration here, only pointing out that I have no issues using SSL in my case.

What happens if you open the camera address (eg. https://fluidd-server/webcam/?action=stream) on a separate tab? Does the video load?

[masterblaster79]

Yes, that's the weird thing. When I open https://fluidd-server/webcam/?action=stream with freshly started browser, everything is fine (obviously, because nginx sends this to ustreamer bypassing fluidd and moonraker). I can reload the URL, no problem.

But: When I go there from fluidd, I can open the stream URL only once, then the bug (or config issue) is triggered, I cannot open the stream URL anymore in the same browser session. Even when I close the fluidd tab and open a new empty tab, paste the stream URL -> nothing. It's weird, because it should not even touch fluidd and proxy should send it directly to ustreamer. Only closing the whole browser helps then. (Closing seems to be enough, clean cache not neccessary)

When I open fluidd with a clean browser, it starts caching stuff (I think for this "offline" functionality or something), maybe it's related? Is there a way to disable this config wise?

[pedrolamas]

Fluidd does indeed have a service worker, but this is configured to bypass the /webcam/ url.

The way to confirm that it is bypassing the service worker is by:

• Open Fluidd on the browser
• Press F12 to open the Developer Tools
• Change to the Network tab
• Press F5 to refresh the page
• Check on the list for "/webcam/" requests, click one of the requests
• Check the "Fulfilled by" column, it should NOT say "(ServiceWorker)"

[masterblaster79]

OK, it definitively seems to be related to the offline caching. I thought that it works only once , but that's because I was doing my tests too quickly.

It works as long as fluidd hasn't finished caching. Once it finished caching and this "you can work offline" popup appeared, the stream URL is broken.

Remember, in the log file I saw this:

GET /webcam/assets/index-252af6e9.js
GET /webcam/assets/index-93bc11ba.css

These requests should never happen in /webcam/, so fluidd somehow has the wrong context. Also this seems to be stored deep inside the browsers cache, because the stream URL as such is unrelated to fluidd but still the browser sends fluidd related requests.

... but this still doesn't explain why you cannot reproduce it in your setup and it doesn't explain why without ssl it's fine.

My best bet still is https://github.com/fluidd-core/fluidd/commit/9df4addb12eee47ecf98369dc2727906ee995576 Running out of time now, to be continued.

[pedrolamas]

When you have the chance, please take a screenshot of your camera settings and post it here please.

[masterblaster79]

Check the "Fulfilled by" column, it should NOT say "(ServiceWorker)" ..but this is exactly what I see there for the not working stream URL in the pop up tab.

[pedrolamas]

Took me a while to set this up, but I have now managed to reproduce the issue and indeed, it seems to have been caused by 9df4addb12eee47ecf98369dc2727906ee995576, which was in turn related to #1221

I will need to double-check all of the code, but I have a feeling I will have to revert that change to make sure that these URLs are not service by the service worker.

[pedrolamas]

I have just pushed 7a001dd9a1866a7df0ef433ffe16f4f5031e3e4b which I believe fixes this issue, any chance I can get you to test it from the dev branch and see if it helps?

[masterblaster79]

I haven't done extensive testing now, but a quick check is looking good. No issue so far. Thanks for your very quick reaction and this great software, mate. You can close here if you want.

[pedrolamas]

I've pushed some more changes in fc017922bb44c3d3a04694a87e6a2095d0279caf that simplify the worker code, I will test these for a few more days and if everything is ok I will close this ticket

[thijstriemstra]

I was about to create a ticket to flag the issue with not being able to close fullscreen camera (webrtc) mode but this ticket looks pretty similar. I'm not using https though.

[pedrolamas]

Fixed via fc017922bb44c3d3a04694a87e6a2095d0279caf