search

alexbelgium / hassio-addons

My homeassistant addons
MIT License
1.52k stars 214 forks source link

[calibre-web] Ingress doesn't work from a domain #345

Closed met67 closed 2 years ago

met67 commented 2 years ago

Which addon?

Describe the bug

After installing and configuring Calibre-web add-on, "OPEN WEB UI" doesn't work (it opens a blank page). Same thing happens when with the "Show in sidebar" option: "Calibre-web" in sidebar opens a blank page. Accessing Calibre-web through port 8083 work fine, though.

To Reproduce

Full addon config

System

alexbelgium commented 2 years ago

Hi, thanks! I think it is a mistake from my part, I thought I had removed it

Thanks for the information I'll check what I can do

alexbelgium commented 2 years ago

Hi, actually it works for me - but is that the issue you were referencing?

image
met67 commented 2 years ago

🤔 this is strange... I also have restarted HA... image

alexbelgium commented 2 years ago

mmh I can see ingress but actually it doesn't open :-) I'm stuck at the loading screen

I'll check what I can do but probably I'll simply disable ingress ;-)

met67 commented 2 years ago

When I try to use ingress, these lines show in the add-on log:


[03/Jun/2022:17:11:57 +0200] 302 A.B.C.D, 172.30.32.1(172.30.32.2) GET / HTTP/1.1 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36)
[03/Jun/2022:17:12:01 +0200] 302 A.B.C.D, 172.30.32.1(172.30.32.2) GET / HTTP/1.1 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36)```
met67 commented 2 years ago

I'll check what I can do but probably I'll simply disable ingress ;-)

Too bad!!! 😭😭😭

alexbelgium commented 2 years ago

Ok I see! Ingress works fine from an insecure connection. It's when using https from a domain that it breaks. I'll investigate

alexbelgium commented 2 years ago

I haven't made progress yet. If I access it with ip:port, or with the domain but with the HA android app, ingress works fine. If I access it from domain name with a standard port such as 443, its works partially. If I access it with a domain name with a custom port, it doesn't work.

I don't know nginx enough to understand how to solve this but I'll try in the next days. Normally it should work :)

met67 commented 2 years ago

Uhmmm.... Chrome's Inspect console gives the following error:

Mixed Content: The page at 'https://domain.duckdns.org:1936/db21ed7f_calibre-web/dashboard' was loaded over HTTPS, but requested an insecure frame 'http://domain.duckdns.org:1936/api/hassio_ingress/hcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8/login?next=%2Fapi%2Fhassio_ingress%2Fhcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8%2F'. This request has been blocked; the content must be served over HTTPS.
alexbelgium commented 2 years ago

Good observation, this also occurred on this addon (https://github.com/alexbelgium/hassio-addons/issues/341#issuecomment-1144101901) I'll look in the coming days how to solve that

met67 commented 2 years ago

In the server response (taken with Fiddler), the Location: header has the wrong protocol... The link inside the generated HTML, instead, is relative so doesn't have this problem

HTTP/1.1 302 Found
Server: nginx
Date: Tue, 07 Jun 2022 07:21:46 GMT
Connection: keep-alive
Location: http://domain.duckdns.org:1936/api/hassio_ingress/hcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8/login?next=%2Fapi%2Fhassio_ingress%2Fhcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8%2F
Content-Security-Policy: default-src 'self'  'unsafe-inline' 'unsafe-eval'; font-src 'self' data:; img-src 'self'  data:
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000;
Vary: Cookie
Set-Cookie: session=.eJwNykEOgyAQQNGrTGZtDCAq9RTdN8YMdFBTKk2HnfHusvqL909cYiLZWHB6nQilBr8sQitjg8_EJAwpr7AfUDJQCBWhbLvArz4tztfcYJB_XEr-8IET-tgrbTplnbKD0qN5GzLeDeERqaPeej06cjHgdQM5aCd2.Yp78ig.rVtZ0K3qfmkNcv-ojHx-2CCBnYE; HttpOnly; Path=/; SameSite=Lax
Access-Control-Allow-Origin: *
Content-Type: text/html
Content-Length: 500

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="/api/hassio_ingress/hcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8/login?next=%2Fapi%2Fhassio_ingress%2Fhcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8%2F">/api/hassio_ingress/hcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8/login?next=%2Fapi%2Fhassio_ingress%2Fhcko9ExD6UF8_6J_9ibrYo4ZaSc-4uAbkoyp-Ys2kp8%2F</a>. If not click the link.
met67 commented 2 years ago 
X-Forwarded-Proto $scheme

didn't work for me, but I found a useful workaround. If you add an header like

proxy_set_header        HA-Calibre-User admin;

and enable Allow Reverse Proxy Authentication in Calibre-web Basic Configuration section in Admin, using HA-Calibre-User as Reverse Proxy Header Name, then you are automatically logged in as user admin... (it bypasses the login screen redirect that causes the Mixed Content)

Ideally this should be a configuration option of the add-in that, enabled by enterning the name of an existing calibre-web user. In this case there should be a ACL to somewhow restrict access from other networks...

alexbelgium commented 2 years ago

Wow great idea!! Knowing this nginx redirect is only for ingress, this could be enabled by default without security impact.

Thanks I'll look how to implement that!

alexbelgium commented 2 years ago

Mmh doesn't look that easy to implement - there doesn't seem to be an environment variable to set this, and I don't see a config file either I fear that this option would be stored in the database

met67 commented 2 years ago

Mmh doesn't look that easy to implement - there doesn't seem to be an environment variable to set this, and I don't see a config file either I fear that this option would be stored in the database

You probably could fiddle with sqlite and user and settings tables in app.db, but I gues it should be easier (and maybe overall better) to let the final user decide if and how to enable this. It should be possible to add

        allow   172.30.32.2;
        deny    all;
        proxy_set_header X-WebAuth-User admin;

(this is from Grafana add-on) only if user has set a username in the config of the add-on.

Another option would be to always use admin (like Grafana), which is the default and use sqlite just to replace the default settings (in settings table) at build time:

sqlite3 app.db 'update settings set config_reverse_proxy_login_header_name="X-WebAuth-User",config_allow_reverse_proxy_header_login=1;'
alexbelgium commented 2 years ago

Thanks! in parallel I've nearly found a way to make ingress work - the last remaining element is that I need to find a way to get homeassistant external port and add it in my nginx script

However, both our solutions can run in parallel. Where and when would you run the sqlite command? By modifying one of the calibre-web files through a tool like sed, or with a script at addon boot ?

met67 commented 2 years ago

bitmoji I'd prefer not to mess with calibre-web files, so definitely as a script at boot or even in the Docker file, to modify the default at build time

alexbelgium commented 2 years ago

ah ah, nice image is that you? ;-) I'll look at that - first finalize the ingress part then move on to test the automatic user logon

alexbelgium commented 2 years ago

Ingress is working in version -test4, could you please check? You need to clear browser cache for that, or (more easily) use incognito mode on your browser

Thanks!

met67 commented 2 years ago

Ingress is working in version -test4, could you please check? You need to clear browser cache for that, or (more easily) use incognito mode on your browser

Thanks!

I'll check it as soon as my HA sees -test4 version... ATM it only shows -test3 (and I don't have much time left to play with HA as of today)

alexbelgium commented 2 years ago

You can force it by manually refreshing the repo!

met67 commented 2 years ago

Ok, I’ll try as soon as possible… I have seen that you disabled strong session protection with a patch to CW code… As an alternative have you thought to explicitly configure Scheme in the addin configuration? If you “manually” use proxy_set_header X-Scheme http or https according to your ha installation, it just works…

/M

met67 commented 2 years ago

Ingress is working in version -test4, could you please check? You need to clear browser cache for that, or (more easily) use incognito mode on your browser

Thanks!

Nope! I have updated to -test4 and used Incognito mode in Chrome, and still got Mixed Content error...

EDIT Wait, maybe my install got messed up, I'll remove it and add again

EDIT 2 Nope, even after a fresh reinstall it still doesn't works... moreover, when I access it through port 8123 (usually I have a port forward to access it from the outside on a different port...), it redirect me to this URL, asking again for HA login...

https://domain.duckdns.org:8123/auth/authorize?response_type=code&redirect_uri=https%3A%2F%2Fdomain.duckdns.org%3A8123%2Fdb21ed7f_calibre-web%2Fdashboard%3Fauth_callback%3D1&client_id=https%3A%2F%2Fdomain.duckdns.org%3A8123%2F&state=eyJoYXNzVXJsIjoiaHR0cHM6Ly9sYWxpenphLmR1Y2tkbnMub3JnOjgxMjMiLCJjbGllbnRJZCI6Imh0dHBzOi8vbGFsaXp6YS5kdWNrZG5zLm9yZzo4MTIzLyJ9

met67 commented 2 years ago

Give proxy_set_header X-Scheme a try... BTW, I guess you already know, but in just in case you don't you can test new configurations live on the container by connecting to the container's console, editing config files with vi and reloading nginx with nginx -s reload)

alexbelgium commented 2 years ago

Hi, indeed that's exactly what I do ;-) I connect with console using portainer to make the changes, then push a new version only when it seems to work.

With -test4 it should in theory convert all http in $scheme, and add the port for url that don't have it which seemed to me the 2 key issues https://github.com/alexbelgium/hassio-addons/blob/a4fcc2fc5781ea89ca374c2dec7183af9e96ba32/calibre_web/rootfs/etc/nginx/servers/ingress.conf#L20-L21

When doing that I can open ingress fine either when loading HA from IP or domain... I'll check if I can still see the mixed content error

EDIT : I've already a X-Scheme header with https://github.com/alexbelgium/hassio-addons/blob/a4fcc2fc5781ea89ca374c2dec7183af9e96ba32/calibre_web/rootfs/etc/nginx/servers/ingress.conf#L17 that should in theory provide what you propose...

alexbelgium commented 2 years ago

I have seen that you disabled strong session protection with a patch to CW code…

Actually I think that this is not required anymore, I'll remove it in -test5

met67 commented 2 years ago

The problem is that, at least in my setup (which is quite standard...), $scheme turns out to be http, so

       proxy_set_header        X-Scheme                $scheme;

doesn't work while

       proxy_set_header        X-Scheme                https;

do. I have found this by adding a 

       add_header              X-Test  $scheme;

That's why I suggested to add an explict Scheme option in configuration...

Im my tests, forcing X-Schemeto https was enough to make it work.

alexbelgium commented 2 years ago

Strange! Thanks I'll put an addon variable then - it can be used to force http or https, and if not used it will use the default $scheme...

What does add_header X-Test $scheme do ?

met67 commented 2 years ago

Strange! Thanks I'll put an addon variable then - it can be used to force http or https, and if not used it will use the default $scheme...

What does add_header X-Test $scheme do ?

It just adds an Header named X-Test so that I can check the value that $scheme has in that context from the browser (I use Fiddler, BTW)... I'm old school, coming from C printf() debugging 😁

alexbelgium commented 2 years ago

Nice method indeed. My way of troubleshooting is to open the direct ingress url (like https://domain:8123/api/hassio_ingress/pHdNlkLrbT4eYkLFMmHgwBfH5uBmq-hcGx6-t-iuPpQ/) then use Inspect to see any errors during page load in console or network. Probably less efficient then other tools, but at least I can manage with it to troubleshoot a bit :-)

Programming is not my job, I am more self taught using existing code and troubleshooting via internet so sometimes my solution is not the most elegant :-) Two of my key elements are mimicry (by mimicking what other people do to solve comparable issues) and iterative design (by improving based on feedback) which for the moment have allowed it to work in several contexts :-)

Btw if you have solutions for issues don't hesitate to propose PR as I see you seem to be very knowledgeable on those systems!

On our issue at hand, I've pushed a new version with an optional boolean force_scheme_https that uses sed to replaces all $scheme instances with https if enabled

met67 commented 2 years ago

On our issue at hand, I've pushed a new version with an optional boolean force_scheme_https that uses sed to replaces all $schemeinstances withhttps if enabled

I've seen it and gave it a try... to make it work I had to remove these lines:

       proxy_redirect          http://$host/           $scheme://$host:8123/;
       proxy_redirect          $scheme://$host/        $scheme://$host:8123/;

(which is somehow expected, since I don't use port 8123 from the outside...)

alexbelgium commented 2 years ago

in theory the 8123 comes from bashio::core.port which provides the external port exposed by the HA system. In my system indeed it is the default, 8123. https://github.com/alexbelgium/hassio-addons/blob/adfd3610105191e3d6a7e5c4e4d19e5e0534444c/calibre_web/rootfs/etc/cont-init.d/32-nginx.sh#L32

This line is needed when using a non-standard port (such as 80 or 443) as I observed that some urls from calibre-web were lacking the port and giving a 404 on my system...

Are you using 8123 locally but another port is exposed by your router?

Edit : latest pushed version is -test7 Edit2 : if I remove those lines, I can't connect to my own ingress Edit3 : the need for addition of this line for non-standard ssl port is referenced also in the official documentation for setting nginx https://github.com/janeczku/calibre-web/wiki/Setup-Reverse-Proxy#nginx Edit4: only solution I see is to propose you an option for specifying the external port...

met67 commented 2 years ago

Are you using 8123 locally but another port is exposed by your router?

Yes, this is my case: locally 8123 but another port expoxed externally with a port forward... I'll try locally and see what's going on...

Edit : latest pushed version is -test7 Edit2 : if I remove those lines, I can't connect to my own ingress Edit3 : the need for addition of this line for non-standard ssl port is referenced also in the official documentation for setting nginx https://github.com/janeczku/calibre-web/wiki/Setup-Reverse-Proxy#nginx Edit4: only solution I see is to propose you an option for specifying the external port...

Nice finding, I'll try it... from the nginx docs (proxy_redirect) this could be a better solution...

EDIT: I can access ingress path locally from 8123 WITHOUT proxy_redirect... I'm puzzled.

met67 commented 2 years ago

Btw if you have solutions for issues don't hesitate to propose PR as I see you seem to be very knowledgeable on those systems!

ATM I don't have a development setup with a build system for HA repositories, and don't have too much time to spend trying to set up one...

met67 commented 2 years ago

OK, another try (I'd prefer not to mess with port configuration if not strictly necessary...)

First, my setup is fairly standard: HAOS on a Raspberry, running in HTTPS on port 8123, certificate and external DNS registration with DuckDNS Addon, external access with port forwarding from a different port.

When this thread started, ingress.conf was like this:

    location / {
       add_header              Access-Control-Allow-Origin *;
       proxy_pass              http://127.0.0.1:8083;
       proxy_buffering         off;
       proxy_read_timeout      30;
       proxy_set_header        Connection              "Upgrade";
       proxy_set_header        Upgrade                 $http_upgrade;
       proxy_bind              $server_addr;
       proxy_set_header        Host                    $http_host;
       proxy_set_header        X-Forwarded-For         $proxy_add_x_forwarded_for;
       proxy_set_header        X-Scheme                $scheme;
       proxy_set_header        X-Script-Name           /api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g;  # IMPORTANT: path has NO trailing slash
    }

and, when I first try to access Calibre-Web in HA through Ingress, Calibre-Web redirect me to the login page with this response (captured with Fiddler):

HTTP/1.1 302 Found
Server: nginx
Date: Wed, 08 Jun 2022 10:41:39 GMT
Connection: keep-alive
Location: http://domain.duckdns.org:5678/api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g/login?next=%2Fapi%2Fhassio_ingress%2FbxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g%2F
Content-Security-Policy: default-src 'self'  'unsafe-inline' 'unsafe-eval'; font-src 'self' data:; img-src 'self'  data:
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000;
Vary: Cookie
Set-Cookie: session=.eJwNyjEOgzAMQNGrWJ5RVUygDqdgrxBKjQOolFR1NsTdm-kP7584xT3Yqob980TIJfhRs7AoVjjsGkxhTwtsB-QEQaQg5HUz-JbnhuM1Vij2i1NObz2wx8CdkCd5sKs7XzvWu2u5Cc1ML56JpFWK0TNefzzfJ2k.YqB84w.097GuS2H93WvomKeHCpSxII9tJg; HttpOnly; Path=/; SameSite=Lax
Access-Control-Allow-Origin: *
Content-Type: text/html
Content-Length: 500

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="/api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g/login?next=%2Fapi%2Fhassio_ingress%2FbxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g%2F">/api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g/login?next=%2Fapi%2Fhassio_ingress%2FbxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g%2F</a>. If not click the link.

This fails with Mixed Content error since the redirect location

Location: http://domain.duckdns.org:5678/api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g/login?

uses http instead of https.

Using proxy_redirect , we can change http to https in the Location with a RegEx:

       proxy_redirect          ~^http:(.+)$            https:$1;

and now the response is correct:

HTTP/1.1 302 Found
Server: nginx
Date: Wed, 08 Jun 2022 10:48:42 GMT
Connection: keep-alive
Location: https://domain.duckdns.org:5678/api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g/login?next=%2Fapi%2Fhassio_ingress%2FbxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g%2F
Set-Cookie: session=.eJydi0EOgjAQAL-y2XNjpBTc8grvhpC6bIFYqXF7I_zdvsHTHGbmwCmmoKsoDo8DoVTgW1TDImjwniSoQMoLbDuUDIG5SijrpvCpzQXH0_z5jQZZv3Eq-SU7DhioZ-st38g1vW8cydV11IZ2tk-areVObIye8PwBeCI5-w.YqB-ig.bi55mGb6pZWZst4uasfRt_Iu5xM; HttpOnly; Path=/; SameSite=Lax
Content-Security-Policy: default-src 'self'  'unsafe-inline' 'unsafe-eval'; font-src 'self' data:; img-src 'self'  data:
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000;
Vary: Cookie
Access-Control-Allow-Origin: *
Content-Type: text/html
Content-Length: 500

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="/api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g/login?next=%2Fapi%2Fhassio_ingress%2FbxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g%2F">/api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g/login?next=%2Fapi%2Fhassio_ingress%2FbxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g%2F</a>. If not click the link.

Since I din't find any way to know if we are using http o https within the plugin, the force_ssl_schema option seems the only way to add the proxy_redirect line when needed...

alexbelgium commented 2 years ago

Good idea to summarize. Now, I've tested your proposal as I understand it :

location / {
       add_header              Access-Control-Allow-Origin *;
       proxy_pass              http://127.0.0.1:8083;
       proxy_buffering         off;
       proxy_read_timeout      30;
       proxy_set_header        Connection              "Upgrade";
       proxy_set_header        Upgrade                 $http_upgrade;
       proxy_bind              $server_addr;
       proxy_set_header        Host                    $http_host;
       proxy_set_header        X-Forwarded-For         $proxy_add_x_forwarded_for;
       proxy_set_header        X-Scheme                $scheme;
       proxy_set_header        X-Script-Name           /api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g;  # IMPORTANT: path has NO trailing slash

# NEW ADDITION
       proxy_redirect          ~^http:(.+)$            https:$1;
    }

However, the page doesn't load for me. Looking in firefox inspect, I see that 2 elements fail to load with the request URL : https://MYDOMAIN/api/hassio_ingress/pHdNlkLrbT4eYkLFMmHgwBfH5uBmq-hcGx6-t-iuPpQ/login?next=%2Fapi%2Fhassio_ingress%2FpHdNlkLrbT4eYkLFMmHgwBfH5uBmq-hcGx6-t-iuPpQ%2F

As you can see, it is missing the port - reason why it doesn't connect. For me, this is the (exact case described in the app documentation)[https://github.com/janeczku/calibre-web/wiki/Setup-Reverse-Proxy#nginx] and that I can only solve by adding proxy_redirect https://$host/ https://$host:8123/; (considering that the http -> https command was already given before

What I don't understand is that you don't face the same issue although you also seem to have a non standard ssl port of 5678!

ps : Fiddler is a nice tool, thanks for referencing it

met67 commented 2 years ago

This is strange... I guess that you access your HA through https://MYDOMAIN:8123/.... Have you tried to capture the whole flow with Fiddler? I have no idea why it would behave like this...

alexbelgium commented 2 years ago

Here is what I see.

Request : 

GET /api/hassio_ingress/pHdNlkLrbT4eYkLFMmHgwBfH5uBmq-hcGx6-t-iuPpQ/ HTTP/1.1
Host: MYDOMAIN:8123
Connection: keep-alive
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="102", "Microsoft Edge";v="102"
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Upgrade-Insecure-Requests: 1
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36 Edg/102.0.1245.33
sec-ch-ua-platform: "Windows"
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: navigate
Sec-Fetch-Dest: empty
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,en-US;q=0.8,fr;q=0.7
Cookie: ingress_session=e0d974aa352e9668d6ab41cf1b4565bd496110c9459cb1385f2bf6b1f876d76b179eb5b7a90435912d171fa2fec5fd84174d52df05db095591b69eb09018e352; session=.eJyrVopPy0kszkgtVrKKrlZSKAFSSrmpxcWJ6alKOkoBOamJxakKOfnpCpl5CiX5ConJyUBJhZKMzGKFAqAaPaXYWp1RfVj1xdYCALr2dLQ.YqCnig.zj5Vd3JosHh_VhyaXN6E8uqtdOU

Response : 

HTTP/1.1 302 Found
Server: nginx
Date: Wed, 08 Jun 2022 13:44:11 GMT
Content-Type: text/html
Content-Length: 500
Connection: keep-alive
Location: https://MYDOMAIN/api/hassio_ingress/pHdNlkLrbT4eYkLFMmHgwBfH5uBmq-hcGx6-t-iuPpQ/login?next=%2Fapi%2Fhassio_ingress%2FpHdNlkLrbT4eYkLFMmHgwBfH5uBmq-hcGx6-t-iuPpQ%2F
Set-Cookie: session=.eJyrVopPy0kszkgtVrKKrlZSKAFSSrmpxcWJ6alKOkoBOamJxakKOfnpCpl5CiX5ConJyUBJhZKMzGKFAqAaPaXYWp1RfVTVF1sLADb4h0Y.YqCnqw.KSf3xsh63wqlnTMHU3OwMgulc2I; HttpOnly; Path=/; SameSite=Lax
Content-Security-Policy: default-src 'self'  'unsafe-inline' 'unsafe-eval'; font-src 'self' data:; img-src 'self'  data:
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000;
Vary: Cookie
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubDomains

As you can see, the location is missing the port directive. As such, the next page opened is with the default ssl port, 443

met67 commented 2 years ago

I have no idea... does MYDOMAIN contain some char that could mess RegEx? I guess not, since as I understand it you had the same behaviour even before testing the RegExp...

alexbelgium commented 2 years ago

Indeed, I've had this behavior from the start and it is a documented behavior from the app official documentation (need for manual port addition in nginx when using a non standard port)... What I don't understand if why you don't get that! Having thought about it, I think I'll make specifying the port mandatory in the addon options. It is only a one time action, and would ensure that it works for everyone. The force_https flag would still remain optional

met67 commented 2 years ago

Wow, what a thread!!! 😂 AW, I'd leave the port optional, maybe most people won't need it and it could confuse them. I have tested -test8 version, and it works for me also, with a port different from 8123... on a second thought it could be that these rules 

       proxy_redirect          http://$host/           https://$host:8123/;
       proxy_redirect          https://$host/        https://$host:8123/;

will not match in my config, maybe http://$host/ will not match http://domain.duckdns.org:5678/ (don't know the exact rules behind)

AW, as it is now it works for me...

met67 commented 2 years ago

I just found that you CAN infer the "context" in ngninx by using $http_referer var, you can see how it work using this line:

      add_header              X-REFERER   $http_referer;

Now we must find a way to use it to avoid force_ssl option 😜

met67 commented 2 years ago

OK, this should give everything we need:

       if ($http_referer ~* "^(http[s]?)://([^:]+):(\d*)(/.*)$") {
         set $x_scheme $1;
         set $x_host   $2;
         set $x_port   $3;
       }

To see what values are assigned to these new vars, I used

       add_header              X-SCHEME $x_scheme;
       add_header              X-HOST   $x_host;
       add_header              X-PORT   $x_port;

and in my setup they where correctly set:

       X-SCHEME https
       X-HOST   domain.duckdns.org
       X-PORT   5678

So it shoud be possible to use them to make ingress work without other settings...

       proxy_set_header        X-Scheme                $x_scheme;
       proxy_set_header        X-Script-Name           /api/hassio_ingress/bxDM6mkTGKUDxIB6ACdPx10S0iQvsBUxj7G6KF3HF2g;  # IMPORTANT: path has NO trailing slash
       #proxy_set_header        HA-Calibre-User         admin;
       proxy_redirect          http://$host/           $x_scheme://$x_host:x_port/;
       proxy_redirect          https://$host/          $x_scheme://$x_host:x_port/;

I have tested this config in my environment and it works...

alexbelgium commented 2 years ago

This is an extremely elegant solution. I've test both with ip and domain access, both with http and https, and everything worked as expected. Thanks very much I'll implement this in the addon

alexbelgium commented 2 years ago

I've tested again with different scenario and it all worked! you're the best :-)

Honestly thanks very much for the collaborative debugging this was a very insightful and learning experience for me. And in the end this is an elegant solution that allows maximum user experience without having to set any manual options.

I've pushed a test9 version built with this nginx configuration that seems to work very well

met67 commented 2 years ago

Just giving back and helping build tools that I use :-) BTW, thanx for your Plex with NAS!

I have only a doubt that it could fail when using a standard port (e.g. with CloudFlare and port 443…) in such a scenario it could redirect to http(s)://domain:/ which would fail…

/M

Inviato da iPhone

Il giorno 8 giu 2022, alle ore 20:50, Alexandre @.***> ha scritto:

 I've tested again with different scenario and it all worked! you're the best :-)

Honestly thanks very much for the collaborative debugging this was a very insightful and learning experience for me. And in the end this is an elegant solution that allows maximum user experience without having to set any manual options.

I've pushed a test9 version built with this nginx configuration that seems to work very well

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

alexbelgium commented 2 years ago

Good idea - indeed... for that, couldn't we add the ":" in the "$x_port" variable ? so it contains ":port". And as the first regex wouldn't trigger, we could have a second regex that only triggers when there is no port in the url and that generates a null "$x_port"

Something like : 

       if ($http_referer ~* "^(http[s]?)://([^:]+):(\d*)(/.*)$") {
         set $x_scheme $1;
         set $x_host   $2;
         set $x_port   ":$3";
       }
       if ($http_referer ~* "^(http[s]?)://([^:]+)(/.*)$") {
         set $x_scheme $1;
         set $x_host   $2;
         set $x_port   "";
       }

       proxy_redirect          http://$host/           $x_scheme://$x_host$x_port/;
       proxy_redirect          https://$host/          $x_scheme://$x_host$x_port/;

EDIT : yes, working I've pushed a v10!

Glad that the addons are useful for you! And thanks for the discussions ;-)

met67 commented 2 years ago

Just updated and can confirm that it is working fine!!! bitmoji

alexbelgium commented 2 years ago

Yeah! What a great collaboration :-) Thanks very much I'll close the issue now. Have a nice day