handle 401 on websocket connection

[h3ndrik]

Since the websocket update, I have difficulties logging in from remote.

Home Assistant release (hass --version): 0.38.3 Python release (python3 --version): Python 3.5.3 Component/platform: webui

Description of problem:

My (slightly more complex) setup contains a nginx proxy which does https and enforces http basic auth if you're outside my home network. This results in the following behaviour:

1. everything works fine at home
2. when i'm away from home, the browser got the UI already in cache and only requests /api/websocket and receives a http 401, tries again /api/websocket, receives 401... 2.1. this is repeated several times and then the browser shows the cached version of the login screen, but again - there is no new request to the server. 2.2. the 401 responses to the websocket connection don't trigger any new username/password prompt

The home-assistant authentication is disabled: # api_password:

I'm not exactly an expert on web-development... I think the problem is the web-ui (javascript) not handling the http status 401 correctly. Is it possible to disregard the browser cache and reload "/" or something so the browser will show a new authentication popup? I'd like to use the http basic auth from nginx so i can have multiple logins/accounts.

Probably(?) related issue: #5954

nginx configuration:

map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
}

server {
        listen 443 ssl;
        listen [::]:443 ssl;

        # SSL configuration
        ssl_certificate /etc/letsencrypt/live/<...>/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/<...>/privkey.pem;

        add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";

        access_log      /var/log/nginx/smarthome_access.log;
        error_log       /var/log/nginx/smarthome_error.log;

        charset                 utf-8;

        server_name <...>;

        location / {
                satisfy                 any;
                allow                   192.168.0.0/24;
                auth_pam                "Smarthome";
                auth_pam_service_name   "nginx";

                proxy_pass      http://192.168.0.10:8123;
                proxy_buffering         off;
                proxy_set_header        Host            $http_host;
                proxy_set_header        X-Real-IP       $remote_addr;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header        X-Forwarded-Proto       $scheme;
                proxy_redirect http:// https://;
                proxy_http_version      1.1;
                proxy_set_header        Upgrade         $http_upgrade;
                proxy_set_header        Connection      $connection_upgrade;
        }
}

[camrun91]

the server name line <...> I assume you have the actual DNS name there? I know I was having issues with this until I completely matched the example config here https://home-assistant.io/docs/ecosystem/nginx/ at the bottom of the page. There are a few differences between that config and your attached config.

[andriej]

@camrun91 config is not the issue, as I have it set like on example page. Additional headers which @h3ndrik have shouldn't affect the proxy. Everything was working ok before 0.38

[hessu]

Hi, it appears that some browsers do not pass over the authentication information from plain HTTP requests session to the WebSocket requests. The 401 response is coming from nginx, not hass.

To support those browsers with WebSocket API, the authpam directives would need to be excluded for /api/websocket URL as described in #5954, comment by https://github.com/home-assistant/home-assistant/issues/5954#issuecomment-279949890 – the example of adding a separate "location /api/websocket" block without auth* directives works, although the description around it does not describe why it's needed.

[h3ndrik]

@camrun91 Yes, the server name line contains the actual DNS name.

If i exclude the /api/websocket URL from the authentication as @hessu pointed out, wouldn't that leave me with a security issue there? Maybe i have to dig into the code and learn how websockets work.

But after reading https://github.com/home-assistant/home-assistant/issues/5954#issuecomment-279949890 again... it could well be the browser that is using some cached, pre 0.38-code? I purged the cache and -for now- it is working. But i'm not sure if it will keep working after the session times out again and nginx will send a 401. I'll try that next.

[demonspork]

This issue is in polymer see the issue here : https://github.com/home-assistant/home-assistant-polymer/issues/110

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

[KlaasH]

Running 0.44.2, no change.

[abmantis]

Also having this issue on 0.45.

[cmsimike]

Shot in the dark here - what browser/os? Safari/MacOS and iOS do not pass client certificates when making a websocket connection. If you are using similar os/browser, maybe that's it? And if so, can you try in Chrome?

[abmantis]

@cmsimike I've tried with both Chrome and Firefox on desktop and Chrome and Opera on Android.

[elupus]

Was just hit with this issue too. Clearing cache resolves is for one session.

I think a workaround would be to let nginx change cache headers on the main screen, so it gets loaded an re-authenticated.

Still a better solution would be preferable

[elupus]

This seem eerie similar to: #1303

[elupus]

Also i just noticed that it's not just cache going one here. Home Assistant register a service_worker.js which seems to handle requests without them hitting server.

[elupus]

An ugly workaround to temporarily restore connectivity without clearing cache is to have nginx serve a special logout url that resides at an url location that is whitelisted in service_worker.js (/api, /local, ...).

This logout url will de-register the service_worker and clear authentication, allowing you to re-login.

In your nginx server block add an include for the below logout snippet.

/etc/nginx/snippets/logout.conf

location /local/logout {
        return 401;
}

error_page 401 /local/errors/401.html;

location /local/errors {
        auth_basic off;
        ssi        on;
        ssi_types  text/html;
        alias /var/www/html/errors;
}

/var/www/html/errors/401.html

<!DOCTYPE html>
<script>
navigator.serviceWorker.getRegistrations().then(function(registrations) {
         for(let registration of registrations) {
                   registration.unregister()
         } })
</script>
<p>You're not authorised. <a href="<!--# echo var="scheme" -->://<!--# echo var="host" -->/">Login</a>.</p>

[andkit]

Thx for the tip with the whitelisted uris (google lead me here after I made out the service worker as the culprit responsible for eating away my requests)

For anyone else with this issue, you probably don't even have to deregister the service worker, you just need an url that will trigger the authentication dialog (the websocket requests don't and everything else comes out of the cache)

I'm going to try this setup for a while (the same with a 200 static response worked for me on desktop and android, the redirect would just be added convenience but I'm almost certain it should work too)

    location = /local/auth_me {
        add_header Cache-control private;
        add_header Cache-control no-cache;
        add_header Cache-control no-store;
        if ($remote_user != "") {
            return 302 /;
        }
    }

[aderusha]

I'm seeing this same problem using a Netscaler for advanced authentication.

[vondruska]

At least using Chrome, another workaround is to prevent the service worker from loading through nginx. You'll lose things that depend on the worker like HTML5 Push Notifications, but I don't use those features in my environment. Here's the steps I used:

1. Add the deny to service_worker.js in your nginx config:

   location = /service_worker.js {
   deny all;
   }

2. Save and apply/reload/restart nginx config
3. In Chrome, visit chrome://serviceworker-internals.
4. Find the domain of your Home Assistant instance, click the Unregister button
5. Clear the browser cache on the domain
6. Revisit Home Assistant

And you should be good to go. You'll need to do step 3-5 on each device that you've visited HA on. Clearing the cache could be enough but I've had mixed results here.

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

[demonspork]

I saw another issue related to this get closed on its own citing "we won't degrade everyone's performance for an edge case" but I did not understand.

Why can't the service worker or something see a 301 response... and then just follow it to authenticate?

If it gets an access denied error of some sort it should still ask for login, but I don't see how allowing it to follow a redirect could be harmful to anyone's experience while at the same time will allow these "edge cases" to work better.

[andriej]

I have just moved and got my HA server out of the box - just moving all configs to newest version. Will see if problem still occures, but still I can't imagine just opening port straight for service withoit nginx in between. Especially that I use it together as a shared port with VPN.

To the point - I've re-registered notifications with HASS and don't want to get this ticket closed without being resolved...

[xentac]

I was able to use the hints in this issue that /local isn't cached and put my auth urls under there (so they always hit the server). Then I wrote some javascript using extra_html_url (from https://github.com/home-assistant/home-assistant/pull/9150) to request the auth url on load. If it comes back as 401, I change window.location to my auth url, bypassing all of the websocket retrying and the broken user/pass auth of home assistant.

Testing the html was the hardest because the service worker caches so aggressively, I had to clear site data every time I changed something so that the changes would be picked up.

It's still not ideal that the code assumes any non-successful websocket connection means we need to pass a username and password.

[andriej]

Just checked; The problem still persists in latest version.

[jurriaan]

Also having this problem, which is probably a bug in Chrome (at least I am having this issue in Chrome) where a 401 does not trigger the authentication. See https://bugs.chromium.org/p/chromium/issues/detail?id=623464

Unfortunately they say it won't be fixed for some time, so I'm also going to disable the service worker.

[andriej]

Now the problem is even worse - translations are getting 401 even with proper authentication...

[avocadosalsa]

I am seeing the same issue @andriej mentioned about the translations triggering a 401 response even with proper authentication.

[linvinus]

hi guys! maybe you have the same problem? https://github.com/home-assistant/home-assistant-polymer/issues/667

[andriej]

@linvinus: I have same problem with translations, but the problem in this issue won't be solved by that PR there.

[andriej]

@xentac does it work for you?

[iliketoprogram14]

I am hitting this same issue in Chrome on both Android and Windows when using basic auth. I get similar symptoms when I try to use client certificates, but that may be unrelated.

A reliable way to reproduce this problem on Android is to do the following:

• Launch Home Assistant in Chrome or some Chromium-based browser.
• Log in, and wait for the page to load.
• Kill the Chrome app from the multi-tasking screen.
• Relaunch the Chrome app.
• Launch Home Assistant, and observe a 401 error.

I really hope this issue does not get buried because proxy auth isn't very popular with HA users at the moment. Maybe basic auth and client certificates behind a proxy aren't popular because they don't work properly out of the box? Most users should try to strengthen their security around their HA installation if they're exposing it to the internet, and frankly, just adding a SSL server certificate and doing nothing else shouldn't be enough. At least getting basic auth working reliably is a good step in the right direction.

[andriej]

It seems like I got it fixed with update to 0.59 - could others please check if it's working for them also? If not, I may have ugly work-around.

[vondruska]

@andriej from what I'm seeing, I think the "source" of the issue has changed. I'm seeing the service worker throw the 401 now instead of the Websocket silently fail on the 401.

Could be running into this Chromium issue now?

But I think I agree, as written, it's fixed in 0.59. But I'll probably leave my nginx deny rules in place since the auth box doesn't pop from the service worker.

[andriej]

Well, you're right. I can't get 'push notifications' working now - registration fails...

[xentac]

Using the html snippet made it possible for me to detect the 302 state and force a redirect before the auth prompt was displayed.

Now though the service worker seems to be caching the 302 for /, which means that any authentication attempts just end up with an unreasonable redirect back to the login page, until I clear storage, which unregisters the service worker.

[xentac]

I'm considering allowing / through without an auth redirect, but I'm worried about data injection. A glance at the index page handler doesn't look like any user-provided url strings are passed through in any way.

[iliketoprogram14]

@andrej I'm still hitting the issue using my repro steps above.

Notification registration is also failing for me 100% of the time in both Android and Windows. Chrome inspector has the following errors when I try to register for notifications:

• GET https://my-site.com/manifest.json 401 ()
• Manifest: Line 1, column 1, Unexpected token. (Note: that's because manifest.json is empty)
• DOMException: Registration failed - missing applicationServerKey, and manifest empty or missing

I think the regression is in the front end repo, as I didn't see any relevant changes between the last working version (0.58.1) and 0.59.1, but then again, I'm new here.

Edit: The regression isn't real, see xentac's response below to allow manifest.json through without authentication.

[xentac]

It looks like /local was changed from networkOnly to fastest, so we can't use that url anymore.

I wrote up a gist explaining how I do authentication that seems to work with the new version (https://gist.github.com/xentac/64e022efee918f704400b6e1b75897b7), I doubt I can make it any more detailed than that. Hope it works for everyone.

[xentac]

@iliketoprogram14 You will have to allow /manifest.json through without authentication for it to work. Pretty sure that's a requirement for html5 notifications. Check my gist for configuration suggestions.

[iliketoprogram14]

@xentac That worked. I'm kind of surprised that I didn't have that problem before. I'll definitely check out your gist. Ultimately, I would like to use client certificates ultimately, and I'll see if I can adapt your oauth gist to work with basic auth first and then maybe client certs. Thanks for helping us out.

[andriej]

@iliketoprogram14 @xentac can you somehow manage to edit the webpage/wiki and help getting - altogether - SSL + proxy + notifications + basic_auth working altogether? I can't manage to get any of config out there working with everything.

[iliketoprogram14]

I gave up and made the switch to SSL+NGINX+oauth2_proxy+notifications. Once I got that working, I haven't had any problems since.

[andriej]

Can you share config(s) except the oauth's secrets? or perfectly make them as a tutorial on website (I can help after get it to work)

[andriej]

or it's rather question to @xentac especially for the nginx file.

-- EDIT: I've managed to do some working config with notifications (will see if they last few hours) Although I can write how-to, but last file I need is something you've described:

I have configured /api/__checkauth to return a 200 or a 404 if authenticated and
a 401 if not authenticated.

in gist. currently I've got 404 with it, but solution works.

[andriej]

@xentac - any chance on getting the file/config in nginx from you?

[dgomes]

Can any of you update the documentation with a solution for this ? Then we could close this issue :)

[andriej]

Well, there's option to handle this error by running in example the oAuth way, but still there's problem with basic-auth method and I didn't find any 'ugly workaround' for this.

[iliketoprogram14]

@dgomes this issue is still technically unsolved with no documented workaround, so it should stay open.

[dgomes]

What about @xentac solution ?

[andriej]

It's totally different, as BASIC AUTH means username and password, while oAuth is logging with external authorization platform (gmail, facebook etc.)

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

[abmantis]

Still an issue.