Getting 400 Bad Request for POST /microsoft/auth-callback/

greglever commented 5 years ago

Django Microsoft Authentication Backend version: 1.1.1
Python version: 3.6.6
Operating System: Ubuntu 18.04 LTS

Description

I ran my local server and navigated to http://0.0.0.0:8000/admin/login/?next=/admin/ in the browser.
ngrok is set up to forward https://ed0c12f9.ngrok.io -> localhost:8000

After I authenticate with my Microsoft credentials I see the following in the logs:

{'code': '...removed for brevity...', 'state': '...removed for brevity...', 'session_state': '...removed for brevity...'}
Bad Request: /microsoft/auth-callback/
[01/Nov/2018 11:14:24] "POST /microsoft/auth-callback/ HTTP/1.1" 400 668

Any ideas what I'm doing wrong ?

greglever commented 5 years ago

on further investigation the context is:

{'base_url': 'https://ed0c12f9.ngrok.io/', 'message': '{"error": "bad_state", "error_description": "An invalid state variable was provided. Please refresh the page and try again later."}'}

Any ideas how I can get my state to not be bad ?

AngellusMortis commented 5 years ago

When I try to use your authentication end point, I am getting an error on Microsoft's side.

 error=unauthorized_client
 error_description=The client does not exist. If you are the application developer, configure a new application through the application management site at https://apps.dev.microsoft.com/

Which means you did not configure the OAuth properly.

greglever commented 5 years ago

so it works fine for me when I use requests_oauthlib:

from django.shortcuts import render, redirect
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import LegacyApplicationClient

class TestLogin(View):
    CLIENT_ID = '*******'
    REDIRECT_URI = "http://localhost:8080/api/2/test-login-callback/"
    AUTHORIZATION_BASE_URL = "https://login.microsoftonline.com/{tenant_id}/oauth2/authorize"
    TENANT_ID = "*******"

    def get(self, request, *args, **kwargs):
        return render(request=request, template_name='ngis_test/login.html')

    def post(self, request):
        # OAUTH STEP 1 - POST as a result of clicking the LogIn submit button
        azure_session = OAuth2Session(self.CLIENT_ID, redirect_uri=self.REDIRECT_URI)
        # do the outreach to https://login.microsoftonline.com/{tenant_id}/oauth2/authorize
        authorization_url, state = azure_session.authorization_url(
            self.AUTHORIZATION_BASE_URL.format(tenant_id=self.TENANT_ID)
        )
        resp = requests.get(authorization_url)
        # go to the login page of NGIS AAD & authenticate
        return redirect(resp.url)

class TestLoginCallBack(View):

    # TODO(Greg): Import these from django.conf settings
    CLIENT_ID = '*******'
    REDIRECT_URI = "http://localhost:8080/api/2/test-login-callback/"
    AUTHORIZATION_BASE_URL = "https://login.microsoftonline.com/{tenant_id}/oauth2/authorize"
    BASE_TOKEN_URL = "https://login.microsoftonline.com/{tenant_id}/oauth2/token"
    TENANT_ID = "*******"
    CLIENT_SECRET = "*******"

    context = {'initialize': ''}
    azure_session = OAuth2Session(CLIENT_ID, redirect_uri=REDIRECT_URI)

    def get(self, request, *args, **kwargs):
        code = request.GET.get("code")
        # OAUTH STEP 2 - go fetch the token
        token_dict = self.azure_session.fetch_token(
            token_url=self.BASE_TOKEN_URL.format(tenant_id=self.TENANT_ID),
            code=code,
            client_secret=self.CLIENT_SECRET,
            resource=self.CLIENT_ID,
        )
        id_token = token_dict.get("id_token")
        plaintext_token = jwt.decode(
            jwt=id_token,
            algorithms=['none'],
        )
        return JsonResponse(plaintext_token)

greglever commented 5 years ago

ie, I can log in and authenticate with the Oauth AD that's been set up

greglever commented 5 years ago

I was just hoping to use django_microsoft_auth to save me having to write a lot of custom code

AngellusMortis commented 5 years ago

I am already using requests_oauthlib on the backend. If you are getting an error from Microsoft, it means you have something configured wrong. If you get an error about an invalid state, it means your CSRF token is expired so refresh the page. I am using Django's CSRF token code to generate the state variables to pass to Microsoft to complete the OAuth and validate the request that comes back.

Microsoft uses standard OAuth, so if you cannot figure out how to get this to work just do it yourself like you are. You are by no means required to us this package. I mostly made it for personal use since I manage to figure out how to get the Xbox Live authentication to work in Python. The Microsoft OAuth is the first step of Xbox Live auth, so that is why they are both bundled together. If you can find a way to make the package better or want to work on the docs some (I have been lazy and not really done too much yet), make a pull request.

greglever commented 5 years ago

thanks for the help @AngellusMortis -- yup I might stick to doing it myself. But if I find anything that might be useful to add here I'll certainly make a PR.

blueshed commented 5 years ago

I am seeing the same behaviour. I'm currently debugging it now. It appears that the check generates an incompatible token against the callback request compared to the one that was generated for the login form.

So I am getting a log that says 'Re-using previously supplied state' with a state token and then a printout of the dict submitted with the callback that contains that state and then the same message 'Re-using previously supplied state', but with a different token.

[DBG] Re-using previously supplied state RxMtubeH7owlQpJzorJgA00taAtnIFAEcFQD72du1G7bgyKZYJzwuHhu1gdXiBzC.
[01/Nov/2018 12:58:58] "POST /admin/login/?next=/admin/ HTTP/1.1" 200 3551
{'code': '...', 'state': 'RxMtubeH7owlQpJzorJgA00taAtnIFAEcFQD72du1G7bgyKZYJzwuHhu1gdXiBzC', 'session_state': '32b9ed4e-18a3-4eb1-b34e-4fe413c12c6c'}
[DBG] Re-using previously supplied state nlMNnuJzHQHGX16CplhFXLqSeuLROtpbLuLZyg9h5kI4mqypMqTVawmMYgJwRsKI.
Bad Request: /microsoft/auth-callback/
[WRN] Bad Request: /microsoft/auth-callback/
        request: <WSGIRequest: POST '/microsoft/auth-callback/'>
    status_code: 400
[01/Nov/2018 13:01:22] "POST /microsoft/auth-callback/ HTTP/1.1" 400 668

Any thoughts would be appreciated.

luskbo commented 5 years ago

I am already using requests_oauthlib on the backend. If you are getting an error from Microsoft, it means you have something configured wrong. If you get an error about an invalid state, it means your CSRF token is expired so refresh the page. I am using Django's CSRF token code to generate the state variables to pass to Microsoft to complete the OAuth and validate the request that comes back.

Microsoft uses standard OAuth, so if you cannot figure out how to get this to work just do it yourself like you are. You are by no means required to us this package. I mostly made it for personal use since I manage to figure out how to get the Xbox Live authentication to work in Python. The Microsoft OAuth is the first step of Xbox Live auth, so that is why they are both bundled together. If you can find a way to make the package better or want to work on the docs some (I have been lazy and not really done too much yet), make a pull request.

Refreshing the page doesn't help.. Definitely think it is the CSRF token that is causing the bad state, but refreshing the page doesn't fix it.

zen4ever commented 5 years ago

I'm having similar issue with Django Zappa with message "Re-using previously supplied state", unfortunately it is unclear how to debug it further

AngellusMortis commented 5 years ago

I cannot help you troubleshoot something without details about your setup. What OS are you using? What Python version are you using? Are you using Django dev server or are you deploying it with a WSGI application server and a HTTP reverse proxy? Are you using HTTPS? What are the steps to reproduce your environment?

zen4ever commented 5 years ago

Sorry @AngellusMortis. My environment is AWS lambda deployment using Zappa. It sits behind AWS API gateway and https. Python version 3.6. I think the issue is with the way Django interacts with API Gateway.

AngellusMortis commented 5 years ago

I unfortunately do not know enough about AWS to help you with that. If you are able to get logs of the network traffic or trace it through AWS, I can probably help you. Shoot me an email (it is on my profile) and we can connect via Discord or something and try to troubleshoot through it if you get something.

zen4ever commented 5 years ago

Thanks @AngellusMortis I'll send you an email with the request headers. I think the issue is that CSRF token somehow is not being read. Probably a misconfiguration on my side.

aviv-ebates commented 5 years ago

This looks like a very generic error, but I nailed at least one version of this to a cookie and CORS.

The final step of the MS oauth (At least at right now) is to POST to the callback URI (/microsoft/auth-callback/). This is done by means of rendering a form and using javscript to form.submit().
CSRF token is normally kept in a cookie. The default setting for Django CSRF cookie is SameSite=Lax.
The spec (draft) defines Lax and Strict as reasonable values for SameSite. I can't find what its says about "no value", but this formal-looking site says that Lax is the default.
Chrome will not send the cookie in the final POST if it's set to SameSite=Lax. It will send it if SameSite is empty.
From Chrome's Status Page, Edge and Safari both ignore SameSite (updated June 2018).
Other OAUTHs (i.e. Google's) finish the setup with a GET, not a POST. I did not check the Chrome policy about SameSite while in Get.
The spec says this about oauth:

Likewise, some forms of Single-Sign-On might require authentication in a cross-site context; these mechanisms will not function as intended with same-site cookies.

I'm not an expert on cookies, oauth, or CSRF, but I assume there are two possible solutions here - either (1) CSRF-exempt the login flow, or (2) make the CSRF cookie super lax w.r.t. SameSite (i.e. make it always be sent). I suspect (2) is what we had prior to SameSite being implemented.

AngellusMortis commented 5 years ago

This is done by means of rendering a form and using javscript to form.submit().

It actually is not. Microsoft is making the POST directly. /microsoft/auth-callback/ disallows GET requests completely.

but this formal-looking site says that Lax is the default.

OWASP is very formal. If you are not familiar with the org, you should read up on them and checkout the OWASP top 10 list they put together.

(1) CSRF-exempt the login flow

This is not an option.

The main underlaying issue is that I still have never seen this behavior. I have now tested this with every major browser on Windows 10, Ubuntu 18 LTS, and Android 9. My main suspicion is that this is actually a Safari only issue, which means I have no way of testing for solving the problem myself as I do not and do not plan to ever own an Apple device. The only in depth details I have seen on this issue have also only been from @zen4ever, which was via Safari.

I do not suspect it is a SameSite issue, because as you said, Safari apparently does not support it and the Django default is Lax, which should allow cookies when going cross domain. I suspect it is one of Apple "privacy" features to prevent tracking cookies, but it is blocking a legitimate cookie.

First and foremost, I need a minimal set of steps to reproduce this behavior. Steps to reproduce meaning exactly how you set up the site in the way you did and what browser(s) you used on which OS. If someone with a Mac can verify it is a Safari only issue, that would help a ton. Also, if you set up a test site and want to email the URL so I can see if I can reproduce the issue on your site, that would be great.

Without steps to reproduce, my best guess on possible ways to fix it would be one of the following (feel free to make an issue and a PR if you actually find one of these to work):

Change microsoft_auth.views.AuthenticateCallbackView to use get instead of post and remove response_mode="form_post" from microsoft_auth.client.MicrosoftClient. I originally did the POST here because it was working just fine for me on all of the browsers I tested and it is more secure as there is less of a risk of leaking the authentication code in some way via the GET params in the request.
Remove the requirement for the CSRF cookie. As an alternative, you will have to store a CSRF token in the some sort of temporary storage in mcirosoft_auth.context_processors.microsoft and then pull it back out of the temporary storage in microsoft_auth.views.AuthenticateCallbackView._check_csrf. Sessions will likely work, but might be an issue if someone wants to use a cookie based session backend. If you do not use sessions, you will have to find a way to map the CSRF token back to the originating requester, most likely via IP + user agent getting stored with the CSRF token.
Add support for (in addition to) for the whole flow to happen in a single browser window via redirects. I originally implemented this in this a two window flow on purpose as I preferred it that way. You do not lose your place on the originating site and all of the Microsoft authentication requests are in a separate window. It makes the boundaries much more clear between the originating site and Microsoft's site and it is in line with how I have seen many other sites do Microsoft OAuth as well. Ideally this would be switch via a Django setting to choose between which of the two modes you want to use and if it can be shown this is a Safari only issue, you can use user agent parsing to make sure Safari always uses the redirect flow if the mutli-window flow is chosen for other browsers.

I am locking this issue for further conversation. Please open a new issue with detailed steps to reproduce, including minimal Django site setup instructions and/or a PR with one of the three above solutions if you can verify one of them work. Also, all of these issues are unrelated to Greg's original issue so we can stop adding this this issue, which have since been solved.

AngellusMortis commented 5 years ago

Good news, @zen4ever and @aviv-ebates. I finally had this issue happen to me. It started happening as soon I made a second log in page that used the Microsoft authentication backend (one that was not under /admin). I do not know why it started happening all of a sudden, but I happened to have a good idea on how to improve the state validation without any really extra work as I thought it would take before (points I outlined above).

State validation now takes your current CSRF token and signs it with Django's cryptographic signer. As long as the signature on the state can be verified by Django and the state was generated in the last 5 minutes, validation will pass. This should hopefully remove any changes of this random bad state validation.

Please updated to 1.3.3 to get these changes. And thanks for the patience it likely took to deal with me trying to figure this out.

AngellusMortis / django_microsoft_auth

Getting 400 Bad Request for POST /microsoft/auth-callback/ #128

Description