Error: invalid_scope from oauth2

[BartGarciaNathan]

Hey, I just installed Hangups and when trying to set it up, when I try to open the webpage url given by hangups I get this message :

400. That's an error Error: invalid_scope Request Details scope=https://www.google.com/accounts/OAuthLogin response_type=code redirect_uri=urn:ietf:wg:oauth:2.0:oob client_id=936475272427.apps.googleusercontent.com

The recently used devices list by google does not show anything 'new' that could be hangouts. Am I doing something wrong? Do you need any other info? I am running Linux 4.1.31-1-MANJARO

Any help will be much appreciated

[EionRobb]

I've just had 2 people trying to use purple-hangouts report the same thing. There's probably a new scope needing to be requested :s

[KwisatzHaderach]

Changing scope in hangups/auth.py to https://www.googleapis.com/auth/plus.login gets you the token, but KeyError: 'SAPISID' happens.

Apparently we need 3 scopes according to this: https://developers.google.com/+/hangouts/running This is still not returning SAPISID cookie. Bloody google.

[gravitypriest]

Interestingly, this showed up on my Gmail a little earlier

[dequis]

This didn't work for me with purple-hangouts, but mentioning it for the sake of documenting what was tried, the android app requests these scopes to https://android.clients.google.com/auth, separated by spaces and prefixed by oauth2:

https://www.googleapis.com/auth/chat
https://www.googleapis.com/auth/plus.me
https://www.googleapis.com/auth/plus.peopleapi.readwrite
https://www.googleapis.com/auth/hangouts
https://www.googleapis.com/auth/identity.plus.page.impersonation
https://www.googleapis.com/auth/chat.native

Specifically, it fails at https://accounts.google.com/accounts/OAuthLogin?source=pidgin&issueuberauth=1 with status 403 and Error=badauth in the body.

I guess I need an iOS device to figure this one out.

[dequis]

Forgot to mention, out of those scopes, the last two are rejected as invalid_scope by the oauth method we use:

https://www.googleapis.com/auth/identity.plus.page.impersonation
https://www.googleapis.com/auth/chat.native

The second one is recognized by the error page as A native application: iOS Account Manager, which is why I said we should look at the iOS app instead (also that the client_id seems to be used by other iOS google apps)

The Error=badauth stuff I got was after getting a token that didn't have those scopes, which seems obvious in hindsight.

[ghost]

Hi! Did anyone found solution for this problem? I am using scope "https://www.googleapis.com/auth/plus.login" and after that get SAPISID cookie, but request to "https://account.google.com/MergeSessions"doesn't work :( So I can't use multi-account because can't get cookie after each re-login (refresh token) :(

[eddielu]

It looks like this is an issue for third party Hangout clients: https://productforums.google.com/forum/#!topic/hangouts/LU2YZNDhN_s

[tdryer]

Might be related to this announcement: Modernizing OAuth interactions in Native Apps for Better Usability and Security

One possible workaround is to revert to the old pre-OAuth login method (simulating a browser login): https://github.com/tdryer/hangups/blob/82786f40567acf811cc83340162e7bfc2e1dfa65/hangups/auth.py

[tdryer]

I tried logging into Hangouts for iOS while running mitmproxy, but Google blocks this now (probably certificate pinning). It did capture this one response which contains some OAuth details:

https://accounts.google.com/embedded/setup/ios?scope=https://www.google.com/accounts/OAuthLogin+https://www.googleapis.com/auth/userinfo.email&client_id=936475272427.apps.googleusercontent.com&delegated_client_id=183697946088-m3jnlsqshjhh5lbvg05k46q1k4qqtrgn.apps.googleusercontent.com&hl=en-US&device_name=iPad&auth_extension=ADa53XJhK96KggHR83VRjcIJuKYWqPQPMmAUB9gG7qDOz8WkZ45RmsGC30ftQLlvctIoWPTNE-oqvvtAxbRWv41tFM7NqoNLl0OChtAWkFoE6xHujd3DwDU&nosignup=1&system_version=9.3.4&app_version=11.5.129049038

[davibe]

Apparently you CAN request that scope to https://accounts.google.com/embedded/setup/ios but not to https://accounts.google.com/o/oauth2/auth

[davibe]

@tdryer i tried your url in a browser, inspected calls but i can't find a code

[dmunafo]

is there any update on this one? Was this issue root cause? Any workaround?

[avaidyam]

@tdryer Is there a way to check Web Hangouts instead? Perhaps that'd be easier.

[dmunafo]

Can it be something related to the client ID and secrets?

[Cbhihe]

Same issue here as posted by @BartGarciaNathan, for the last 6 days as of 2016.09.07.

[Pidgin 2.10.9; pidgin-hangouts 1.0; purple-hangouts 1.0] [Ubuntu Desktop 14.04.4 - 3.16.0-50-generic x86_64 GNU/Linux)]

[JJLeMay]

@tdryer If you are still interested in inspecting that iOS app's traffic, Check this out: https://github.com/nabla-c0d3/ssl-kill-switch2

It will disable ssl certificate validation allowing you to MiTM and inspect traffic. It requires root on the device, however.

[tdryer]

I haven't had a chance to work on this. If anyone is interested in helping out, I'd suggest trying to update the old pre-OAuth login method: https://github.com/tdryer/hangups/blob/82786f40567acf811cc83340162e7bfc2e1dfa65/hangups/auth.py

[dequis]

Got a workaround here!

Using one of the urls linked above, you can get to a programmatic_auth url that sets the oauth code to a cookie. This is somewhat awkward to use, but certainly possible and better than being locked out.

I've made a video explaining how to get it:

https://www.youtube.com/watch?v=hlDhp-eNLMU

It's 41 secs long, no sound, with annotations. Used chrome but firefox is pretty much the same thing.

The programmatic_auth url:

https://accounts.google.com/o/oauth2/programmatic_auth?hl=en&scope=https%3A%2F%2Fwww.google.com%2Faccounts%2FOAuthLogin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email&client_id=936475272427.apps.googleusercontent.com&access_type=offline&delegated_client_id=183697946088-m3jnlsqshjhh5lbvg05k46q1k4qqtrgn.apps.googleusercontent.com&top_level_cookie=1

Text version:

1. Go to that url
2. Enter your username, click next.
3. Right click page background, inspect
4. Go to the network tab.
5. Enter your password, click sign in
6. Click the first row in the network panel, the one that says "programmatic_auth"
7. Scroll down in the right-side panel, find "set-cookie"
8. Your code should be there, after "oauth_code=", up to but not including the semicolon.
9. Copy it and use it.

[Ezwen]

@dequis Wow, this indeed works. Thanks!

[mydongistiny]

Where do I put the code once I got it?

[Finlay87]

Copy and use it... Where?

[dequis]

In the same place where you'd put the code after clicking the url that throws the invalid_scope error.

I'm being rather generic here because I don't use hangups, but purple-hangouts. The method is pretty much the same for any client that displays the url.

If you're using a hangups derivative that doesn't ask for the code after displaying the url, but uses an embedded browser, ask the devs of that project for help.

In the case of yakyak this comment says how to use it it's fixed in 1.3.2 and there's no need to do this.

[Finlay87]

Thank you good sir.

[danhimalplanet]

I got it working with libpurple hangups + bitlbee

[mydongistiny]

@dequis Thanks man. I got it working in Pidgin until I can figure out the right syntax for yakyak.

[turturel]

Thanks @dequis!

One thing to note tho, make sure you are logged out from any Google account before trying the URL. I wasn't and I waited a couple of minutes before realising :))

[avaidyam]

@turturel Not necessarily; I found the oauth code just fine when I used it and I was logged in.

[turturel]

For me it didn't work until I logged out of GMail... Anyway if someone notices that it takes the page a long time to load (a "please wait" message is displayed) you can try logging out of GMail - that's what did it for me.

On Tue, Sep 13, 2016 at 5:06 PM, Aditya Vaidyam notifications@github.com wrote:

@turturel https://github.com/turturel Not necessarily; I found the oauth code just fine when I used it and I was logged in.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tdryer/hangups/issues/260#issuecomment-246692482, or mute the thread https://github.com/notifications/unsubscribe-auth/AVJYwaiUpmB8qnIYu2cpcMkvQIsQZzxkks5qpq30gaJpZM4Jwu12 .

[willSGN]

@turturel I had the same issue, the trick is watching the network tab before loading the URL.

[davibe]

I applied this to yakyak, seems to work fine. Thanks a lot @dequis you've been amazing :)

[CRThaze]

@dequis link worked for me, though since I was already logged in, I just needed to have the network tab open before loading the page (or refreshing it)

[tdryer]

Nice find, @dequis! That should work nicely for graphical clients with an embedded browser.

Unfortunately for hangups we still need a way to get the code out of the browser.

[james-lawrence]

I'm curious what the values in the url are from? delegate_client_id in particular attempting to use this without a browser and getting 404. The requested URL was not found on this server.

[jornis]

@mydongistiny I added "return rs([oauth_code_from_cookie]);" just below "var code, scr, title, url;" in app/login.js. Then started Yakyak. (I'm on Mac...)

@danhimalplanet How?! Please share?

[yohanes-ai]

@dequis Hey, you made my day! Thanks man.

[cdellin]

I wrote a little Python script to automate @dequis's sweet workaround. If you sign on correctly in the window that pops up, it will dump the oauth_code found to the terminal. Works for me in Debian.

https://gist.github.com/cdellin/6b9bae1310f2ee6b47ba3663bdcb1ded

[davidgiven]

@cdellin Your script is just returning no auth_codes found for me. Any ideas?

[hastebrot]

@dequis Thanks for this workaround (nice video; that was TEH URN)!

It seems we can remove access_type, delegated_client_id, top_level_cookie, and the part with userinfo.email in scope from the OAuth 2.0 URL. It worked for my hangupsjs chat client, which uses only a few functions. I did not test this with Pidgin.

https://accounts.google.com/o/oauth2/programmatic_auth?hl=en&scope=https%3A%2F%2Fwww.google.com%2Faccounts%2FOAuthLogin&client_id=936475272427.apps.googleusercontent.com

Regarding the client_id (936475272427), which was also previously used. It seems to generally refer to some types of Apple devices (iPhone, iPad).

Update: When using this URL make sure you're not logged into any Google account, or it will show you the page after sign-in directly instead of the sign-in form.

[maxemillianian]

@hastebrot When I click the link in your post (or when click the link from @dequis Youtube video), the resulting new-page never progresses beyond the 'One moment please' loading splash.

I've tried on different browsers (Firefox and Chromium), and the same issue is present.

Any ideas?

Thanks!

[hastebrot]

@maxemillianian You need to log out your account in the browser first (or use an incognito window), as already mentioned in this issue:

make sure you are logged out from any Google account before trying the URL

There is also a nice blogpost that analyses this authentication approach: https://www.essabalochi.com/how-google-instagram-and-linkedin-authenticate-users-in-their-native-mobile-apps/

[maxemillianian]

Ah my bad, apologies - don't know how I missed that!

Thanks a lot, working now.

[tdryer]

The new login method is ready for testing in master. It's based on the programmatic_auth solution suggested by @dequis, and works by simulating a browser submitting login credentials into the form. If anyone encounters an error please share the stacktrace, which is printed when logging in with this command:

python -m hangups.auth

[AlexandrePTJ]

Hi,

Right after the verification code, here is what I get:

Traceback (most recent call last):
  File "/home/alexandre/devel/hangups/hangups/auth.py", line 132, in get_auth
    raise GoogleAuthError("Refresh token not found")
__main__.GoogleAuthError: Refresh token not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/alexandre/devel/hangups/hangups/auth.py", line 183, in submit_form
    form.select(selector)[0]['value'] = value
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/alexandre/devel/hangups/hangups/auth.py", line 329, in <module>
    print(get_auth_stdin('refresh_token.txt'))
  File "/home/alexandre/devel/hangups/hangups/auth.py", line 64, in get_auth_stdin
    return get_auth(CredentialsPrompt(), refresh_token_cache)
  File "/home/alexandre/devel/hangups/hangups/auth.py", line 138, in get_auth
    session, credentials_prompt
  File "/home/alexandre/devel/hangups/hangups/auth.py", line 227, in _get_authorization_code
    {VERIFICATION_CODE_SELECTOR: verfification_code}
  File "/home/alexandre/devel/hangups/hangups/auth.py", line 186, in submit_form
    'Failed to find input {!r} in form'.format(selector)
__main__.GoogleAuthError: Failed to find input '#totpPin' in form

[tdryer]

@AlexandrePTJ: Added some more logging. Can you try again after pulling master and check for log output like this after entering the verification code?

INFO:__main__:Submitting form on page 'https://accounts.google.com/signin/challenge/totp/2'
INFO:__main__:Page contains forms: ['challenge', None]
INFO:__main__:Page contains inputs: ['challengeId', 'challengeType', None, None, None, None, 'gxf', 'totpPin', 'submit', 'trustDevice']

[AlexandrePTJ]

Here it is:

INFO:__main__:Submitting form on page 'https://accounts.google.com/signin/challenge/ipp/2'
INFO:__main__:Page contains forms: [None, None, 'challenge', None]
INFO:__main__:Page contains inputs: ['challengeId', 'challengeType', None, None, None, None, 'gxf', 'idvPreregisteredPhonePin', 'submit', 'trustDevice']

[tdryer]

OK, phone verification codes should also work now.

[AlexandrePTJ]

I'll try later this day, thanks for the great work.

[rozzin]

@tdryer writes:

The new login method is ready for testing in master. It's based on the programmatic_auth solution suggested by @dequis, and works by simulating a browser submitting login credentials into the form.

A couple of issues with this approach:

• Reading The Google Dev Blog's "Modernizing OAuth interactions in Native Apps for Better Usability and Security" post, it really sounds like they're intent on trying to make this not work in the near future
• Even if Google doesn't manage to prevent this from working, it seems to rely on a data-path that's unacceptable for some applications, e.g.: I run a Hangouts/XMPP gateway based on https://github.com/ZeWaren/jabber-hangouts-transport; I really don't want to have people sending their passwords through my server, and my users don't want to have to give me their passwords either.

[dequis]

@rozzin Yeah, we don't want to ask users for their passwords either, but google isn't giving us any other option. It's either that or messing with a web browser like the app does. As far as I know, the client id for the hangouts iOS app is configured on the server to only allow requesting the scopes we need through this method - if there's anything else, we don't know about it.

(The other apps, android/chrome/website, use even worse auth methods)

it really sounds like they're intent on trying to make this not work in the near future

I don't see anything in that post suggesting that, if anything, it refers to the method that already stopped working

[tdryer]

I don't see anything in that post suggesting that, if anything, it refers to the method that already stopped working

They want to stop apps from using embedded browsers for logging in, which is exactly what hangups is doing now.