New login issue (Was working fine for months)

ReessKennedy commented 2 years ago

The issue

I've been using this code successfully for the past few months (thank you!) and then it suddenly stopped working. I am getting the error posted below. I am wondering if this is the result of security changes at Google? Or what I am overlooking in my attempt to debug.

My attempts at solving

Two factor turned on
App pass created using this link: https://myaccount.google.com/apppasswords ... then I copy and paste that pass into the sample code below.
Running recent version of Python3 (3.8) and receive same error on local dev and remote
Manually unlocked captcha using following link and have tried this many times UnlockCaptcha manually
I am logged into GoogleKeeo in my browser.

I tried turning off two-factor and reseting and then creating a new app pass but that produced same error. Again, any help is appreciated as I depend on this for my workflow.

Error

File "/Users/Me/Projects/gkeep/test.py", line 4, in <module> success = keep.login('me@gmail.com', 'apppassword') File "/usr/local/lib/python3.9/site-packages/gkeepapi/__init__.py", line 695, in login ret = auth.login(username, password, device_id) File "/usr/local/lib/python3.9/site-packages/gkeepapi/__init__.py", line 59, in login raise exception.LoginException( gkeepapi.exception.LoginException: ('NeedsBrowser', 'To access your account, you must sign in on the web. Touch Next to start browser sign-in.')

Running this code

`import gkeepapi

keep = gkeepapi.Keep() success = keep.login('myemail@gmail.com', 'workingapppasswork')

note = keep.createNote('Todo', 'Eat breakfast') note.pinned = True note.color = gkeepapi.node.ColorValue.Red keep.sync()`

kiwiz commented 2 years ago

Possibly - here's some things you can try:

Downgrading the requests version (See https://github.com/kiwiz/gkeepapi/issues/81)
Testing with another account
Testing from a new IP

While it won't help with the current situation, I also recommend caching the login authentication data like so: https://github.com/kiwiz/gkeepapi/blob/master/examples/resume.py. This means less authentications, which means Google is less likely to mark the requests as suspicious.

Beubus commented 2 years ago

I encountered the same issue, while it was working fine for more than one year. Two factor is off on my side.

What i tried and doesn't work :

test with another account
test with a new IP
test on Linux & Python 3.8.10
test on Windows & Python 3.10.4

Is it working for someone or the gkeepAPI is currently no longer compatible with Google Keep ?

kiwiz commented 2 years ago

Just tried reauthenticating (I've been using cached auth data) and also get a "NeedsBrowser" response. :/

I did some digging into the browser flow as mentioned here: https://github.com/simon-weber/gpsoauth/issues/13. This results in a oauth_token cookie, but that doesn't appear to be compatible with the endpoint that retrieves an OAuth token for Google Keep. Unfortunately, this means that this library will be broken until the token exchange is figured out.

bergercookie commented 2 years ago

Hi @kiwiz ,

Unfortunately, this means that this library will be broken and until the token exchange is figured out.

Could you give a bit more details as to what needs to be fixed? The issue you linked to is open since 2016 so I'm not sure how it's related to this one.

bergercookie commented 2 years ago

Putting a breakpoint in gkeepapi/__init__.py after the gpsoauth.perform_master_login call, and printing the content of the request shows a Url to use for sign-in via the browser.

Pasting this URL in your browser walks you through a two-factor authentication. At the end of this, though, you end up with a page that has a spinner and a "One moment please..." message which never goes away. Repeating the `authentication, leads to the same result.

kiwiz commented 2 years ago

gkeepapi relies on being able to get a "master token", which is used to generate an OAuth token with the necessary Google Keep scopes.

When you get a NeedsBrowser error and complete the authentication flow a cookie (oauth_token) is set on your browser. On Android, what probably happens is that the cookie is retrieved and used to generate OAuth tokens for individual services. We would need to reverse engineer the relevant code in Google Play Services to determine how that happens.

EthanArbuckle commented 2 years ago

I've reached the same conclusion. If you snoop a real device's traffic and grab the AAS token, it can be hardcoded here and playstore token generation will then work. Some RE work on what has changed regarding AAS token generation is needed -- i'm looking into this and will share my findings.

Is Aurora broken too? They usually have up-to-date implementations of the auth mechanisms

kiwiz commented 2 years ago

Good call on checking out Aurora. It looks like the new flow might already be implemented here: https://gitlab.com/AuroraOSS/AuroraStore/-/blob/master/app/src/main/java/com/aurora/store/view/ui/account/GoogleActivity.kt#L80-92

kiwiz commented 2 years ago

... and the implementation is here: https://gitlab.com/AuroraOSS/AuroraStore/-/blob/master/app/src/main/java/com/aurora/store/util/AC2DMTask.kt#L27-43. From a quick look, it looks like we just need to set ACCESS_TOKEN. gpsoauth just needs to be updated to support this new flow here: https://github.com/simon-weber/gpsoauth/blob/master/gpsoauth/__init__.py

If anyone wants to take a stab at this, feel free. Otherwise, I'll take a look at it when I have time.

EthanArbuckle commented 2 years ago

nice find. the ac2dm flow is new to me and i'll need some time to incorporate and test it. I can't open a PR on this project for work-related reasons but I can privately share snippets of the implementation with you (assuming I get something working).

CyberAltra commented 2 years ago

Hey guys, I've ran into this issue on another project that uses gpsoauth. I've managed to work around this in a quick and dirty way by awaiting a auth code input, opening a browser tab with the url from the error, and then taking the auth token from your browser cookie to exchange for a master token using a simple http request.

Here's an example request that I made by intercepting my own device's auth flow, it can probably be stripped down further and implemented better into gpsoauth.

https://android.googleapis.com/auth?androidId=&lang=en-US&google_play_services_version=221514037&sdk_version=30&build_product=redfin&build_brand=google&Token=&build_device=redfin&service=ac2dm&get_accountid=1&ACCESS_TOKEN=1&callerPkg=com.google.android.gms&add_account=1&callerSig=38918a453d07199354f8b19af05ec6562ced5788

ReessKennedy commented 2 years ago

Happy I am not alone here and that my sharing yielded some shared expertise here.

Failures with official GoogleKeep API

Hoping this results in a fix as I was never able to get the official GoogleKeep API that was announced/released working and my conclusion was that the necessary scopes for the official are actually only available if I have an enterprise Google account, which I don't have.

Praise for Documentation for this project

Meanwhile, I am pretty invested in the amazing work @kiwiz here and all. The ReadTheDocs documentation here is excellent! Some great code on Github but great documentation for this code is rare.

oAuth grief

In general I get the oAuth might be more secure but it's also annoying.

kiwiz commented 2 years ago

https://github.com/simon-weber/gpsoauth/pull/41

awkj commented 2 years ago

I try fix gpsoauth code in my site-pacakge, but not work, is have to chagne gkeepapi code ?

[INFO] ('NeedsBrowser', 'To access your account, you must sign in on the web. Touch Next to start browser sign-in.')

ejpenney commented 2 years ago

gpsoauth released a new version this morning (1.0.1) addressing this. Is there anything other than updating the dependency that needs to happen here?

felixschndr commented 2 years ago

New Release and Commit that fixes the issue

I'll try to update the dependency manually and see if it fixes the issue

Update: I currently have a functioning python3 env, my pip broke. I have to figure out what happend there first. Maybe someone else can check if the new version of gpsoauth works

ejpenney commented 2 years ago

I use ha-google-home which has a dependency on gpsoauth, their update forced the upgrade on my instance. gkeepapi is working for me now with gpsoauth 1.0.1 in my environment.

Not a super clean test, but it looks good from here!

newbprogrammerpi commented 2 years ago

I updated gpsoauth and scripts are working now.

kiwiz commented 2 years ago

Thanks for testing. I've bumped the dependency on gpsoauth to 1.0.2 and published 0.14.0, which should fix the issue.

felixschndr commented 2 years ago

https://github.com/kiwiz/gkeepapi/issues/123#issuecomment-1142441004 https://github.com/kiwiz/gkeepapi/issues/123#issuecomment-1144051670

Can confirm: works now perfectly!

homopoluza commented 2 years ago

Damn. No luck for me. All worked fine in April. I use gpsoauth 1.0.2 and gkeepapi 0.14.2.

Saw this in Google Account Help page: To help keep your account secure, from May 30, 2022, Google no longer supports the use of third-party apps or devices which ask you to sign in to your Google Account using only your username and password.

felixschndr commented 2 years ago

Try using an app password which can be created here instead of a password. I just checked and gpsoauth 1.0.2, gkeepapi 0.14.2 on python 3.10.0 works

homopoluza commented 2 years ago

Try using an app password which can be created here instead of a password. I just checked and gpsoauth 1.0.2, gkeepapi 0.14.2 on python 3.10.0 works

I did that. I did all as topic starter did. Again all worked just fine in April with python 3.10. I upgraded gpsoauth and gkeepapi today but no luck. I honestly don't know what to do else.

homopoluza commented 2 years ago

There is my simple code if it helps.

def keep_update(): keep = gkeepapi.Keep() login = keep.login('my@email', 'app_password')
active_notes = keep.find(archived = False, trashed = False) text = '' for note in active_notes: st = str(note) text += st + '\n' keep_label.config(text=text, justify='left', foreground='white', background='black', font=90) keep_label.after(3600000, keep_update)

gkeepapi.exception.LoginException: ('NeedsBrowser', 'To access your account, you must sign in on the web. Touch Next to start browser sign-in.')

felixschndr commented 2 years ago

Does the error occur at keep.login(...)?

felixschndr commented 2 years ago

I use the same code

import gkeepapi
keep = gkeepapi.Keep()
login = keep.login(...,...)

which works

Can you assure that you are using the correct versions (i.e. in the python interactive mode)? Sometimes python does some funky stuff with its modules versions...

homopoluza commented 2 years ago

Does the error occur at keep.login(...)?

yes, it does.

I use the same code
import gkeepapi
keep = gkeepapi.Keep()
login = keep.login(...,...)
which works

Can you assure that you are using the correct versions (i.e. in the python interactive mode)? Sometimes python does some funky stuff with its modules versions...

versions are correct

I gave up and switched to simplenote

oluc commented 2 years ago

Same here, I fail to login

$ pip install gkeepapi
(...)
Successfully installed gkeepapi-0.14.2 gpsoauth-1.0.2

$ python
Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import gkeepapi
>>> keep = gkeepapi.Keep()
>>> keep.login('user.name@gmail.com', 'xxxxxxxxxxxxxxxx')

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/.local/lib/python3.10/site-packages/gkeepapi/__init__.py", line 697, in login
    ret = auth.login(email, password, device_id)
  File "~/.local/lib/python3.10/site-packages/gkeepapi/__init__.py", line 62, in login
    raise exception.LoginException(res.get("Error"), res.get("ErrorDetail"))
gkeepapi.exception.LoginException: ('BadAuthentication', None)

I tried with either my google password, either with a created "app password" as instructed at [1]

[1] https://support.google.com/mail/answer/185833

any clue? how can I proceed to move forward? help warmly welcome 🙏❤️

felixschndr commented 2 years ago

It is a mystery to me why it doesn't work on your machine...

┌─[22:12:48]-[:)]-[openhabian@openhab]-[/etc/openhab2/ (master)]
└──> python3
Python 3.10.0 (default, May 31 2022, 22:11:26) [GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gkeepapi
>>> keep = gkeepapi.Keep()
>>> keep.login('xxxx@gmail.com', 'yyy')
True
>>>

┌─[22:14:22]-[:)]-[openhabian@openhab]-[/etc/openhab2/ (master)]
└──> pip show gpsoauth
Name: gpsoauth
Version: 1.0.2
...
Requires: pycryptodomex, requests
Required-by: gkeepapi

┌─[22:33:48]-[:)]-[openhabian@openhab]-[/etc/openhab2/ (master)]
└──> pip show gkeepapi
Name: gkeepapi
Version: 0.14.2
...
Requires: future, gpsoauth
Required-by:

┌─[22:34:59]-[:)]-[openhabian@openhab]-[/etc/openhab2/ (master)]
└──> pip freeze | grep 'gpsoauth\|pycryptodomex\|requests\|future\|gkeepapi'
future==0.18.2
gkeepapi==0.14.2
gpsoauth==1.0.2
pycryptodomex==3.15.0
requests==2.28.0

The only small difference I see is the python version (3.10.4 vs 3.10.0)

oluc commented 2 years ago

Hi, thank you so much for taking the time!

I can try downgrading python, if that's possible, would this make sense?

Maybe another difference : I'm running a Raspberry pi 400 (with Ubuntu 22.04), could this have any effect?

Also, I have a very poor internet connection, would a long answer delay be taken as failure?

How could I trace what's happening precisely?

kyrlon commented 1 year ago

I am not sure if anyone else is coming across this issue but once I attempt to login I get this Exception error:

raise exception.BrowserLoginRequiredException(res.get("Url"))
gkeepapi.exception.BrowserLoginRequiredException: https://accounts.google.com/signin/continue?sarp=1&scc=1&continue=https://accounts.google.com/o/android/auth?hl%3Den_us%26xoauth_display_name%3DAndroid%2BLogin%2BService%26source%3DAndroid%2BLogin&plt=AKgnsbt9-wth_fRL5fQNWo_0H9cGfSE2Eaw0NwnE5HoNy8m9Z8w92dzGO5_SX0LaXZ1GgXRLhkf0Y8xP1xuwUjkJXdTVR7To13KJXW4joooC4k9RlJV4jEsTnMhSCa5rVz5rLMW-97cr

I click on the link, and after prompted to login, it is at an endless loop on the page saying "One moment please" that looks like this even after an hour:

UPDATE: I forgot that I used an App Password. Generating a new one solved this endless loop.

csingh27 commented 1 year ago

An app password needs to be generated. This is the solution.

ije896 commented 1 year ago

Hey y'all, I know this is a separate project, but I'm really lost at this point. I'm working with keep-it-markdown which uses this repo and I keep getting LoginException(BadAuthentication, None) when I try to login. I've gone through the process of triple-checking:

my default browser setting is correct
I'm only signed into my default google account
clicking the security captcha link in the default browser
generating an app password each time I try to login with 2FA on
disabling 2FA and using my regular password
restarting computer and terminal
I'm using the correct login information (I'm able to login on other apps with a generated app-password AND my regular password)
I'm on the latest version of gkeepapi (and all dependencies)
tried running with multiple google accounts, same issue across the board
tried using older versions of the requests lib

Running on macOS 12.6.1 and using python 3.9.6, so I really don't get what gives here. Any help is greatly appreciated.

birdpump commented 1 year ago

Generating a google app password solved the issue

ilesm commented 11 months ago

I have this BadAuthentication problem also (OS X 13.5.1, Python 3.11.5, gkeepapi 0.14.2, gpsoauth 1.0.2, requests 2.31.0, Google 2FA with app password). I had no luck downgrading Python, requests, etc. as others describe. In the end I'm successfully able to use https://hub.docker.com/r/rcdeoliveira/gkeepapi (which simply uses latest Python3 on Ubuntu with requests 2.23.0) to run my code that uses gkeepapi.

nygma2004 commented 9 months ago

I am reading this now thinking that the issue is resolved. I am sing your work, because I wanted to integrate Google Keep Todo list to MagicMirror. I installed everything today, but getting the same issue as @kyrlon mentioned above. I generated a new app password, but I get the gkeepapi.exception.BrowserLoginRequiredException error and after logging on to the URL in the error message, "One moment please..." just keep spinning. Did I miss something?

airosB commented 9 months ago

According to my investigation, this login issue caused by gpsoauth. gkeepapi uses gpsoauth internally.

The sample code below caused NeedsBrowser gpsoauth error.

from gpsoauth import perform_master_login, perform_oauth

def getToken(email, password, android_id):
    response = perform_master_login(email, password, android_id)
    if "Token1" not in response:
        print("Master login failed.")
    if response.get("Error") == "NeedsBrowser":
        print(response.get("Url"))
    print(response["Token1"])

getToken('********@gmail.com', 'password', '3ee9002270d00157')

Speculation: google's specification change?

nygma2004 commented 9 months ago

@airosB find is strange that Google makes accessing Keeps so difficult compared to some of their other products. I was wondering if I should look for a workaround, like somehow backing up the Todo items in Keep to a Google Sheet and see if I can access that any easier. After all I only need to see the items in MagicMirror, no need to update them externally.

kiwiz / gkeepapi