Python errors and login problem

[kalico1]

I am having a hard time getting this installed. Seems to be a problem with python but every attempt to correct it has failed. Initially it was just telling me that gkeepapi was not installed, but everything I do to fix it (including trying to install gkeepapi independently of requirements.txt) seems to make it worse. Here's where I stand now....hope someone can help.

The following happens whether I use pip or pip3:

kalico@MacBook-Air keep-it-markdown-0.4.3 % pip install -r requirements.txt 
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = ':/Users/kalico/Library/Python/3.8/bin'
  program name = '/Library/Developer/CommandLineTools/usr/bin/python3'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = '/Library/Developer/CommandLineTools/usr/bin/python3'
  sys.base_prefix = '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8'
  sys.base_exec_prefix = '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8'
  sys.executable = '/Library/Developer/CommandLineTools/usr/bin/python3'
  sys.prefix = '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8'
  sys.exec_prefix = '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8'
  sys.path = [
    '',
    '/Users/kalico/Library/Python/3.8/bin',
    '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip',
    '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
    '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 914, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1342, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1311, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1275, in _path_importer_cache
PermissionError: [Errno 1] Operation not permitted

I set the following in my .zhsrc

export PATH="${PATH}:/Users/kalico/Library/Python/3.8/bin"
export PYTHONPATH="${PYTHONPATH}:/Users/kalico/Library/Python/3.8/bin"

I'm assuming something is wrong with my Python setup but I'm not sure exactly what.

[djsudduth]

This looks suspicious-> Python path configuration: PYTHONHOME = (not set)

[djsudduth]

Take a look at this

https://stackoverflow.com/questions/65184937/fatal-python-error-init-fs-encoding-failed-to-get-the-python-codec-of-the-file

[kalico1]

Ok, that helped! I just removed the entries in .zshrc and it was fine. (I did that because of an earlier error and a different thread on stackexchange....not going to try to figure out why it borked everything.)

But....now I've got a new problem: I can't log in to my account.

kalico@MacBook-Air keep-it-markdown-0.4.3 % python3 keep-test.py

Enter your Google account username: **********@gmail.com
Enter your Google Password: 
Login attempt failed
 Technical Error Message: AttributeError("attribute 'post_handshake_auth' of '_ssl._SSLContext' objects is not writable")
Please start your browswer and copy-paste this URL in the address bar: https://accounts.google.com/DisplayUnlockCaptcha - then, try logging in again.

I have followed the instructions to copy/paste that URL and it said my account was unlocked, but it still failed with the same message.

So I followed the next step advised by google to create an app password. Still the same message as above. (Tried twice with a diff app password each time.)

So then I also tried this from another user upgrade keyrings, but still no joy.

I've got my Google account open in Chrome, not logged in as any other user, I'm on a MacBook Air (M1, 2020), and I've got Python 3.8.

Any thoughts?

[kalico1]

I also discovered a security check email in my inbox to confirm it was me. I generated a new app password, confirmed via the email link BEFORE using it, but still got the same message as above.

[djsudduth]

See if this helps:

“I moved to a linux system today and had issues, seems the default/latest gpsoauth (0.4.3) which was installed with gkeepapi has issues. I changed to use a previous gpsoauth (pip install gpsoauth==0.4.2) and that now works. AttributeError: attribute 'post_handshake_auth' of '_ssl._SSLContext' objects is not writable”

[kalico1]

Thanks for all your help!

I had already tried #37 (it was linked in my post above but not with the #, I'll keep that in mind in the future, thanks)

I tried the goauth install and that helped a little? It got rid of the handshake/not writable error, but now I've got this:

kalico@MacBook-Air keep-it-markdown-0.4.3 % python3 keep-test.py

Enter your Google account username: **********@gmail.com
Enter your Google Password: 
Login attempt failed
 Technical Error Message: LoginException('BadAuthentication', None)
Please start your browswer and copy-paste this URL in the address bar: https://accounts.google.com/DisplayUnlockCaptcha - then, try logging in again.

I've tried using my actual google password, doing the extra authentication/captcha step, and finally created an app password again. Tested after each step and got the above error each time.

Then I tried turning off 2FA in my google account. Went through the process again, and now I'm getting this error:

kalico@MacBook-Air keep-it-markdown-0.4.3 % python3 keep-test.py

Enter your Google account username: **********@gmail.com
Enter your Google Password: 
Login attempt failed
 Technical Error Message: LoginException('UNKNOWN_ERR', None)
Please start your browswer and copy-paste this URL in the address bar: https://accounts.google.com/DisplayUnlockCaptcha - then, try logging in again.

I've been googling around and there is a lot of discussion about this "Technical Error Message: LoginException('BadAuthentication', None)" but nothing that looks like a solid solution. Got some clues that it could be related to being on an M1 but I'm not fully understanding it all.

My searches all lead back to this article https://github.com/kiwiz/gkeepapi/issues/81 but it started in 2020 and the latest items in 2021 are not really clear on a solution either, unless I'm missing/misunderstanding it.

[kalico1]

I also tried allowing less secure apps in google acct, but I'm back to the "bad authentication" error. I have double checked my password, by logging in with it and pasting it to a text file to be sure that what I'm pasting is accurate. I have no further option to set up an app password because I turned off 2FA.

[kalico1]

I'm going to try this (maybe tomorrow, it's late)

https://github.com/kiwiz/gkeepapi/issues/81#issuecomment-770389348

TL;DR: "So I think finally important was pinning the requests library and updating to the latest python3 version."

[kalico1]

Ok, before doing that, I thought...maybe I neglected to restart the terminal at some point when I should have. So I closed and reopened.

Then this happened:

kalico@macbook-air keep-it-markdown-0.4.3 % python3 keep-test.py                                   
Traceback (most recent call last):
  File "/Users/kalico/Downloads/keep-it-markdown-0.4.3/keep-test.py", line 2, in <module>
    import gkeepapi
ModuleNotFoundError: No module named 'gkeepapi'

So I decided to try the install again

kalico@macbook-air keep-it-markdown-0.4.3 % pip3 install -r requirements.txt
Collecting gkeepapi
  Using cached gkeepapi-0.13.7-py2.py3-none-any.whl (35 kB)
Collecting keyring
  Using cached keyring-23.5.0-py3-none-any.whl (33 kB)
Collecting click
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting gpsoauth>=0.4.1
  Using cached gpsoauth-1.0.0-py3-none-any.whl (6.4 kB)
Collecting six>=1.11.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting future>=0.16.0
  Using cached future-0.18.2.tar.gz (829 kB)
  Preparing metadata (setup.py) ... done
Collecting importlib-metadata>=3.6
  Using cached importlib_metadata-4.11.3-py3-none-any.whl (18 kB)
Collecting requests>=2.0.0
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting pycryptodomex>=3.0
  Using cached pycryptodomex-3.14.1.tar.gz (3.4 MB)
  Preparing metadata (setup.py) ... done
Collecting zipp>=0.5
  Using cached zipp-3.8.0-py3-none-any.whl (5.4 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Using legacy 'setup.py install' for future, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pycryptodomex, since package 'wheel' is not installed.
Installing collected packages: certifi, zipp, urllib3, six, pycryptodomex, idna, future, click, charset-normalizer, requests, importlib-metadata, keyring, gpsoauth, gkeepapi
  Running setup.py install for pycryptodomex ... done
  Running setup.py install for future ... done
Successfully installed certifi-2021.10.8 charset-normalizer-2.0.12 click-8.1.3 future-0.18.2 gkeepapi-0.13.7 gpsoauth-1.0.0 idna-3.3 importlib-metadata-4.11.3 keyring-23.5.0 pycryptodomex-3.14.1 requests-2.27.1 six-1.16.0 urllib3-1.26.9 zipp-3.8.0

So that was a successful install...apparently. (But I have since upgraded some of those, including gkeepapi and gpsoauth, so I'm not sure what's going to happen next.)

Ok, moving on, I tried the login again and got a new error.

Enter your Google account username: **********@gmail.com
Enter your Google Password: 
Login attempt failed
 Technical Error Message: LoginException('NeedsBrowser', 'To access your account, you must sign in on the web. Touch Next to start browser sign-in.')
Please start your browswer and copy-paste this URL in the address bar: https://accounts.google.com/DisplayUnlockCaptcha - then, try logging in again.

Well I'm already signed in to my browser. So for some reason the connection isn't happening. I checked my Chrome and I see that there is a Chrome update pending now. I'll install that and see if there's any progress. But I wanted to post this first so I don't lose it when I reopen Chrome.

[kalico1]

Same "needs browser" error. I'm going to bed!

[djsudduth]

Take a look at https://github.com/kiwiz/gkeepapi/issues/20

Someone noted re-enabling app pw:

I experienced this issue myself when i was trying to test out gkeepapi. Funny thing is, I had disabled two-step authentication and the issue still occurred. So I thought what would happen if I enable two step authentication for my google account to be able to generate app password for my script. It turns out that it worked that way :) Hope this helps someone.

[djsudduth]

PS - can you give me your full system environment specs? PC model, OS version, Python installation used, default browser (seems you have a Mac - but is Safari or Chrome your default)?

[djsudduth]

@kalico1 I tried it on my Macbook Pro running Sierra 12.3.1 and couldn't get kim to work. But, when I ran as admin using -> sudo python kim-test.py that worked! Maybe give that a try!

[kalico1]

Hey thanks for the thoughts. Really appreciate it. I want my Keep notes in Obsidian something fierce! lol

1) I've already tried using an app pw, and that hasn't worked yet. 2) Specs got lost in an earlier post, here they are again along with something I forgot - the OS version! I've got my Google account open in Chrome not logged in as any other user MacBook Air (M1, 2020) Big Sur 11.6.1 Python 3.8 3) I didn't think I could run KIM until I got logged into my google account via keep-test.py? I think I tried sudo on keep-test.py, but I'll try it again (with and without an app pw) and get back to you.

[kalico1]

Yahoo, I got it working!!!

I kept on getting the "Need Browser" error, and I finally connected it with something you asked: what is my default browser?

I use Chrome all the time, and I had Chrome open, and I was doing everything in Chrome as the instructions stated. But what if my MBA (terminal, in particular) thinks Safari is the default? I had to jump through all the hoops of logging into Google on Safari, try the DisplayUnlockCaptcha (no dice), reenabled 2FA, and finally, created an app pw (again). And at long last, after getting an app pw, IT FINALLY WORKED!

All that was with keep-test.py. Once I got a successful login, I ran kim.py and then I had to start that whole login process all over again, so I ended up making another app pw. It worked again! (and now I understand why you mentioned kim.py above)

I am currently in the process of finally getting --all of my Keep notes in markdown. Woot woot!

Thanks so much for all of your help and guidance. I hope this all helps someone else at some point!

P.S. Total converted notes: 1335 😄

[djsudduth]

Wow! So sorry for all the hoops to jump through to get there. But, congrats!! Great job sleuthing it out!!!

The official Google Keep API is now available. Once kim moves to it, all these problems should go away.

[kalico1]

Thanks for all your help!

[jamesmortensen]

Wow! So sorry for all the hoops to jump through to get there. But, congrats!! Great job sleuthing it out!!!

The official Google Keep API is now available. Once kim moves to it, all these problems should go away.

@djsudduth Quick question, when you move to the official Google Keep API, does this mean this tool won't be available for regular gmail accounts? I was under the impression that the official Google Keep API would only work with Google Workspace accounts. Thank you.

BTW, I just discovered this tool, and it looks awesome. I have wished Google Keep used markdown for years, and I've made a habit of using markdown syntax in my notes in hopes that one day I might export everything to something that can parse markdown.

[djsudduth]

@jamesmortensen - right now that seems to be the case (tho I haven't dug into it much yet). My assumption is that the API will be general use at some point since so many users don't have Workspace accounts. Rest assured, this utility will stay on the unofficial API until Google provides the general one.

[jamesmortensen]

@kalico1 I am running into the same problem but an trying to login from a GitHub Actions runner so that I can automate the process of backing up my Keep notes. I have 2FA and an app password set up and I get the same NeedBrowser error.

The Safari vs Chrome issue you overcame, do you know how I would deal with that from a GitHub Action runner? These are essentially fresh VMs where we have never logged into Google accounts before.

The resolution to your issue makes me think it has something to do with the device you're logged in from, but I have no idea how I would handle that from an action runner.

[djsudduth]

@jamesmortensen - does this help?

https://github.com/browser-actions/setup-chrome

[jamesmortensen]

Thanks for sending @djsudduth Are you saying I need to somehow log into my Google account in GitHub Actions via Chrome before I can go through kim (or the python gkeepapi) with an app password?

[djsudduth]

Not sure @jamesmortensen - just thought it might aim you towards a VM setup and login.

[djsudduth]

@kalico1 - can you outline the steps you took to setup the app password for google's 2FA? I have another system encountering your issue and the app password isn't working. We think it might be related to approval notification - but maybe the app pw is setup incorrectly.

[djsudduth]

Note for everyone: I was able to get KIM to work with 2FA on. Key is to make sure you have the latest version of gkeepapi and use Python 3.8+ now. You must use an app password for the first time logging in. If you use an app password for keep-test.py, then you must create a 2nd app password for first time in kim.py.

[ije896]

Having the same issue with keep-test.py. I've gone through the process of triple-checking:

• my default browser setting is correct
• I'm only signed into my default google account
• clicking the security captcha link in the default browser
• generating an app password each time I try to login with 2FA on
• disabling 2FA and using my regular password
• restarting computer and terminal
• I'm using the correct login information (I'm able to login on other apps with a generated app-password AND my regular password)
• I'm on the latest version of gkeepapi (and all dependencies)
• tried running with multiple google accounts, same issue across the board
• tried using older versions of the requests lib

Running on macOS 12.6.1 and using python 3.9.6, so I really don't get what gives here. Any help is greatly appreciated.

[djsudduth]

Hmmmm - I keep wondering if there is something odd about certain app password situations. I have a Windows 10 laptop with the same issue - I can't get it to recognize the app pw. However, on another Windows 10 laptop with the same network it works. Python environments, perhaps??? Not sure... I also installed Virtualbox with a Linux OS on the first Windows laptop and that worked fine with the app pw. Perplexing....

You could try a virtual os like Virtualbox and see if that works on your Mac @ije896

[djsudduth]

also, @ije896 - try running kim with sudo -> sudo python kim.py

[ije896]

tried it with sudo - no luck. virtualbox doesn't have a stable version for m1 macs :/

ran keep-test on repl, and that worked! but of course kim didn't work because of keyring requirements

I already have my Google Keep data downloaded through Takeout, but it looks like your script manipulates the data from the site without downloading, right?

I tried using pandoc on the local files, but it didn't convert well and everything looks horrible in obsidian, argggg

[djsudduth]

@ije896 yes, KIM calls an API vs data from Takeout. I'm almost done with version 0.5.0 of KIM. It has been refactored so that the next logical step will be much easier to add code to pull from the Takeout files as well (v 0.6.0).

How about trying a container on the Mac? Docker or Colima?

[djsudduth]

@ije896 for keyring on repl - does this help at all? Issue - https://github.com/djsudduth/keep-it-markdown/issues/37

[djsudduth]

Just tried the app pw on an M1 Mac - it worked with sudo. Be sure to do security captcha in Safari !! @ije896

[ije896]

Awesome, being able to run it on local Takeout files would be helpful in this case. I don't have any containers set up on this machine, but that keyring step worked for repl! Does that -r command work with keyring.alt?

Started fresh today on my machine, renewed app-password, clicked the captcha prompt, tried with sudo, and still no luck, that pesky bad auth error. But I'll just work with repl, can't for the life of me understand what's off - even tried from different network connections and IP's.

I was running into a parse issue with a note from archive, but then I deleted the empty notes from archive and it worked - maybe worth adding functionality/a check for that to make this even more robust? Thank you for your help with this, super useful once I got it going

[djsudduth]

@ije896 glad you worked it out! On your M1 mac did you do the captcha on Safari? There was an issue that if you did it on Chrome or Firefox the auth kept failing - but somehow doing it on Safari fixed the issue.

Anyway, can you describe the bad notes in your archive? Were they completely empty?

[ije896]

Yes, I changed my default browser to Safari did the captcha on there - still no luck. This is a pretty fresh and unmodified macOS install as well

Yeah, the notes were completely empty, they just showed 'Empty Note' on Keep, so an empty body and title. Not 100% sure they were the culprit, but it would hit a snag yesterday when pulling from Archive and then it worked today once I removed those notes.

[djsudduth]

@ije896 -ok, I'll try that test out - thanks.

PS, I could use a tester for the next version 0.5.0. It is in the 'develop' branch and you would just have to copy the 0.4.3 files to another directory and then just download the kim.py from develop. It has a few fixes in it and the Logseq flag (-l) to add bullets to note paragraphs. Still beta at the moment but almost done. The biggest change is media - I think there is an error in 0.4.3 downloading all images and audios correctly (especially if you use the --all switch). Most of the changes tho are just refactoring code.

[djsudduth]

You can also ask for some simple features soon if you want - I'm close to releasing it

[djsudduth]

Note - just fixed a bug in develop a few minutes ago

[RCheesley]

I'm also getting the Bad Authentication error on a new M2 macbook. Have 2FA enabled and have created the app password.

[djsudduth]

@RCheesley - please take a look at this post from the Google Keep API project - seems similar to your issue? It may be worthwhile using pyenv to try different versions of python without pain of installing.

https://github.com/kiwiz/gkeepapi/issues/81#issuecomment-1528720780

Also, from the notes above did you try using sudo??

[tinarowe]

Hello

I don't understand much of any of this, though I get the idea and it's a great idea. so I've been trying to follow. I've got python installed and even just upgraded, tiny yay. I get to the point of logging in and it just throws me out. I've saved and removed and verified the password I created for the app a few times. I'm getting this error message every time.

Username or password is incorrect (AttributeError("module 'urllib3.util.ssl_' has no attribute 'DEFAULT_CIPHERS'"))
Could not excute KIM

My password and username are not incorrect.

I don't know what a default cipher is but I tried googling for an answer and was confronted with a load of other issues related to other processes and apps. I can't sift the relevant from the irrelevant, so please could someone help me.

Thanks

[djsudduth]

@tinarowe hi!! A few people have been having this issue. (See this issue #62)

But, try this:

pip install urllib3==1.26.1

And then create a new app pw in Google and try it again. Hope you solve it!!

[tinarowe]

Hello

thank you for your prompt reply.

I've got rid of half of the errror! but I'm still getting this:

Enter your Google account username: madeup@gmail.com
Enter your Google Password:

One moment...

Username or password is incorrect (LoginException('BadAuthentication'))

I've put my userid and password in really carefully, I've tried pasting it but it's difficult to tell if that works because the cursor doesn't move when I type a pw.

[djsudduth]

@tinarowe ok, you're close now. Follow the setup carefully in the README on creating a one-time google app password and use that password the first time you try to run kim.py

Once that works you can use your normal google password every time after that.

I would suggest rebooting and being out of any apps to try this. Also, you might have to try creating a one time pw again.

[tinarowe]

Hello

I rebooted, made a new pw and it worked. I could not possibly be happier. Thank you!!

Tina

[djsudduth]

@tinarowe fantastic!!! Thank you for persevering - your comments here will help others!

Hope you find it useful!