OAuth authentication fails with invalid credentials

[phrxmd]

This may be a beginner question, but I can't seem to get OAuth authentication against my main Google calendar to work. I used to use org-gcal from here which worked; but trying to use the same client ID and client secret with org-caldav I face the following issues:

1. After pasting the OAuth response string from the browser, org-caldav asks me for a username and password for https://accounts.google.com/o/oauth2/token and I don't know what to enter here; my Google password seems not to work (I have 2-factor authentication enabled) and a specially created app password doesn't work either. Isn't the whole point of OAuth to stop the exchange of usernames and passwords over HTTP?
2. I get lots of popup windows asking me for credentials for the encrypted plstore, even though plstore-cache-passphrase-for-symmetric-encryption is set to t. GPG_AGENT_INFO is unset, both in .emacs and using unset GPG_AGENT_INFO in bash. Nevertheless, Emacs uses the external GPG agent.
3. After entering the passphraze dozens of times, nevertheless in the end I get a 401 error ("Unauthorized") and a message "Invalid token".

I'm using unmodified org-caldavfrom here, oauth2 from MELPA and emacs 25.2.1 on Linux Mint 19. Here's my org-caldav configuration:

(use-package org-caldav
  :init
  ;; Set environment variable in order to avoid being asked for the passphrase lots of times
  (setq plstore-cache-passphrase-for-symmetric-encryption t)
  (setenv "GPG_AGENT_INFO")

  ;; Actual calendar configuration edit this to meet your specific needs
  (setq org-caldav-url 'google)
  (setq org-caldav-oauth2-client-id "client id")
  (setq org-caldav-oauth2-client-secret "client secret")
  (setq org-caldav-calendars
        '((:calendar-id "my-google-username@googlemail.com" ; Taken from calendar settings
           :files ("~/Dropbox/Org/inbox.org" "~/Dropbox/Org/personal.org")
           :inbox "~/Dropbox/Org/gcaldav.org")
          ))
  (setq org-caldav-backup-file "~/Dropbox/Org/gcaldav-backup.org")
  (setq org-caldav-save-directory "~/Dropbox/Org/")

  :config
  (setq org-icalendar-alarm-time 1)
  ;; This makes sure to-do items as a category can show up on the calendar
  (setq org-icalendar-include-todo t)
  ;; This ensures all org "deadlines" show up, and show up as due dates
  (setq org-icalendar-use-deadline '(event-if-todo event-if-not-todo todo-due))
  ;; This ensures "scheduled" org items show up, and show up as start times
  (setq org-icalendar-use-scheduled '(todo-start event-if-todo event-if-not-todo))
  )

The configuration is based on this post, the client ID and client secret are working with org-gcal and should be fine.

The compile log has the following warnings:

.cache/emacs/elpa/oauth2-0.11/oauth2.elc:Warning: reference to free variable
    ‘url-http-extra-headers’
.cache/emacs/elpa/oauth2-0.11/oauth2.elc:Warning: reference to free variable
    ‘oauth--token-data’
.cache/emacs/elpa/oauth2-0.11/oauth2.elc:Warning: reference to free variable
    ‘url-callback-function’
.cache/emacs/elpa/oauth2-0.11/oauth2.elc:Warning: reference to free variable
    ‘url-callback-arguments’

Here is the debug log:

========== Started sync.
Syncing first calendar entry:
(:calendar-id "my-google-username@googlemail.com" :files ("~/Dropbox/Org/inbox.org" "~/Dropbox/Org/personal.org") :inbox "~/Dropbox/Org/gcaldav.org")
Check connection for https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/.
Got error while checking connection (will try again):
(error "Error while checking for OPTIONS at URL https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/: 401 Unauthorized")
Check connection for https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/.
Got error while checking connection (will try again):
(error "Error while checking for OPTIONS at URL https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/: 401 Unauthorized")
Check connection for https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/.
Got error while checking connection (will try again):
(error "Error while checking for OPTIONS at URL https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/: 401 Unauthorized")
Check connection for https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/.
Got error while checking connection (will try again):
(error "Error while checking for OPTIONS at URL https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/: 401 Unauthorized")
Check connection for https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/.
Check connection for https://apidata.googleusercontent.com/caldav/v2/my-google-username@googlemail.com/events/.

[andyhunne]

Do you by chance have 2 factor authentication enabled on your Google account? If so, you might need to generate a "app" password, as detailed here.

https://support.google.com/mail/answer/185833?hl=en

I was having the problem you described, then recalled a similar issue authorising another app to use Google's SMTP server. I tried creating an app password and it has now worked. I did need to blow away my emacs directory and reinstall it to get it to finally work. I'm sure that's a pretty extreme solution and isn't necessary for people with more knowledge of emacs/spacemacs. I couldn't figure out how to reauth the app via oauth2 to retry with an app password.

[thishappymadness]

Hi Philippe Did you by any chance manage to fix your issue? I am facing what seems to be a similar issue but have not yet found a solution to the problem faced.

Thank you in advance for any help.

[GregorySchwartz]

I have this same issue. It works, but it constantly asks for my credentials (even after checking remember this information) and I need to delete the oauth2.plstore every now and then.

[GregorySchwartz]

It has now completely stopped working and I just cannot sync with google at all due to "Invalid Credentials".

[phrxmd]

Same thing for me. I never got it working properly and now it's not working at all. Safe to say that this project is dead.

[BenediktBroich]

Same issue here.

[BenediktBroich]

Found the solution. You have to enable CalDAV API for your application in the Google Developer Conole

[GregorySchwartz]

Unfortunately I already have that enabled and it did not work.

[GregorySchwartz]

Unless the credential must be updated through the Developer Console...I should try that.

[elken]

Found the solution. You have to enable CalDAV API for your application in the Google Developer Conole

I've just hit this myself, this does indeed resolve it and should be added to the README :)

I know that google seems to be frowned upon but many of us require it for work and having good instructions for setting it up would be great! If it would be accepted, I'm happy to attempt a minor rewrite of the README

[jackkamm]

If it would be accepted, I'm happy to attempt a minor rewrite

Yes, update to the google documentation would be very much appreciated.

If you also want to do more extensive rewrite of the docs, I suggest checking in this issue to avoid duplicating effort: https://github.com/dengste/org-caldav/issues/265

By the way, is oauth2 really necessary for google, or is App Password sufficient? I don't use org-caldav with google, but I do use mbsync/msmtp with gmail, and I just use an App Password for that, I don't think I use oauth2 (but maybe I misremember).

[elken]

By the way, is oauth2 really necessary for google, or is App Password sufficient?

I think it depends if you want bi-directional sync or not. In my case I was also missing a bunch of meetings, ones that weren't company-wide I think

[ononotofu]

Same issue, this suddenly started failed this week with no changes to the configuration or the account. Have tried deleting oauth2.plstore.

HTTP/1.1 401 Unauthorized WWW-Authenticate: Bearer realm="https://accounts.google.com/", error="invalid_token" Vary: Origin Vary: X-Origin Vary: Referer Content-Type: application/json; charset=UTF-8 Date: Sun, 01 Oct 2023 22:01:02 GMT Server: ESF Content-Length: 0 X-XSS-Protection: 0 X-Frame-Options: SAMEORIGIN X-Content-Type-Options: nosniff Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

and

org-caldav-url-dav-get-properties: Error while doing PROPFIND for ’getetag’ at URL https://apidata.googleusercontent.com/caldav/v2/xredactedx@group.calendar.google.com/events/: 401 Unauthorized

CalDAV API has been enabled from the start. I've had this happen a few times and it can only be fixed by using a new calendar, making me believe that somehow org-caldav is pushing a malformed event which then causes that calendar to permanently fail, no matter the credentials used to fetch.

[jackkamm]

So, I think this thread contains a couple unrelated issues regarding google calendar setup. But regarding the 401 invalid token error specifically, I think it might have to do with problems refreshing an expired oauth2 token. From https://developers.google.com/calendar/api/guides/errors :

401: Invalid Credentials Invalid authorization header. The access token you're using is either expired or invalid. Suggested actions: Get a new access token using the long-lived refresh token. If this fails, direct the user through the OAuth flow, as described in Authorizing requests with OAuth 2.0

I think I ran into this problem when trying to re-enable my old google calendar config for testing. I got the 401 error, but worked around it by creating a new app/project/client ID in the Google console which then worked.