search

OfflineIMAP / offlineimap

Read/sync your IMAP mailboxes (python2) [LEGACY: move to offlineimap3]
http://www.offlineimap.org
Other
1.78k stars 360 forks source link

access_token error despite set refresh_token #374

Closed ewilliam closed 8 years ago

ewilliam commented 8 years ago 
OfflineIMAP 7.0.6
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Account sync Ewilliam:
 *** Processing account Ewilliam
 Establishing connection to imap.gmail.com:993 (Ewilliam-Remote)
 ERROR: While attempting to sync account 'Ewilliam'
  'access_token'
 *** Finished account 'Ewilliam' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'Ewilliam'
  'access_token'

Traceback:
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/accounts.py", line 271, in syncrunner
    self.__sync()
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/accounts.py", line 334, in __sync
    remoterepos.getfolders()
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/repository/IMAP.py", line 448, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 525, in acquireconnection
    self.__authn_helper(imapobj)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 428, in __authn_helper
    if func(imapobj):
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 361, in __authn_xoauth2
    imapobj.authenticate('XOAUTH2', self.__xoauth2handler)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 705, in authenticate
    typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 1692, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 1418, in _command
    literal = literator(data, rqb)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 2283, in process
    ret = self.mech(self.decode(data))
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 267, in __xoauth2handler
    self.oauth2_access_token = resp['access_token']

.offlineimaprc

[general]
accounts = Ewilliam
pythonfile=~/.mutt/offlineimap.py
fsync = False

[Account Ewilliam]
localrepository = Ewilliam-Local
remoterepository = Ewilliam-Remote
status_backend = sqlite
postsynchook = notmuch new

[Repository Ewilliam-Local]
type = Maildir
localfolders = ~/.mail/xxx
nametrans = get_remote_name

[Repository Ewilliam-Remote]
maxconnections = 1
type = Gmail
remoteuser = xxx
remotepasseval = get_password("xxx")
realdelete = no
sslcacertfile = /usr/local/etc/openssl/cert.pem

# XOAuth2 authentication for Gmail
# https://github.com/OfflineIMAP/offlineimap/blob/master/offlineimap.conf#L764
# TODO: use oauth2_client_id_eval etc
oauth2_client_id = xxx
oauth2_client_secret = xxx
oauth2_refresh_token = xxx
oauth2_request_url = https://accounts.google.com/o/oauth2/token

nametrans = get_local_name
folderfilter = is_included
nicolas33 commented 8 years ago

Is this reproducible? Does it work with v7.0.5? offlineimap -V?

nicolas33 commented 8 years ago

Also, please try with -d imap.

ewilliam commented 8 years ago

i'm on 7.0.6. I was on 7.0.4 when i first encountered this error... been using gmail webgui :/

offlineimap -d imap

OfflineIMAP 7.0.6
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Debug mode: Forcing to singlethreaded.
Now debugging for imap: IMAP protocol debugging
Now debugging for : Other offlineimap related sync messages
Account sync Ewilliam:
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 *** Processing account Ewilliam
 Establishing connection to imap.gmail.com:993 (Ewilliam-Remote)
 [imap]: Attempting XOAUTH2 authentication
 [imap]: xoauth2handler: url "https://accounts.google.com/o/oauth2/token"
 [imap]: xoauth2handler: params "{'client_secret': 'xxx', 'grant_type': 'refresh_token', 'refresh_token': 'xxx', 'client_id': 'xxx'}"
 [imap]: xoauth2handler: response "{u'error': u'invalid_grant'}"
 ERROR: While attempting to sync account 'Ewilliam'
  'access_token'
 ['  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/accounts.py", line 271, in syncrunner\n    self.__sync()\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/accounts.py", line 334, in __sync\n    remoterepos.getfolders()\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/repository/IMAP.py", line 448, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 525, in acquireconnection\n    self.__authn_helper(imapobj)\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 428, in __authn_helper\n    if func(imapobj):\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 361, in __authn_xoauth2\n    imapobj.authenticate(\'XOAUTH2\', self.__xoauth2handler)\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 705, in authenticate\n    typ, dat = self._simple_command(\'AUTHENTICATE\', mechanism.upper())\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 1692, in _simple_command\n    return self._command_complete(self._command(name, *args), kw)\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 1418, in _command\n    literal = literator(data, rqb)\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 2283, in process\n    ret = self.mech(self.decode(data))\n', '  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 267, in __xoauth2handler\n    self.oauth2_access_token = resp[\'access_token\']\n']
 *** Finished account 'Ewilliam' in 0:01
 ERROR: Exceptions occurred during the run!
 ERROR: While attempting to sync account 'Ewilliam'
  'access_token'

Traceback:
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/accounts.py", line 271, in syncrunner
    self.__sync()
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/accounts.py", line 334, in __sync
    remoterepos.getfolders()
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/repository/IMAP.py", line 448, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 525, in acquireconnection
    self.__authn_helper(imapobj)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 428, in __authn_helper
    if func(imapobj):
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 361, in __authn_xoauth2
    imapobj.authenticate('XOAUTH2', self.__xoauth2handler)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 705, in authenticate
    typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 1692, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 1418, in _command
    literal = literator(data, rqb)
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/bundled_imaplib2.py", line 2283, in process
    ret = self.mech(self.decode(data))
  File "/usr/local/Cellar/offlineimap/HEAD-59d254c/libexec/offlineimap/imapserver.py", line 267, in __xoauth2handler
    self.oauth2_access_token = resp['access_token']
nicolas33 commented 8 years ago

Response from server is [imap]: xoauth2handler: response "{u'error': u'invalid_grant'}".

nicolas33 commented 8 years ago

Is your client id correct?

ewilliam commented 8 years ago

sorry, just came back from vacation. client_id and client_secret are correct, i just double checked

nicolas33 commented 8 years ago

I don't know what's the real root cause. Google only returns "invalid_grant".

Here's what we have from the official documentation:

When you try to use a refresh token, the following returns you an invalid_grant error:

* Your server's clock is not in sync with network time protocol - NTP.
* The refresh token limit has been exceeded.
ewilliam commented 8 years ago

interesting... i commented out oauth2_refresh_token and it "passes"

offlineimap

OfflineIMAP 7.0.6
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Account sync Ewilliam:
 *** Processing account Ewilliam
 Establishing connection to imap.gmail.com:993 (Ewilliam-Remote)
 XOAUTH2 authentication failed: AUTHENTICATE command error: BAD ['Client aborted AUTHENTICATE command. cn9mb200386477igc']. Data: KKIG2 AUTHENTICATE XOAUTH2

Folder archive [remote name [Gmail]/All Mail] [acc: Ewilliam]:
 Syncing [Gmail]/All Mail: Gmail -> Maildir
Folder drafts [remote name [Gmail]/Drafts] [acc: Ewilliam]:
 Syncing [Gmail]/Drafts: Gmail -> Maildir
Folder flagged [remote name [Gmail]/Starred] [acc: Ewilliam]:
 Syncing [Gmail]/Starred: Gmail -> Maildir
Folder INBOX [acc: Ewilliam]:
 Syncing INBOX: Gmail -> Maildir
Folder sent [remote name [Gmail]/Sent Mail] [acc: Ewilliam]:
 Syncing [Gmail]/Sent Mail: Gmail -> Maildir
Folder spam [remote name [Gmail]/Spam] [acc: Ewilliam]:
 Syncing [Gmail]/Spam: Gmail -> Maildir
Folder trash [remote name [Gmail]/Trash] [acc: Ewilliam]:
 Syncing [Gmail]/Trash: Gmail -> Maildir
Account sync Ewilliam:
 Calling hook: notmuch new
 Hook stdout: No new mail.

Hook stderr:

 Hook return code: 0
 *** Finished account 'Ewilliam' in 0:12

offlineimap -d imap

OfflineIMAP 7.0.6
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Debug mode: Forcing to singlethreaded.
Now debugging for imap: IMAP protocol debugging
Now debugging for : Other offlineimap related sync messages
Account sync Ewilliam:
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 *** Processing account Ewilliam
 Establishing connection to imap.gmail.com:993 (Ewilliam-Remote)
 [imap]: Attempting XOAUTH2 authentication
 [imap]:   12:18.66 Account sync Ewilliam last 20 log messages:
 XOAUTH2 authentication failed: AUTHENTICATE command error: BAD ['Client aborted AUTHENTICATE command. ku4mb156173885obc']. Data: KCFM2 AUTHENTICATE XOAUTH2

 [imap]: Attempting PLAIN authentication
 [imap]: __plainhandler: returning xxx(passwd hidden for log)
 []: Filtering out '[Gmail]/Important'[Ewilliam-Remote] due to folderfilter
 []: Copying folder structure from Ewilliam-Remote to Ewilliam-Local
 []: Not syncing filtered folder '[Gmail]/Important'[Ewilliam-Remote]
 Syncing [Gmail]/All Mail: Gmail -> Maildir
 Loading message list for Maildir[archive]
 Message list for Maildir[archive] loaded: 13920 messages
 Loading message list for Gmail[[Gmail]/All Mail]
 [imap]: calling imaplib2 fetch command: '1:*' (FLAGS UID INTERNALDATE)
 [imap]: __options2hash returning: {'INTERNALDATE': '"16-Sep-2011 02:54:35 +0000"', 'FLAGS': '(\\Seen)', 'UID': '1'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"16-Sep-2011 02:54:35 +0000"', 'FLAGS': '(\\Seen)', 'UID': '2'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"16-Sep-2011 02:54:35 +0000"', 'FLAGS': '(\\Seen)', 'UID': '3'}
nicolas33 commented 8 years ago

XOAUTH authentication fails and offlineimap fallbacks to PLAIN authentication method. I'd say we might want to always use the fallback system.

EFraim commented 8 years ago

refresh_token authentication fails for me with the same error ('invalid_grant') If I specify the access token retrieved from gmail-oauth2-tools the authentication succeeds, with the same client id and secret.

nicolas33 commented 8 years ago

Please try v7.0.8. Some errors are not silent anymore while downloading the token.

EFraim commented 8 years ago

Sorry, I should have mentioned I am using today's git HEAD.

nicolas33 commented 8 years ago

@EFraim Could you provide your own debug session log, please? (with -d imap)

EFraim commented 8 years ago

OfflineIMAP 7.0.8 Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception) Debug mode: Forcing to singlethreaded. Now debugging for imap: IMAP protocol debugging Now debugging for : Other offlineimap related sync messages Account sync personal: [imap]: Using authentication mechanisms ['XOAUTH2'] * Processing account personal Establishing connection to imap.gmail.com:993 (personal-remote) [imap]: Attempting XOAUTH2 authentication [imap]: xoauth2handler: url "https://accounts.google.com/o/oauth2/token" [imap]: xoauth2handler: params "{'client_secret': 'xxxxxxxxxxxxxxxxxxxxxxxx', 'grant_type': 'refresh_token', 'refresh_token': 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy', 'client_id': 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz.apps.googleusercontent.com'}" [imap]: xoauth2handler: response "{u'error': u'invalid_grant'}" XOAUTH2 authentication failed: xoauth2handler got: {u'error': u'invalid_grant'} ERROR: All authentication types failed: XOAUTH2: xoauth2handler got: {u'error': u'invalid_grant'} [' File "/home/evgeny/build/offlineimap/offlineimap/accounts.py", line 271, in syncrunner\n self.sync()\n', ' File "/home/evgeny/build/offlineimap/offlineimap/accounts.py", line 334, in sync\n remoterepos.getfolders()\n', ' File "/home/evgeny/build/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders\n imapobj = self.imapserver.acquireconnection()\n', ' File "/home/evgeny/build/offlineimap/offlineimap/imapserver.py", line 540, in acquireconnection\n self.__authn_helper(imapobj)\n', ' File "/home/evgeny/build/offlineimap/offlineimap/imapserver.py", line 415, in __authn_helper\n "failed:\n\t%s"% msg, OfflineImapError.ERROR.REPO)\n'] * Finished account 'personal' in 0:01 ERROR: Exceptions occurred during the run! ERROR: All authentication types failed: XOAUTH2: xoauth2handler got: {u'error': u'invalid_grant'}

Traceback: File "/home/evgeny/build/offlineimap/offlineimap/accounts.py", line 271, in syncrunner self.sync() File "/home/evgeny/build/offlineimap/offlineimap/accounts.py", line 334, in sync remoterepos.getfolders() File "/home/evgeny/build/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders imapobj = self.imapserver.acquireconnection() File "/home/evgeny/build/offlineimap/offlineimap/imapserver.py", line 540, in acquireconnection self.__authn_helper(imapobj) File "/home/evgeny/build/offlineimap/offlineimap/imapserver.py", line 415, in __authn_helper "failed:\n\t%s"% msg, OfflineImapError.ERROR.REPO)

nicolas33 commented 8 years ago

@EFraim Is the system time in sync with NTP?

EFraim commented 8 years ago

The time is within 2 minutes of the correct one.

On Thu, Oct 13, 2016 at 3:18 PM, Nicolas Sebrecht notifications@github.com wrote:

@EFraim https://github.com/EFraim Is the system time in sync with NTP?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OfflineIMAP/offlineimap/issues/374#issuecomment-253497034, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdXCTzyAnuUirUqfMYsbDli85rMRdl6ks5qziGwgaJpZM4JqWsB .

nicolas33 commented 8 years ago

@EFraim I'm not plain sure it's enough.

EFraim commented 8 years ago

I did an adjustment with ntpdate, it had no effect.

EFraim commented 8 years ago

Update: For some reason the problem totally disappeared when I generated a new refresh token from the same client ID and secret.

nicolas33 commented 8 years ago

@EFraim Thank you much for this feedback. As said above, "invalid_grant" is raised for two root causes.

When you try to use a refresh token, the following returns you an invalid_grant error:

* Your server's clock is not in sync with network time protocol - NTP.
* The refresh token limit has been exceeded.

However, I'm not sure what Google means by "token limit has been exceeded".

Anyway, I'll add your tip in the configuration file that recreating the refresh token might help.

nicolas33 commented 8 years ago

Done.