nylas / sync-engine

:incoming_envelope: IMAP/SMTP sync system with modern APIs
https://nylas.com/docs/platform
GNU Affero General Public License v3.0
3.5k stars 354 forks source link

arcihve/delete/read actions not syncing to gmail accounts #378

Closed audibleblink closed 7 years ago

audibleblink commented 8 years ago

I'm currently running the self-hosted version of of the sync-engine on linux from home. In order to access the sync-engine from outside the network on my laptop, I bind the server's 5555 port to my laptop's localhost with ssh -L 5555:localhost:5555 $nylas_server and the laptop client begins to receive emails like normal. When reading, archiving, and deleting mail, everything seems to be working fine locally, but none of those actions are actually reflected on gmail.com. I'm not seeing any errors in the activity window.

entropitor commented 8 years ago

I'm experiencing the same issue using a VPS.

The actions are also not syncing back to IMAP accounts.

trestletech commented 8 years ago

Same symptoms but I'm getting errors like:

[ERROR] {"greenlet_id": xxx, "account_id": 3, "error_tb": "Traceback (most recent call last):\n\t  File \"/home/jeff/workspace/sync-engine/inbox/transactions/actions.py\", line 382, in execute_with_lock\n    self.func(*func_args)\n\t  File \"/home/jeff/workspace/sync-engine/inbox/actions/base.py\", line 50, in mark_unread\n    set_remote_unread(cri
spin_client, account_id, message_id, unread)\n\t  File \"/home/jeff/workspace/sync-engine/inbox/actions/backends/generic.py\", line 81, in set_remote_unread\n    not unread)\n\t  File \"/home/jeff/workspace/sync-engine/inbox/actions/backends/generic.py\", line 69, in _set_flag\n    crispin_client.conn.add_flags(uids, [flag_name], silent=True)\n\tTypeError: add
_flags() got an unexpected keyword argument 'silent'\n", "timestamp": "2016-09-20T20:49:38.004957Z", "error_message": "add_flags() got an unexpected keyword argument 'silent'", "provider": "fastmail", "module": "inbox.transactions.actions:404", "extra_args": {"unread": false}, "action": "mark_unread", "action_log_id": 100, "error": "TypeError", "record_id": 36
504, "level": "error", "event": "Uncaught error"}
[INFO] {"account_id": 3, "timestamp": "2016-09-20T20:49:38.016648Z", "module": "inbox.transactions.actions:421", "record_id": 36504, "greenlet_id": xxx, "duration": 30, "event": "Syncback task retrying action after sleeping", "level": "info", "extra_args": {"unread": false}, "action_log_id": 100, "action": "mark_unread"}
[ERROR] {"greenlet_id": 139654360882864, "account_id": 1, "error_tb": "Traceback (most recent call last):\n\t  File \"/home/jeff/workspace/sync-engine/inbox/transactions/actions.py\", line 382, in execute_with_lock\n    self.func(*func_args)\n\t  File \"/home/jeff/workspace/sync-engine/inbox/actions/base.py\", line 50, in mark_unread\n    set_remote_unread(cri
spin_client, account_id, message_id, unread)\n\t  File \"/home/jeff/workspace/sync-engine/inbox/actions/backends/generic.py\", line 81, in set_remote_unread\n    not unread)\n\t  File \"/home/jeff/workspace/sync-engine/inbox/actions/backends/generic.py\", line 69, in _set_flag\n    crispin_client.conn.add_flags(uids, [flag_name], silent=True)\n\tTypeError: add
_flags() got an unexpected keyword argument 'silent'\n", "timestamp": "2016-09-20T20:49:38.180193Z", "error_message": "add_flags() got an unexpected keyword argument 'silent'", "provider": "gmail", "module": "inbox.transactions.actions:404", "extra_args": {"unread": false}, "action": "mark_unread", "action_log_id": 26, "error": "TypeError", "record_id": 36484,
 "level": "error", "event": "Uncaught error"}

And sporadically

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 534, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/jeff/workspace/sync-engine/inbox/transactions/actions.py", line 456, in _run
    gevent.with_timeout(task.timeout(self.task_timeout), task.execute)
  File "/usr/local/lib/python2.7/dist-packages/gevent/timeout.py", line 253, in with_timeout
    return function(*args, **kwds)
  File "/home/jeff/workspace/sync-engine/inbox/transactions/actions.py", line 312, in execute
    task.execute_with_lock()
  File "/home/jeff/workspace/sync-engine/inbox/transactions/actions.py", line 427, in execute_with_lock
    gevent.sleep(self.retry_interval)
  File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 194, in sleep
    hub.wait(loop.timer(seconds, ref=ref))
  File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 630, in wait
    result = waiter.get()
  File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 878, in get
    return self.hub.switch()
  File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 609, in switch
    return greenlet.switch(self)
Timeout: 60 seconds
<SyncbackWorker at 0x7f03d085d870> failed with Timeout

or

[ERROR] Exception during reset or similar
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 636, in _finalize_fairy
    fairy._reset(pool)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 776, in _reset
    pool._dialect.do_rollback(self)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/dialects/mysql/base.py", line 2526, in do_rollback
    dbapi_connection.rollback()
ProgrammingError: (2014, "Commands out of sync; you can't run this command now")
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 534, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/jeff/workspace/sync-engine/inbox/transactions/actions.py", line 456, in _run
    gevent.with_timeout(task.timeout(self.task_timeout), task.execute)
  File "/usr/local/lib/python2.7/dist-packages/gevent/timeout.py", line 253, in with_timeout
    return function(*args, **kwds)
  File "/home/jeff/workspace/sync-engine/inbox/transactions/actions.py", line 312, in execute
    task.execute_with_lock()
  File "/home/jeff/workspace/sync-engine/inbox/transactions/actions.py", line 382, in execute_with_lock
    self.func(*func_args)
  File "/home/jeff/workspace/sync-engine/inbox/actions/base.py", line 50, in mark_unread
    set_remote_unread(crispin_client, account_id, message_id, unread)
  File "/home/jeff/workspace/sync-engine/inbox/actions/backends/generic.py", line 81, in set_remote_unread
    not unread)
  File "/home/jeff/workspace/sync-engine/inbox/actions/backends/generic.py", line 61, in _set_flag
    uids_for_message = uids_by_folder(message_id, db_session)
  File "/home/jeff/workspace/sync-engine/inbox/actions/backends/generic.py", line 32, in uids_by_folder
    filter(ImapUid.message_id == message_id).all()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2588, in all
    return list(self)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2736, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2751, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 277, in query
    self.waiter(self.fileno())
  File "/home/jeff/workspace/sync-engine/inbox/ignition.py", line 30, in gevent_waiter
    hub.wait(hub.loop.io(fd, 1))
  File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 630, in wait
    result = waiter.get()
  File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 878, in get
    return self.hub.switch()
  File "/usr/local/lib/python2.7/dist-packages/gevent/hub.py", line 609, in switch
    return greenlet.switch(self)
Timeout: 60 seconds
<SyncbackWorker at 0x7f03d085d730> failed with Timeout
exadeci commented 7 years ago

I got a similar issue with Gmail, I delete a mail it disappears only to come back after the sync, I have to delete it on Gmail's web interface for it to work.

grinich commented 7 years ago

Hi @exadeci -- please open a new bug with more details. Going to close this one as stale / can't repro.