Emails sync failing - Google account connects but sync state is either stuck on "pending" or immediate "failed"

[r0bbie]

Bug Description

Initially tried on a self-hosted install (v0.11.2 Docker). My Google account successfully connects, and calendar sync seems to work flawlessly. Email sync however shows "pending" and never syncs:

To rule out a problem with my specific setup I then tried this on https://app.twenty.com and there it also fails. Again calendar connects and syncs successfully, but here email sync state immediately goes to "failed":

So slightly different behaviour on hosted vs self-hosted but failing on both.

Expected behavior

Expected emails to successfully sync.

Technical inputs

Tried on a self-hosted install, and a newly created workspace on https://app.twenty.com

The account is a Google Workspace account in case relevant. I've been trying to rule out issues with the account itself - my first thought was it could be API access permissions being blocked at the workspace level, but in Google Workspace admin "App access control" has Gmail set to unrestricted. And again calendar sync succeeds, so it's strange.

Please let me know if I can provide any further detail / logs etc!

[greptile-apps[bot]]

Disclaimer: This comment was automatically generated with AI. It can give some useful hints as to where to look for, but sometimes it can also give very inaccurate answers so takes this with a pinch of salt :).

To address the email sync issue, follow these steps:

1. Check Google API Service Configuration: Ensure that the Google API services are correctly configured in services/google_api_service.py.

2. Review Gmail Sync Logic: Inspect the Gmail sync logic in services/gmail_sync_service.py. Look for any discrepancies or errors in the sync process.

3. Fetch Message Content from Cache: Verify the logic for fetching message content from the cache in services/gmail_cache_service.py.

4. Error Handling and Logging: Add or review error handling and logging in the above files to capture any specific errors during the sync process.

These steps should help identify and resolve the issue with the Gmail sync functionality.

References

[bosiraphael]

Hey @r0bbie ! Thank you for documenting on this issue. I think that the self hosted problem comes from the fact that you might have not started the cron jobs which are needed to sync the emails. These are the commands:

• yarn command:prod cron:messaging:gmail-partial-sync
• yarn command:prod cron:messaging:gmail-fetch-messages-from-cache

Also, you need to make sure that your worker is running. For self-hosting, we recommend to have a dedicated container similar to twenty (server) but running the worker. To do that, just duplicate the twenty container in your docker-compose and change the command (entry point) from

• yarn start:prod to yarn worker:prod

Tell me if this solves the problem, or if it's something else :)

I will take a look at the problem on Twenty cloud.

[r0bbie]

Thanks @bosiraphael! (by the way let me know if I should have split this out as two separate issues, wasn't sure if they were the same cause or not!)

On self-hosted install (and stuck on "pending" issue) should say I'm running this self-hosted following Docker instructions at https://docs.twenty.com/start/self-hosting/docker-compose (with additional Google environment variables added though not currently documented there, i.e. AUTH_GOOGLE_CLIENT_ID AUTH_GOOGLE_CLIENT_SECRET etc etc)

Should the cron jobs not run already in the Docker container, or is that not setup by default at the moment? I tried running those commands in the container (i.e. npx nx run twenty-server:command cron:messaging:gmail-partial-sync) but that didn't seem to work - it tried to install a load of missing dependencies with yarn, then said it had to rebuild the app before the command would run, which then broke the live app. Recreating the Docker container got me up and running again easily enough.. but assuming this isn't the right approach anyway, since any changes to the container would just be lost each time it's rebuilt from the image anyway I think right? Maybe I'm being dumb / running them in the wrong place though, if so please let me know 😅

For the in-production issue, please let me know if I can provide any further info that might help :)

[charlesBochet]

We can discuss your self-hosting issues from #help on Discord, you'll get faster answers

Feel free to ping the team using the Support button in the left navigation bar on app.twenty.com

[deurk]

Facing the same issue, some steps are missing for self-hosting the docker versions apparently. I had to dig to find the AUTH_GOOGLE credentials as well, and the commands mentioned above end up with errors ERROR [Error] Nest could not find GmailPartialSyncCronJob element (this provider does not exist in the current context) for example.

Really looking forward to find out the missing pieces, as Twenty looks exciting to work with!

[bonnlei]

I have same issue by self-hosting. It shows always pending. I have seen that the email async added. How to check the sync cron running?