search

gauteh / lieer

Fast email-fetching and sending and two-way tag synchronization between notmuch and GMail
http://lieer.gaute.vetsj.com
Other
494 stars 60 forks source link

gmi push takes hours every time #240

Closed kode54 closed 11 months ago

kode54 commented 11 months ago

I have 425k messages on my remote side, and it apparently has to fetch the metadata for all of them, every single time. Is there some way this can be sped up?

gauteh commented 11 months ago

You can try the resume option on the initial sync. Gmi needs a full initial sync.

søn. 28. mai 2023, 13:51 skrev Christopher Snowhill < @.***>:

I have 425k messages on my remote side, and it apparently has to fetch the metadata for all of them, every single time. Is there some way this can be sped up?

— Reply to this email directly, view it on GitHub https://github.com/gauteh/lieer/issues/240, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAN364HKSKEUJOLABXXZI3XIM33ZANCNFSM6AAAAAAYRZL3BI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

kode54 commented 11 months ago

I already have a full initial sync. It does a full metadata fetch every time I run sync after that.

Is nothing else supposed to touch my gmail account between syncs?

gauteh commented 11 months ago

No, that should be fine. But if the historyId is out-of-bounds gmi needs to do a full sync as well. This usually happens if you haven't run gmi for a long while, but can also happen if you make a lot of changes outside gmi.

The usual reason for this problem though is that the full sync somehow is not fully completed. Then the experience can be a bit better with the resume option (in case the full sync is interrupted).

You can also consider making your own API key, this should make things a bit faster. But I haven't tried that in a long time so I don't know exactly how that works nowadays.

kode54 commented 11 months ago

Yeah, I'm using my own API key, or at least my own OAuth2 secrets. Seems there's only the usual GMail throttling of 250 event points per second, sliding average. Here, it usually manages about 35 seconds of uninterrupted metadata downloading at about 160it/s, then it throttles. This repeats in cycles of approximately two minutes.

kode54 commented 11 months ago

Sorry about this. It turns out there was a huge mess of data to finally sync properly, because of the accident of having new.tags unset in my config, which resulted in all my new mail getting +inbox and +unread, which I needed to reconcile. After all the removal of the local tags, and the syncing, and some deletion of old mail (after archiving it to another notmuch config on spinning rust, specifically indexed for archival purposes), the sync process completes almost immediately.

gauteh commented 11 months ago

Good to hear!

søn. 4. jun. 2023, 10:48 skrev Christopher Snowhill < @.***>:

Sorry about this. It turns out there was a huge mess of data to finally sync properly, because of the accident of having new.tags unset in my config, which resulted in all my new mail getting +inbox and +unread, which I needed to reconcile. After all the removal of the local tags, and the syncing, and some deletion of old mail (after archiving it to another notmuch config on spinning rust, specifically indexed for archival purposes), the sync process completes almost immediately.

— Reply to this email directly, view it on GitHub https://github.com/gauteh/lieer/issues/240#issuecomment-1575473273, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAN36ZKOGGF6KFCFC6QYTDXJRDUTANCNFSM6AAAAAAYRZL3BI . You are receiving this because you commented.Message ID: @.***>