search

gmailgem / gmail

A Rubyesque interface to Gmail, with all the tools you'll need.
Other
397 stars 119 forks source link

Large memory usage when downloading lots of emails #219

Closed lg closed 4 years ago

lg commented 8 years ago

in the Mailbox.emails and Mailbox.emails_in_batches methods you retrieve each email, do the yield call and then .collect on it and keep it in memory to return (again) after the call finishes. when looping over thousands of emails, this will keep them all in memory and slow thing down to a crawl.

i would suggest either requiring the block (so the yield passes the relevant info back) and not returning anything (i.e. no more .collect necessary), OR making it more clear that all the Messages will be returned on a .emails call and to use this for small queries only. seeing as you have the yield already in there, personally i'd say take the first option since the user already has access to the relevant info.

the workaround i'm using now to get around this:

mailbox = @gmail.mailbox('[Gmail]/All Mail')
uids = mailbox.fetch_uids(gm: "#{@search_terms} newer:#{stopped_at}")
uids.each do |uid|
  gmail_email = Gmail::Message.new(mailbox, uid)
  # ...
end
johnnyshields commented 4 years ago

As of version 0.7.0 (Aug 19, 2018) this gem is officially deprecated and will no longer be maintained. Please instead use Google's official Gmail API Ruby Client, which uses the Gmail API rather than IMAP and has significantly better performance and reliability.