search

ProtonMail / proton-bridge

Proton Mail Bridge application
GNU General Public License v3.0
1.17k stars 157 forks source link

High CPU usage #130

Open angristan opened 4 years ago

angristan commented 4 years ago

Expected Behavior

Light CPU usage

Current Behavior

Extreme CPU usage for a few seconds whenever the bridge is in use by Mail.app: fetching mailboxes or emails.

screenshot_2020-11-14_19-38-13@2x

Is this known/expected? Can I provide more info?

angristan commented 4 years ago

Related:

cpnfeeny commented 4 years ago

2020-11-15_15-49

Arch on latest Bridge 1.5; I'm having the same problem, except I don't need to have a client checking emails to experience the problem.

telephon commented 4 years ago

yes, same here, where the reason is that PMB re-dowloads thousands of emails. A fix is presumably on the way, but of course it is tedious and a waste of energy.

andrzejsza commented 3 years ago

thanks all for reporting. yes, we are well aware of the problem and are working on the solution. will keep you posted here with any progress.

JulienPalard commented 3 years ago

Seen this today on Debian bullseye with protonmail-bridge 1.5.2:

 PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
7047 mdk       20   0 4795368   1.1g   1.0g S 513.3   7.3  85:25.73 protonmail-bridge

I happen a lot currently, I'm not monitoring it closely, but it happened at least 3 times during the last 5 minutes.

Feel also the process is a bit high in memory usage.

tusing commented 3 years ago

This is what the battery impact of ProtonMail looks like on my laptop for a couple minutes after using Mail.app. For reference, Safari doesn't go above 6.0 when using the actual ProtonMail website.

Screen Shot 2020-12-18 at 4 29 45 PM
JulienPalard commented 3 years ago

@angristan you can add "and Linux" at the end of the issue title ☹

ahtremblay commented 3 years ago

On MacOS, protonmail bridge consumes almost 30-40% of my energy usage over a 12h period. Does it mean by battery last almost half as long as it otherwise would? If true, this leaves quite the sour taste...

JulienPalard commented 3 years ago

On MacOS, protonmail bridge consumes almost 30-40% of my energy usage over a 12h period. Does it mean by battery last almost half as long as it otherwise would?

I understand it the same, yes :(

petrmanek commented 3 years ago

I can report equivalent behaviour on arch 5.10.6-arch1-1. Using OfflineIMAP for mail synchronization, the Bridge tends to spin up to ~500% (on my 8-thread CPU) every so often. Please let me know if I can provide any logs that would help you...

horta commented 3 years ago

Same here. It has been CPU intensive for more than a week. Apple Mail on Mac-mini. Do you need logs?

uncletrunks commented 3 years ago

Using like 600 megs of memory on my machine too. Pretty unfortunate, but I guess I'm back to using the webmail client for the foreseeable future.

telephon commented 3 years ago

From what I have heard in the replies to my bug reports, the issue is known and someone seems to be working on it. But given that this has been unchanged for almost a year now, I begin to fear that protonmail has decided that webmail is the most important way to work with it ... For all those of us who are not online all the time and who have to mangage several mail accounts, this is not an option.

andrzejsza commented 3 years ago

I'm sure you all appreciate that the issue of high CPU usage is not straightforward to fix. First of all, Bridge is expected to consume a lot of resources during the initial sync - we want to have it done quickly and allow the users to use their email client without much delay, hence this part should be treated as expected behaviour. The problems starts when Bridge is not doing it as a one-off but when this becomes frequent behaviour. This is obviously not desired and frustrating to all of us. We identified a range of scenarios in which this happening - some are panic, some are related to fetch errors or, quite often, to expunge during not-uid fetch. Most of those problems are not clearly visible to the user and often just work when Bridge retries. We initially struggled to replicate some of them, just like with related https://github.com/ProtonMail/proton-bridge/issues/29. We managed to narrow down the problems though and a lot of fixes will be coming through in upcoming releases - some are already included in the recent 1.5.6, but a wider range will be coming in 1.6.x. Once we fix what we know is wrong, we will definitely reach out to gather what we've missed - but we hope it won't be much. Thanks for your patience guys.

ahtremblay commented 3 years ago

Did you guys successfully fixed this -because on my system it appears so. Proton-bridge has been running for a few days and it reports no 12-hour power usage. Although if I keep an eye on it, then once in a while it peaks instantaneously to 140ish power usage to immediately drops to near zero. It used to do this also before, but -before- and cumulatively it would add up to 10-14 power usage over a 12h period which represented almost 30%-50% of my daily power usage. Now it shows "-" as the 12h power usage which I assume is equivalent to 0. I am not sure if my system is reporting it differently now, or if you have fixed it already? There has been an update or two since then.

andrzejsza commented 3 years ago

Great to hear, but we're not done yet - more improvements to come shortly in 1.6.x. we're also working on introducing some metrics for internal builds, so that we could actually measure the performance. will keep this issues open until all the known problems are resolved.

clwainwright commented 3 years ago

I'm having the same issue: macos 10.14.6, apple mail v12.4, bridge v1.6.3. About once an hour my mail client decides it needs to re-download 10k emails, which then understandably causes the bridge to chew up a huge number of CPU cycles. But this is only a problem with the protonmail bridge. None of my other email accounts ever seem to re-download old emails.

petrmanek commented 3 years ago

After a week or two of testing, I can confirm that this behavior is still present in v1.6.3. Still, from user perspective I can see some progress (either high CPU load incidents take less time, or are less frequent, not sure about which one is the reason). Please keep up the good work! :+1:

andrzejsza commented 3 years ago

thanks @petrmanek. what we're working on now is a new message builder and a connection manager - the former should ensure that the clients does't get confused by some of the messages that might not be rfc compliant; the latter will mitigate any potential connectivity problems (PM server > API > Bridge's IMAP > Client). no eta on be above just yet.

andrzejsza commented 3 years ago

didn't mean to close this ;-)

JulienPalard commented 3 years ago

Some news, according to top: 

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                
273007 mdk    20   0 5838944   1.2g   1.1g S 521.9   7.8   5:10.33 proton-bridge

and:

$ ps -eo pid,lstart,cputime,cmd | grep proton
 272994 Fri Apr 16 23:32:13 2021 00:00:00 protonmail-bridge
 273007 Fri Apr 16 23:32:13 2021 00:10:25 /home/mdk/.cache/protonmail/bridge/updates/1.6.9/proton-bridge --launcher /usr/lib/protonmail/bridge/proton-bridge-launcher

As it's 9:34 AM the process has been started exactly 10 hours ago, and used 10mn of CPU (looks OK), and is using a lot of RAM.

salik1992 commented 3 years ago

I have had the same problem. But since version 1.8.3 it seems to be fixed. I'm running it since the release without turning it off and I haven't noticed any high CPU usage and 0 energy impact according to Activity Monitor.

System details: macOS 11.4, Macbook Air M1

kudrykv commented 3 years ago

Still have this issue, 550% CPU with PMB 1.8.3

macOS 11.2.3 Big Sur, Intel chip

JulienPalard commented 3 years ago

Using 1.8.3: 

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                
624098 mdk       20   0 6202704   1.4g   1.2g S   0.0   9.0  24:56.93 proton-bridge  

     $ ps -eo pid,lstart,cputime,cmd | grep proton
 624087 Thu Jun 10 17:47:04 2021 00:00:00 protonmail-bridge
 624098 Thu Jun 10 17:47:05 2021 00:25:00 /home/mdk/.cache/protonmail/bridge/updates/1.8.3/proton-bridge --launcher /usr/lib/protonmail/bridge/proton-bridge-launcher

It's 06:01:12 PM so the process runs since ~15mn and ate 25mn of CPU, looks a bit high to me.

The RAM consumption looks a bit high to me too, does it stores all my emails in RAM?

stellarhopper commented 3 years ago

With 1.6.9 previously, it consistently spooled up to 2000% CPU as reported by 'top', every minute or so.

Updated to 1.8.9, and now it is consistently at ~100-120% all of the time, and still rises up to 2000% sometimes, but this is maybe slightly less frequent than before. (maybe every 2-3 minutes instead of every minute).

Hopefully this can be improved sooner rather than later, this is really making the bridge untenable, especially on laptops. I don't care about using the webmail interface, and need some way of accessing via a traditional IMAP client. If the bridge is going to be this power hungry, I will unfortunately have to look for alternatives.

Update: After reading some of the Reddit threads linked above, I dropped my poll frequency in my email client (Evolution) to 15 minutes. Previously it was 2 minutes. This seems to have considerably improved the situation, I haven't seen a 2000% spike since I didn this (watching for about 10 minutes now), and the baseline 100% idle seems to have gone away too.

I'll keep an eye on this over the next few days and report back.

petrmanek commented 3 years ago

Unfortunately I can still report over-the-top CPU usage (like what @stellarhopper described) with v1.8.9 on my Thinkpad T590. Thus far I am mitigating by increasing the polling period but I would like to find a better solution as soon as possible.

mapennell commented 2 years ago
Screen Shot 2022-02-13 at 8 26 24 PM

Please help address this ProtonMail Bridge team.

macOS 12.2.1 latest bridge version

petrmanek commented 2 years ago

Same here. Still keeps happening with v2.1.0 on the latest Archlinux kernel (5.16.8-arch1-1).

andrzejsza commented 2 years ago

@petrmanek @mapennell can you confirm that the initial sync finished, i.e. your mailbox is in full sync? also, could you confirm if no improvements with 2.1.1? if not, it would be great if you could contact support - we would like to see the logs.

petrmanek commented 2 years ago

@andrzejsza I can confirm that my entire mailbox appears to be in full sync. For due diligence, I am not sure how to be completely certain of this but the numbers of emails in individual folders match those visible online, and I have been using the bridge like this for several months now.

I would be happy to update you re: v2.1.1 once it becomes available on the AUR and possibly share logs with PM support.

jandrusk commented 2 years ago

I'm still having this issue running Protonmail-Bridge 2.1.1-1 on Debian 11 AMD64. Issue has been open for two years with no patch??? 100% CPU happens on every offlineimap sync. Might just go back to Gmail, since I don't have this issue with fetchmail and POP3.

Paulj585 commented 2 years ago

Hi, I have used PM for 2 years no problem on Win7 Thunderbird also Linux Mint then after receiving a daily notification from PM, which I had turned off BTW, I clicked on it and then my problems began.... My CPU usage went through the roof and slowly and inexorably my system memory and network BW was gobbled up until OS froze. Using my Linux mint system the same thing was happening image Both systems were sync'd and up to date. PM support have said they have "seen this problem but only rarely" It seems like both sysyems are suffering a DDOS attack from PM. Hundreds of Gbytes of data is being downloaded onto my machines windows and linux, of what I have no idea. Despite asking PM what is being sent down the line they have not answered in any detail. It seems to me this is a "drafts" problem. On restart the system seems quiescent just sitting there. Begin composing a mail and when it autosaves to drafts the problem begins. It tries to upload to PM server and then screws up. After going through all the usual and clearing and reinstalling on advice from PM desktop support the problem is still there. My Linux system was left on after setting up T'bird from scratch. The problem was still existing and my entire system memory swap file etc was swallowed up and it seems to have overwritten part of the OS (that's a guess) because the system became locked and would not boot up. In the end I had to reformat and reinstall. The problem still remains. This problem is further widespread than PM admits and looking at the forums has been around for at least 2 years. TBH it looks like that PM are not taking this problem seriously. I have not so far seen any sign of a solution on any of the forums other than stop using bridge. Not really a suitable situation.

kudrykv commented 2 years ago

I'm curious how it works for other people and how this issue isn't widespread.

I have been using both bridges, and both were having the same behavior (legacy and new).

At some point, I changed my Mac to the newer model, and this problem has been there with me from day one. I can't stress this more clearly: I've installed the bridge on the brand new computer, and the issue was there right away. ¯\_(ツ)_\/¯

Paulj585 commented 2 years ago

This problem I'm convinced is a Proton Mail side problem and like many a company pass the problem on and call it a user problem. Interestingly I can see my Linux system has, since I started it 2hr ago, has downloaded over 16Gb of data. To prevent it screwing up my system I have made my swap file 45GByte! It is swallowing it up. Protonmail need to do the decent thing and sort this out. They trade on their "swiss" reputation of openess and honesty but quite simply I am paying for something that simply does not work!

------- Original Message ------- On Friday, May 27th, 2022 at 10:56, Vitaliy Kudryk @.***> wrote:

I'm curious how it works for other people and how this issue isn't widespread.

I have been using both bridges, and both were having the same behavior (legacy and new).

At some point, I changed my Mac to the newer model, and this problem has been there with me from day one. I can't stress this more clearly: I've installed the bridge on the brand new computer, and the issue was there right away. ¯_(ツ)_/¯

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

jandrusk commented 2 years ago

Greetings,

I have been running into the same issue for a number of months as I've been running offlineimap with the proton-bridge app on Debian 11. I finally caved in an opened a support case around it. Sent all of the relevant details and logs and the response was that the engineers are aware of the issue and are working on a fix.

I just switched over to mbsync/isync from offlineimap and the same issue persists, but mbsync seems to update faster than offlineimap, so it's slighlty less annoying. Hopefully it's fixed in the next release ;)

Regards,

Justin Andrusk "Paulj585" @.***> writes:

Hi, I have used PM for 2 years no problem on Win7 Thunderbird also Linux Mint then after receiving a daily notification from PM, which I had turned off BTW, I clicked on it and then my problems began.... My CPU usage went through the roof and slowly and inexorably my system memory and network BW was gobbled up until OS froze. Using my Linux mint system the same thing was happening image Both systems were sync'd and up to date. PM support have said they have "seen this problem but only rarely" It seems like both sysyems are suffering a DDOS attack from PM. Hundreds of Gbytes of data is being downloaded onto my machines windows and linux, of what I have no idea. Despite asking PM what is being sent down the line they have not answered in any detail. It seems to me this is a "drafts" problem. On restart the system seems quiescent just sitting there. Begin composing a mail and when it autosaves to drafts the problem begins. It tries to upload to PM server and then screws up. After going through all the usual and clearing and reinstalling on advice from PM desktop support the problem is still there. My Linux system was left on after setting up T'bird from scratch. The problem was still existing and my entire system memory swap file etc was swallowed up and it seems to have overwritten part of the OS (that's a guess) because the system became locked and would not boot up. In the end I had to reformat and reinstall. The problem still remains. This problem is further widespread than PM admits and looking at the forums has been around for at least 2 years. TBH it looks like that PM are not taking this problem seriously. I have not so far seen any sign of a solution on any of the forums other than stop using bridge. Not really a suitable situation.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

-- Regards, Justin

Paulj585 commented 2 years ago

Hi, Well I admire your confidence on PM sorting this out. From the mail exchanges I've had I get the feeling they don't have a clue whats going on. They have said this problem occurs only rarely but from the message boards it seems like this has been occuring regularly over the last two years. Info has been in very short supply and using the browser for mail is pretty inconvenient. So not terribly impressed with PM at present. They won't even say what it is that is being downloaded onto our machines. I stopped mine the other day at over 20Gbytes had taken over my swap file. It seems to me that putting the onus on the customer to do all the work of rectification with solutions that clearly don't work don't help either. The annoying thing is that they don't even answer questions properly just evade and evade. Being swiss is no garantee of anything it seems! Good Luck! Paul .

------- Original Message ------- On Thursday, June 2nd, 2022 at 15:24, Justin R. Andrusk @.***> wrote:

Greetings,

I have been running into the same issue for a number of months as I've been running offlineimap with the proton-bridge app on Debian 11. I finally caved in an opened a support case around it. Sent all of the relevant details and logs and the response was that the engineers are aware of the issue and are working on a fix.

I just switched over to mbsync/isync from offlineimap and the same issue persists, but mbsync seems to update faster than offlineimap, so it's slighlty less annoying. Hopefully it's fixed in the next release ;)

Regards,

Justin Andrusk "Paulj585" @.***> writes:

Hi, I have used PM for 2 years no problem on Win7 Thunderbird also Linux Mint then after receiving a daily notification from PM, which I had turned off BTW, I clicked on it and then my problems began.... My CPU usage went through the roof and slowly and inexorably my system memory and network BW was gobbled up until OS froze. Using my Linux mint system the same thing was happening image Both systems were sync'd and up to date. PM support have said they have "seen this problem but only rarely" It seems like both sysyems are suffering a DDOS attack from PM. Hundreds of Gbytes of data is being downloaded onto my machines windows and linux, of what I have no idea. Despite asking PM what is being sent down the line they have not answered in any detail. It seems to me this is a "drafts" problem. On restart the system seems quiescent just sitting there. Begin composing a mail and when it autosaves to drafts the problem begins. It tries to upload to PM server and then screws up. After going through all the usual and clearing and reinstalling on advice from PM desktop support the problem is still there. My Linux system was left on after setting up T'bird from scratch. The problem was still existing and my entire system memory swap file etc was swallowed up and it seems to have overwritten part of the OS (that's a guess) because the system became locked and would not boot up. In the end I had to reformat and reinstall. The problem still remains. This problem is further widespread than PM admits and looking at the forums has been around for at least 2 years. TBH it looks like that PM are not taking this problem seriously. I have not so far seen any sign of a solution on any of the forums other than stop using bridge. Not really a suitable situation.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

-- Regards, Justin

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

telephon commented 2 years ago

I've had good responses to the various bug reports with regards to the current issue. The preliminary fix (the bridge caching the mails so that at least we don't download all of them all the time) has been an improvement. But still it is true that it is a massive waste of energy and resources (computer heating up).

My suspicion has always been that it is connected to filter rules in the client that moves mails to offline folders, combined with a large number of mails processed.

Paulj585 commented 2 years ago

Hi, I tried everything they suggested and it just made the problem worse, including the caching adjustments. For me the errant behaviour began after starting to compose a mail. If I restarted TB it might download a couple emails then TB would just sit there all nice. As I began writing when TB tried to save a draft the memory and cpu usage would just start creeping up until my machine became unusable. So I cannot now use TB anymore. My frustrations with the helpdesk are principally that they are not telling me what is all this gbytes of stuff thats being pushed onto my machine? Why are they not saying? Could it be the contents of others accounts? What are they doing their end other than giving us ineffective tasks? I can no longer use TB and Bridge which is very awkward and PB was the reason I switched to the paid service. It's further annoying because swapping email addresses is no mean thing and customers like us put a lot of faith in companies when they transfer. Sadly at present Proton Mail don't seem to be any different to any other corporate entity. If they are saying the next upgrade will solve the problem then that presumably means they know what the problem is. But they are also saying we don't know what this problem is as it happens only "rarely". These two statements don't match up. I'm not sure we'll ever get a proper explanation. Cheers

Paul

Sent with Proton Mail secure email.

------- Original Message ------- On Saturday, June 4th, 2022 at 07:49, Julian Rohrhuber @.***> wrote:

I've had good responses to the various bug reports with regards to the current issue. The preliminary fix (the bridge caching the mails so that at least we don't download all of them all the time) has been an improvement. But still it is true that it is a massive waste of energy and resources (computer heating up).

My suspicion has always been that it is connected to filter rules in the client that moves mails to offline folders, combined with a large number of mails processed.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

FinestMaximus commented 2 years ago

im removing this pos till this issue is fixed.. drained my battery in 45mn... would expect better from something im paying for..

andrzejsza commented 1 year ago

With Bridge v3 (current beta 3.0.7) the resource consumption is much more sane. We've increased the number of workers used for Bridge/Client sync - to improve performance - but are open to tweaking things to see what works best for most users. Do let us know how things are looking now.

sjatkins commented 1 year ago

Running at 179% of cpu is not at all sane. That is what I am currently experiencing on linux. Surely the process could have been niced a bit?

telephon commented 1 year ago

Just for the record: on macOS 12, the issue has disappeared, at least on my mac machine. I hope it will get addressed on linux now.

petrmanek commented 1 year ago

I am currently running v2.3.0-1 on Arch an since the last upgrade my CPU usage spiked up again. Now at ~350% every now and then with a 4-core CPU.

andrzejsza commented 1 year ago

we're still figuring out the optimal number of workers and trying to achieve the best compromise between high performance of sync and CPU usage. will update with more info soon.

tunnell commented 1 year ago

Is it possible to add user options to cap the number of cores and/or schedule any resync (starts at night, then pauses during day, restarts at night)? This keeps killing my battery.

LBeernaertProton commented 1 year ago

To those who reported high CPU usage, have you upgraded to v3 and managed to reproduce the issue or are you still referring to v2?

anthonytw commented 1 year ago

The issue persists for me with v3.3.2 br-157 on macOS 13.4.1. If I'm looking at the activity monitor, it's energy impact is ~130 with the next highest (Brave with a number of tabs open) at around 6. My fan runs pretty much non-stop unless I close the app, which I do periodically, then it seems to reset itself and act normally for a day or two.

I tried to completely refresh the account a few months back, removing the application and account from the mac and trying to find and delete all cached mailbox items. Then I reinstalled and let it sync. But it still regularly gets confused and starts doing who-knows-what, with it taking a minimum of 100% CPU until it's restarted.

LBeernaertProton commented 1 year ago

@anthonytw can you confirm if you finished syncing? We only expect to see high CPU usage when the accounts are syncing the data and building the local cache.

Additionally, we made some improvements to the database layout in 3.4 which should also help. I would recommend trying with that version.

anthonytw commented 1 year ago

As far as I can tell it's synced completely. I just asked it to sync again through the Mail app and it finished quickly. But what prompted me to search out this ticket in the first place was that the last week or so it's been going full-throttle most of the time it's been open (I don't think I have that much mail!) I don't really know how much of this is on the bridge side versus the mail app side, although I would say it doesn't have the same problem syncing with my work's GMail account.

Right now there are no issues. The last time I closed bridge, mail, and rebooted, and now everything is running normally. If I notice the behavior start again I will ping this ticket again with more details.

wrt 3.4, the app tells me 3.3.2 is up to date, maybe 3.4 is a beta? I'll keep beta off for now since it's not causing more than occasional annoyances, but thanks for letting me know. (And I'll know what's going on if it seems to be rebuilding the next time it updates...)

LBeernaertProton commented 1 year ago

Occasionally, the Proton API may send a "refresh" event your way, that causes the all state to be refreshed. There was one wave of them recently. It is possible you received such an event.