search

limpkin / mooltipass

Github repository dedicated to the mooltipass project
https://www.themooltipass.com
522 stars 113 forks source link

Chrome extension using 100% CPU and a lot of RAM #560

Closed vatral closed 5 years ago

vatral commented 5 years ago

Expected behavior

Extension should use very little CPU power. little RAM

Actual behavior

Task manager: Background Page: Mooltipass App: 99.9% CPU, 416720K RAM Extension: Mooltipass Extension: 57% CPU, 227800K RAM

Step by step guide to reproduce the problem

Reproduction so far is unknown, but happens predictably on a long enough timeframe. I run my laptop with uptimes of weeks on a regular basis, with Chrome staying up just as long. Given the data from below it's possible that something leaks constantly, and just becomes visible once it adds up enough.

I'm not skilled in debugging Chrome extensions, but I'm willing to poke around in the debugger.

I tried the DevTools profiler, results so far:

CPU time over one minute is:

1158ms scripting 29828ms Other 28908ms Idle

JS heap goes 5454000 to 5796000 after a GC Listeners go from 15 to 200 Nodes go from 38 to 204

The overall impression is that something is leaking, and over time it adds up to a lot.

The console tab contains:

11moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
connect @ moolticute.js:299
(anonymous) @ moolticute.js:211
moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
connect @ moolticute.js:299
(anonymous) @ moolticute.js:211
DevTools: CPU profile parser is fixing 1 missing samples.
67moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
connect @ moolticute.js:299
(anonymous) @ moolticute.js:211
5Error in event handler for webRequest.onBeforeRequest/1: SyntaxError: Unexpected token , in JSON at position 3
    at Object.toJSON (<URL>)
    at chrome.webRequest.onBeforeRequest.addListener.urls (<URL>)
184moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

Indeed, there's nothing running on that port. Starting a socat produces this, after 5 seconds or so:

GET / HTTP/1.1
Host: 127.0.0.1:30035
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.92 Safari/537.36
Upgrade: websocket
Origin: chrome-extension://ffemldjbbfhimggdkkckolidemlllklk
Sec-WebSocket-Version: 13
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Sec-WebSocket-Key: Zq1OjQgoqyI34wt1Om3TrA==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

Mooltipass Extension

Chromium, extension 1.6.2 on Linux

Mooltipass Application

Mooltipass App, 2.0.81

Mooltipass Device

The Mooltipass Mini, currently not connected. It's a laptop that moves around, so it's not always plugged in.

limpkin commented 5 years ago

Hello,

Thanks for bringing this to our attention! We indeed encountered this previously but weren't able to reproduce it. Can you let us know if in your case the mooltipass always is disconnected or do you think it doesn't matter?

On Sun, Oct 7, 2018, 01:34 Vadim Troshchinskiy notifications@github.com wrote:

Expected behavior

Extension should use very little CPU power. little RAM Actual behavior

Task manager: Background Page: Mooltipass App: 99.9% CPU, 416720K RAM Extension: Mooltipass Extension: 57% CPU, 227800K RAM Step by step guide to reproduce the problem

Reproduction so far is unknown, but happens predictably on a long enough timeframe. I run my laptop with uptimes of weeks on a regular basis, with Chrome staying up just as long. Given the data from below it's possible that something leaks constantly, and just becomes visible once it adds up enough.

I'm not skilled in debugging Chrome extensions, but I'm willing to poke around in the debugger.

I tried the DevTools profiler, results so far:

CPU time over one minute is:

1158ms scripting 29828ms Other 28908ms Idle

JS heap goes 5454000 to 5796000 after a GC Listeners go from 15 to 200 Nodes go from 38 to 204

The overall impression is that something is leaking, and over time it adds up to a lot.

The console tab contains:

11moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED connect @ moolticute.js:299 (anonymous) @ moolticute.js:211 moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED connect @ moolticute.js:299 (anonymous) @ moolticute.js:211 DevTools: CPU profile parser is fixing 1 missing samples. 67moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED connect @ moolticute.js:299 (anonymous) @ moolticute.js:211 5Error in event handler for webRequest.onBeforeRequest/1: SyntaxError: Unexpected token , in JSON at position 3 at Object.toJSON () at chrome.webRequest.onBeforeRequest.addListener.urls () 184moolticute.js:299 WebSocket connection to 'ws://127.0.0.1:30035/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

Indeed, there's nothing running on that port. Starting a socat produces this, after 5 seconds or so:

GET / HTTP/1.1 Host: 127.0.0.1:30035 Connection: Upgrade Pragma: no-cache Cache-Control: no-cache User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.92 Safari/537.36 Upgrade: websocket Origin: chrome-extension://ffemldjbbfhimggdkkckolidemlllklk Sec-WebSocket-Version: 13 Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.9 Sec-WebSocket-Key: Zq1OjQgoqyI34wt1Om3TrA== Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

Mooltipass Extension

Chromium, extension 1.6.2 on Linux Mooltipass Application

Mooltipass App, 2.0.81 Mooltipass Device

The Mooltipass Mini, currently not connected. It's a laptop that moves around, so it's not always plugged in.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560, or mute the thread https://github.com/notifications/unsubscribe-auth/AF4ms4X9Et_Q1L_t7kuCUQj3JFRVVuWRks5uiT4SgaJpZM4XLmhd .

vatral commented 5 years ago

It may matter. For me it's almost always disconnected, as the laptop moves quite a lot, so I only end up connecting it when it's needed.

vatral commented 5 years ago

Some further poking around:

So far my suspicion remains that it's a resource leak -- every time a timer fires, some resources get lost, and eventually managing those resources starts to take serious CPU time.

limpkin commented 5 years ago

It does look like a good idea indeed.
Could you let us know what's your computer configuration?

vatral commented 5 years ago

Fedora 28, Chromium Version 69.0.3497.92 (Developer Build) Fedora Project (64-bit) Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz 32 GB RAM

The problem happens both on the desktop and laptop, but on the laptop it's far more noticeable due to shutting down far less often. On the desktop, CPU usage has climbed up to 8% since I made that previous comment an hour ago.

limpkin commented 5 years ago

I wonder if this isn't OS related... do you have other OSes to try to replicate this bug? Your help is extremely appreciated... I'd really like to finally get rid of that bug.

vatral commented 5 years ago

I can try to setup a VM and seeing what happens. What do you recommend, Ubuntu?

limpkin commented 5 years ago

That's a good one yes.
I wonder if the regular ping messages between extension and moolticute aren't creating this resource usage. Do you also get this problem with a single tab opened?

vatral commented 5 years ago

Okay, I'm on it.

So far I'm able to reproduce the issue even by: closing Chromium, starting it again, going to chrome://extensions/ and inspecting the background page.

vatral commented 5 years ago

Installed Ubuntu 18.04, updated, installed chromium-browser, installed extension (no app yet). Chromium Version 69.0.3497.81 (Official Build) Built on Ubuntu , running on Ubuntu 18.04 (64-bit)

Behavior looks like described, errors connecting to port 30035, nodes and listeners keep going up.

Added chrome app, same thing.

limpkin commented 5 years ago

Could you check on firefox as well? :)

On Sun, Oct 7, 2018 at 7:09 PM Vadim Troshchinskiy notifications@github.com wrote:

Installed Ubuntu 18.04, updated, installed chromium-browser, installed extension (no app yet). Chromium Version 69.0.3497.81 (Official Build) Built on Ubuntu , running on Ubuntu 18.04 (64-bit)

Behavior looks like described, errors connecting to port 30035, nodes and listeners keep going up.

Added chrome app, same thing.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#issuecomment-427669501, or mute the thread https://github.com/notifications/unsubscribe-auth/AF4ms01l9BaQmhKQFRbYWM4_JCpDBGMaks5uijUfgaJpZM4XLmhd .

vatral commented 5 years ago

I can't quite figure out how to get the same information from Firefox.

I got as far as enabling debugging for extensions. I get a lot of this on the console:

 Sending message to content: 
Object { action: "check_for_new_input_fields" }
 2 event.js:4:35
[Show/hide message details.] Unchecked lastError value: Error: Could not establish connection. Receiving end does not exist. event.js:11

How do I get performance stats from an extension in Firefox?

limpkin commented 5 years ago

hmmm, it seems the few tools there are aren't very convenient for what we'd like to do.
Anyway, tomorrow one of our devs will try to replicate again this bug... you may expect him to get in touch with you directly :)

vatral commented 5 years ago

Okay, thank you :) I'll see if I can poke around in here some more meanwhile.

ctnewbold commented 5 years ago

Just to chime in here and say I’m having exactly the same problem with the extension and/or app under Chrome on macOS. I originally reported the issue via e-mail and provided some execution and heap profiling using the Chrome dev tools. Just for fun I also grabbed a native sample of the process hosting the extension using Apple’s Activity Monitor. The thread that’s consuming all the CPU is spending all of it’s time allocating memory from the system. I couldn’t find an easy way to directly correlate this with whatever is going on up in JavaScript land, but it sure is consistent with a continuous and infinite resource leak. Happy to help dig in to the extent that a pure C++ guy can.

vatral commented 5 years ago

Poked around some more. It seems to me so far that moolticuted running makes it a whole lot better behaved, if not fixes the problem entirely. I've been watching the stats and without moolticuted usage starts climbing again. It does so very slowly, it takes hours to see CPU usage go up.

So the scenario to test seems to be the plugin without moolticuted running.

Also, without moolticuted, in DevTools/Network these keep getting piled up:

127.0.0.1 | Finished | websocket | moolticute.js:299 | 0 B | Pending
-- | -- | -- | -- | -- | --
127.0.0.1 | (pending) | websocket | moolticute.js:299 | 0 B | Pending

I'm not sure if this has any meaning, perhaps the 'Pending' status indicates something hasn't been fully released?

limpkin commented 5 years ago

Hello everyone,
Just letting you know we're indeed currently debugging this issue.

limpkin commented 5 years ago

still working on it....

limpkin commented 5 years ago

Ok, I think I got it and if it's true this is going to be hilarious.

@vatral & @ctnewbold could you uninstall your extensions and install the testers one please?

https://chrome.google.com/webstore/detail/mkjlelalgdinanmcljpgkojjolkdcebh , should be version 1.6.3

crossing fingers!

vatral commented 5 years ago

Installed it on two computers so far, been keeping an eye on it.

So far it seems fairly well behaved, although memory usage has grown over time. The App started at 35 MB and is now at 50 MB, while the extension went from 28 MB to 56 MB.

CPU usage is currently hovering around 2-5% for the App and 1-2% for the extension, and it's constant. I'll be keeping an eye on it to see if it gets higher.

I would say that a constant CPU load, even if a low one doesn't look very right to me. It's a password manager -- shouldn't there be nothing for it to do 99.9% of the time? It's not a huge concern on a desktop, but on a laptop it could be a battery drain.

limpkin commented 5 years ago

Please let me know how the CPU and RAM usage evolves. Actually there's a regular parsing of the web page to check for credential fields...

On Fri, Oct 19, 2018, 19:51 Vadim Troshchinskiy notifications@github.com wrote:

Installed it on two computers so far, been keeping an eye on it.

So far it seems fairly well behaved, although memory usage has grown over time. The App started at 35 MB and is now at 50 MB, while the extension went from 28 MB to 56 MB.

CPU usage is currently hovering around 2-5% for the App and 1-2% for the extension, and it's constant. I'll be keeping an eye on it to see if it gets higher.

I would say that a constant CPU load, even if a low one doesn't look very right to me. It's a password manager -- shouldn't there be nothing for it to do 99.9% of the time? It's not a huge concern on a desktop, but on a laptop it could be a battery drain.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#issuecomment-431443932, or mute the thread https://github.com/notifications/unsubscribe-auth/AF4ms4w3dHCCAAndDmhTyGXWIQRznrvHks5umhC_gaJpZM4XLmhd .

vatral commented 5 years ago

Okay, desktop has been on since last post, and mostly idle. Current stats:

App: 61 MB, 4-6%. Extension: 57 MB, 2-3%.

So seems to be still climbing a bit, very slowly. Unfortunately I've got to reboot it now, but tomorrow I plan to leave it running all day and see if it's clearer.

vatral commented 5 years ago

Also installed it on the laptop. Extension 1.6.3, app 2.0.81

App is at 81MB, using 20-40% CPU. Extension is using 10-16% CPU. Closing everything but this tab CPU usage stabilizes around 8-9% for the extension and 20-25% for the App.

Now that you mentioned the regular parsing, this brings more testing ideas to mind. I'll do some experiments here, but I suspect that something might not be freed when a tab is closed.

Also, I noticed CPU usage spikes considerably when playing a youtube video, to around 60% for the app and 30% for the extension. Specifically when playing, usage is lower when the video is paused.

limpkin commented 5 years ago

Are you using the chrome app? Could you remove it and install moolticute instead? This should reveal interesting things

On Sat, Oct 20, 2018, 14:55 Vadim Troshchinskiy notifications@github.com wrote:

Also installed it on the laptop. Extension 1.6.3, app 2.0.81

App is at 81MB, using 20-40% CPU. Extension is using 10-16% CPU. Closing everything but this tab CPU usage stabilizes around 8-9% for the extension and 20-25% for the App.

Now that you mentioned the regular parsing, this brings more testing ideas to mind. I'll do some experiments here, but I suspect that something might not be freed when a tab is closed.

Also, I noticed CPU usage spikes considerably when playing a youtube video, to around 60% for the app and 30% for the extension. Specifically when playing, usage is lower when the video is paused.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#issuecomment-431578335, or mute the thread https://github.com/notifications/unsubscribe-auth/AF4ms1y-AjP1uSsk8Ig-eznybsRF0Nwuks5umx1OgaJpZM4XLmhd .

ctnewbold commented 5 years ago

I know nothing about how these extensions work, but why would you need to continually reparse a page? Isn't there some sort of event mechanism that would notify you that the content changed so you only need to reparse if something changed? Anyway, I will install the test versions today and see what happens. Thanks for all of the effort to investigate this!

On Sat, Oct 20, 2018, 8:55 AM Vadim Troshchinskiy notifications@github.com wrote:

Also installed it on the laptop. Extension 1.6.3, app 2.0.81

App is at 81MB, using 20-40% CPU. Extension is using 10-16% CPU. Closing everything but this tab CPU usage stabilizes around 8-9% for the extension and 20-25% for the App.

Now that you mentioned the regular parsing, this brings more testing ideas to mind. I'll do some experiments here, but I suspect that something might not be freed when a tab is closed.

Also, I noticed CPU usage spikes considerably when playing a youtube video, to around 60% for the app and 30% for the extension. Specifically when playing, usage is lower when the video is paused.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#issuecomment-431578335, or mute the thread https://github.com/notifications/unsubscribe-auth/AN_0Ttxx2tyrenMROtsdQt08ksMsmMpnks5umx1PgaJpZM4XLmhd .

limpkin commented 5 years ago

It's a tad more complex, as most websites do have all sorts of JavaScript running.... Thanks for testing!

On Sat, Oct 20, 2018, 15:11 ctnewbold notifications@github.com wrote:

I know nothing about how these extensions work, but why would you need to continually reparse a page? Isn't there some sort of event mechanism that would notify you that the content changed so you only need to reparse if something changed? Anyway, I will install the test versions today and see what happens. Thanks for all of the effort to investigate this!

On Sat, Oct 20, 2018, 8:55 AM Vadim Troshchinskiy < notifications@github.com> wrote:

Also installed it on the laptop. Extension 1.6.3, app 2.0.81

App is at 81MB, using 20-40% CPU. Extension is using 10-16% CPU. Closing everything but this tab CPU usage stabilizes around 8-9% for the extension and 20-25% for the App.

Now that you mentioned the regular parsing, this brings more testing ideas to mind. I'll do some experiments here, but I suspect that something might not be freed when a tab is closed.

Also, I noticed CPU usage spikes considerably when playing a youtube video, to around 60% for the app and 30% for the extension. Specifically when playing, usage is lower when the video is paused.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/limpkin/mooltipass/issues/560#issuecomment-431578335 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AN_0Ttxx2tyrenMROtsdQt08ksMsmMpnks5umx1PgaJpZM4XLmhd

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#issuecomment-431580036, or mute the thread https://github.com/notifications/unsubscribe-auth/AF4ms_mCCSyK2nzele4dWp_iuJ2gW-BTks5umyEXgaJpZM4XLmhd .

ctnewbold commented 5 years ago

Okay, I've had the testing version installed and running for nearly 24 hours on my heavily-used work iMac. The extension has grown from ~30MB and 0.6% CPU to ~60MB and ~6% CPU usage. I think this is better than before, but there's still something wrong. When I uninstalled the previous version of the extension I also uninstalled the Mooltipass Chrome app and have yet to install it again. So that's one more variable, but at least shows the presence of the Chrome app isn't implicated. Let me know if there's any data I can gather to help with further improvements.

On Sat, Oct 20, 2018 at 12:31 PM Mathieu notifications@github.com wrote:

It's a tad more complex, as most websites do have all sorts of JavaScript running.... Thanks for testing!

On Sat, Oct 20, 2018, 15:11 ctnewbold notifications@github.com wrote:

I know nothing about how these extensions work, but why would you need to continually reparse a page? Isn't there some sort of event mechanism that would notify you that the content changed so you only need to reparse if something changed? Anyway, I will install the test versions today and see what happens. Thanks for all of the effort to investigate this!

On Sat, Oct 20, 2018, 8:55 AM Vadim Troshchinskiy < notifications@github.com> wrote:

Also installed it on the laptop. Extension 1.6.3, app 2.0.81

App is at 81MB, using 20-40% CPU. Extension is using 10-16% CPU. Closing everything but this tab CPU usage stabilizes around 8-9% for the extension and 20-25% for the App.

Now that you mentioned the regular parsing, this brings more testing ideas to mind. I'll do some experiments here, but I suspect that something might not be freed when a tab is closed.

Also, I noticed CPU usage spikes considerably when playing a youtube video, to around 60% for the app and 30% for the extension. Specifically when playing, usage is lower when the video is paused.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/limpkin/mooltipass/issues/560#issuecomment-431578335 , or mute the thread <

https://github.com/notifications/unsubscribe-auth/AN_0Ttxx2tyrenMROtsdQt08ksMsmMpnks5umx1PgaJpZM4XLmhd

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/limpkin/mooltipass/issues/560#issuecomment-431580036 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AF4ms_mCCSyK2nzele4dWp_iuJ2gW-BTks5umyEXgaJpZM4XLmhd

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#issuecomment-431596633, or mute the thread https://github.com/notifications/unsubscribe-auth/AN_0TvWUCe9Xiq6G0HzI9AQQFeh3pOrVks5um0_GgaJpZM4XLmhd .

limpkin commented 5 years ago

Could you try testing with moolticute installed? because when no app is installed the extension is constantly trying to open sockets to the extension. Anyway I'm happy to hear we're on the good way.

okrych commented 5 years ago

@vatral @ctnewbold Could you please uninstall currently installed extension and try with this version of extension? mooltipass-extension.zip, should be version 1.6.4 Thanks

limpkin commented 5 years ago

@vatral @ctnewbold I've just pushed version 1.6.5 of the extension with major improvements. Could you test it? https://chrome.google.com/webstore/detail/mooltipass-extension/mkjlelalgdinanmcljpgkojjolkdcebh?hl=fr

limpkin commented 5 years ago

pinging @vatral and @ctnewbold

vatral commented 5 years ago

Hi, sorry about that, got a bit busy.

Installed.

ctnewbold commented 5 years ago

Will do.

On Wed, Nov 14, 2018, 6:09 PM Vadim Troshchinskiy <notifications@github.com wrote:

Hi, sorry about that, got a bit busy.

Installed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#issuecomment-438853481, or mute the thread https://github.com/notifications/unsubscribe-auth/AN_0Ti1z_AZ3Bvbi5wzzl_30rpKjND6Bks5uvKKRgaJpZM4XLmhd .

vatral commented 5 years ago

It's been running for about a day now, with quite a bit of web browsing and so far it seems to be working excellently well.

Memory usage seems to have stabilized around 75MB, CPU usage is 0% - 1%.

limpkin commented 5 years ago

woohoo!

vatral commented 5 years ago

After the weekend, everything still seems to be in perfect order. Neither memory usage nor CPU usage seems to be growing as far as I can tell.

Starting from Thu 15, the process has accumulated 23 minutes of CPU time. Considering the long uptime of the laptop, the more than 50 tabs of varied content, and that Chrome itself managed to rack up more than 46 hours worth in the same interval, I would say that's very good.

So I think that at least on my part it seems reasonable to consider the matter solved.

I really appreciate your effort, thanks a lot! This was a considerable annoyance for me, as it tended to drain the laptop's battery rather unpredictably.

You're invited to a beer at FOSDEM, if you are coming (I got the device right after your talk there).

limpkin commented 5 years ago

thanks :)

vatral commented 5 years ago

A week later, everything is still working perfectly fine.

limpkin commented 5 years ago

perfect! let's consider it solved then. Thanks everyone for helping!

ctnewbold commented 5 years ago

A bit late here, but confirming that the most recent version appears to be behaving itself for me after nearly a week of browser abuse. Thank you!

On Mon, Nov 26, 2018 at 12:27 PM Mathieu notifications@github.com wrote:

Closed #560 https://github.com/limpkin/mooltipass/issues/560.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/limpkin/mooltipass/issues/560#event-1987415625, or mute the thread https://github.com/notifications/unsubscribe-auth/AN_0TnW1Ha-oglryXWhbWwVINx0l1c7Aks5uzCRugaJpZM4XLmhd .