search

themoeway / yomitan

Pop-up dictionary browser extension. Successor to Yomichan.
https://yomitan.wiki
GNU General Public License v3.0
1.19k stars 94 forks source link

Yomitan crashing during long reading sessions #836

Open mofufuwa opened 5 months ago

mofufuwa commented 5 months ago

Description My experience: Hii yomitan team, I have been having this issue for weeks now since the 18th of march but idk if its been a thing before that as I only started immersing then. I've noticed while I'm reading that yomitan overtime gets slower and slower then begins to stutter, then it takes so long to check words that it seems like its frozen before finally crashing. This is usually after an hour of reading that it begins to slow down and you can notice some lag on lookups. Then by the two hour mark its much more noticeable and on the third hour stuttering and heavy lag and if you haven't crashed by now you will on the fourth hour. Sometimes it varies before things get bad (2 hour mark description) which im assuming has to do with look up amount. But even by the end of my second novel また、同じ夢を見ていた where i was doing much less lookups then i had been at the beginning i would still slowdown heavily overtime and crash eventually. When it crashes, i get prompted by edge to click a window popup thaat appears telling me it crashed, to restart it, however after doing so it still doesnt work till i also refresh ttsu reader itself. If i dont do so i only get a grey box with a dead face X_X on looksups. I have currently only bene reading novels on ttsu so im unsure if this is the same with VNs and textractor pages or manga mokura etc sorry. my specs:

This is the end of my experience and comments. I will put two other people who have commented also experiencing lag and crashes with yomitan below. They are from the thread I made in TMW called "yomitan Crashing" I pinged one of the devs (darius there aswell,. sorry if that wasnt okay)

rbjunk's comment: "Been having the same issues, but instead of crashing mine just gets slower and stutters more often. Only been happening the last few weeks. Firefox btw"

Ikki・ｲｯｷ 🍡's comment: "I've been having this exact problem on Windows (16gb ram, AMD 3700X, RTX 2080), Linux (i5, 24gb ram), and my new M3 Pro Macbook pro with 36gb ram... Tested on Firefox and Chrome.

I've had this problem since I started (1.5y ago, yomichan then yomitan), and I read for long hours on ttsu only.

So my theories are :

The problem however is clear : I've been profiling this issues for weeks 2 months ago, and that's RAM consumption starts at 0, then it slowly increases the more I look up words.

I've been looking into the source code, there is some cache for the previous words seen, but I've understood it's supposed to be limited. As if this limit didn't apply on my machines, this can be the problem if the cache doesn't get discarded sometimes.

However, I think a broken/too heavy dictionary can be the remote cause as to why we have this problem when other people haven't, since it's a differentiating factor. The root cause being yomitan can't discard its memory used. "

Browser version Me: Microsoft Edge Version 123.0.2420.97 (Official build) (64-bit) rbjunk: Firefox (idk version sorry) Ikki・ｲｯｷ 🍡: Firefox and Chrome (idk version sorry)

Yomitan version Me: Yomitan 24.4.16.0 Unsure of other two but I'm assuming newest version cause the extension auto-updates

Exported settings file (If you think it might be relevant, create a settings export file using the Export Settings button on the settings page and attach it to this issue.)

andrewsartor commented 5 months ago

I've been unable to reproduce this locally. Can you open up your browser's process manager (should be shift + esc in edge, firefox, and chrome) during one of these slowdowns and see which specific process is consuming excess RAM / CPU? If you can do that and provide a screenshot showing that it's Yomitan and not some other tab / process, that'll make finding the root cause of the issue at least a little bit easier.

mofufuwa commented 5 months ago

@andrewsartor im so sorry for the late reply, ive never used github before so i wasnt aware of the notification. I just remembered about this 2 hours again and luckily i was about to read so i got a screen shot of 2 hours of reading results. I noticed that whenever i looked up words the ram usage would icnrease and continue to increase as i look more up. scrolling in the yomitan window also increases usage but only temporarily. Also sometimes Cpu shoots up. During this time i have had stutters every now and then and a few times second delay for looksups. Im going to read more later and send another screenshot if it gets worse to show you those results too. ty for your help! 2024_04_18_18-27__TT4

andrewsartor commented 5 months ago

Thanks for the further context. I'd also like to see the contents of your settings file as well as the dictionaries you have installed, if you wouldn't mind providing those. I have a hunch as to where this issue may be cropping up, but seeing the contents of your settings may help me validate that.

mofufuwa commented 5 months ago

@andrewsartor Here you go :D these are my dictionaries and setttings, Im using the settings found in donkuris setup guide but it may have been altered overtime so i exported mine instead of getting that one again. yomitan-settings-2024-04-19-15-47-08.json 2024_04_19_11-46__LdY

2024_04_19_11-46__GI3

Also another person commented on the thread in TMW and said this

4890: "Also experiencing this with ttsu/ kiwi on my ereader. Feels nearly 10x slower to look up after 30 minutes Doesn’t appear tied to total system ram usage as far as I can tell. I think I’ve always had this issue with ttsu, but I only started using ttsu recently I’ll see if I can check yomitans ram usage in kiwi"

Leyki commented 3 months ago

I've been experiencing similar issues since using yomichan, now yomitan and still ongoing. image

I tested feeding it lorem ipsum which resulted in no dictionary hits; whenever i'd hover text memory will go up and then stabilize, contrary to feeding it straight from the clipboard where memory will raise, drop some but not entirely.

cpettitt commented 1 month ago

I have also observed the same memory leak with Yomitan, but it actually went all the way back to the original Yomichan (maybe that is a clue, edit: original said they observed the same). I have to kill Yomitan every couple of days.

I've seen the leak on Windows and Linux, both running Yomitan in Brave (chromium derived) browser.

cpettitt commented 1 month ago

Not super familiar with yomitan's code, but it looks like it is hooking up event listeners in display-anki.js and they are retaining a whole heap of detached DOM elements. I didn't see where the listeners are getting removed on a quick inspection.

Also, I see this retention happening in display-anki.js whether or not Anki integration is enabled.

jamesmaa commented 1 month ago

@cpettitt how are you observing the heap of detached DOM elements? I tried reproducing it on my chrome browser but couldn't find any detached DOM elements. Could you record a heap snapshot and upload it on here so I can see for myself? Do you also happen to have a Minimal reproducible example?

cpettitt commented 1 month ago

Repro steps:

  1. Open a fresh instance of the browser or kill and restart Yomitan (for me Yomitan is always the brave process using the most memory).
  2. Go to a site with some Japanese, but not too much or heap gets big too quickly. (e.g. https://www3.nhk.or.jp/news/easy/)
  3. Open Chrome Memory profiler.
  4. Highlight one word with Yomitan to get Yomitan's popup.html to show up in the profiler.
  5. Take a baseline heap snapshot
  6. Highlight several words on the Japanese page.
  7. Run the GC a few times.
  8. Take second heap snapshot (wait for a while because the dominator tree algorithm is O(n^2)).
  9. Diff the first and second heap snapshots and sort by retained size.

yomitan-screenshot

Do you have somewhere I can post the heap dumps?

jamesmaa commented 1 month ago

I think you can upload the file directly on GH?

cpettitt commented 1 month ago

@jamesmaa I tried as an attachment, but it wasn't a supported file type. Also the second heap dump is too big per https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/attaching-files. Is there another way you're thinking of posting on github?

jamesmaa commented 1 month ago

Following your steps on chrome: Baseline: no yomitan activation

image

1 yomitan scan:

image

Many yomitan scans:

image

Waiting 1 min after:

image

Heap profile of "many yomitan scans" (zipping it went from 25mb to 5mb) Heap-20240820T171532.heapsnapshot.zip

jamesmaa commented 1 month ago

And yeah I can't find "Detached HTMLButtonElement" in my memory profiler

image
jamesmaa commented 1 month ago

BTW we have previously resolved another user's slowness by removing JMDict revision jmdict.rev4. If you happen to have that dictionary, it might be worth removing it to see if your slowness resolves. Please respond to this thread if that fixes your issue.

Source: https://discord.com/channels/617136488840429598/1264302000728965150/1264343629464600576

cpettitt commented 1 month ago

Thanks for the tip about attachments. Will try to get one up in an hour or so.

On Tue, Aug 20, 2024, 18:18 James Maa @.***> wrote:

And yeah I can't find "Detached HTMLButtonElement" in my memory profiler image.png (view on web) https://github.com/user-attachments/assets/9fc5f4a5-d922-400d-b3b4-ef902a598e89

— Reply to this email directly, view it on GitHub https://github.com/themoeway/yomitan/issues/836#issuecomment-2299977783, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAC35UJNXP2TJMKYDKCOZTZSPMFTAVCNFSM6AAAAABGMGUCIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJZHE3TONZYGM . You are receiving this because you were mentioned.Message ID: @.***>

cpettitt commented 1 month ago

@jamesmaa Yeah, I don't think it is particularly the button, but several HTML elements (including divs, spans, etc.) that are being retained through the listener. Your screenshots definitely do not show what I am seeing though.

I'm running mostly monolingual dictionaries. I did have "jitendex", which I think might be jmdict derived, but I removed it and had the same repro. Please see attached heap dumps.

yomitan-snapshots.zip