Not scanning some characters due to "DataError: Failed to read large IndexedDB value"

[darren-moore]

Description Yomichan wasn't scanning some kanji, e.g. with 手, no entries would appear. Then I noticed this in the console while I was playing around on the settings page.

The settings page said it was using ~1.1GB of data. I deleted a couple dictionaries and it seemed to fix it: I could scan 手 at least. Re-adding the deleted dictionaries did not reproduce the broken behaviour. Perhaps some corruption that was resolved? Note that when I switched computers, I copied the IndexedDB file rather than reimporting all dictionaries. This could have been the cause.

Browser version Microsoft Edge Version 106.0.1370.42 (Official build) (64-bit)

Yomichan version 22.9.9.2

Exported settings file yomichan-settings-2022-10-15-17-02-04.txt

[toasted-nutbread]

What dictionaries did you have installed?

[darren-moore]

What dictionaries did you have installed?

I deleted the last 2 and it began scanning 手.

JLPT文法解説まとめ rev.nihongo_kyoshi_v1.03; 2022-05-27; p.o.s. info
旺文社国語辞典 第十一版 rev.OUKOKU11_1.6
明鏡国語辞典 rev.Meikyou1
ハイブリッド新辞林 rev.shinjirin
実用日本語表現辞典 rev.jitsuyou1
新明解国語辞典 第五版 rev.Shinmeikai5
JMdict (English) rev.jmdict4
デジタル大辞泉 rev.daijisen_20210506;2021-07-27
KANJIDIC (English) rev.kanjidic2
VNs Freq rev.frequency2019
JLPT_Level rev.frequency1
JPDB rev.JPDB_by-frequency-global_2022-05-10T03:27:02.930Z
アクセント辞典 rev.pitch1
KKLC Order rev.frequency2022
漢字遣い参考 rev.jmdict_special;2021-07-27
研究社　新和英大辞典　第５版 rev.wadai1
Netflix Freq rev.frequency2019

[darren-moore]

Actually, I can repro with 大. Anything that begins with 大 will not display any entries.

[toasted-nutbread]

Do you still have the .zip files for these dictionaries? Do you know which is the largest? My initial guess is that one of them contains images/media that has a large file size.

Are you able to scan other Japanese text? I.e. does this issue break all text scanning, or only certain words/characters?

[darren-moore]

Are you able to scan other Japanese text? I.e. does this issue break all text scanning, or only certain words/characters?

It breaks only certain words/characters. I'm not sure if there is specific criteria for when it breaks.

Do you still have the .zip files for these dictionaries? Do you know which is the largest? My initial guess is that one of them contains images/media that has a large file size

I do still have them. I tried deleting the largest(~300MB, lots of media) and brought the storage size under 1GB, yet the problem persisted.

I tried creating a new browser profile, added the dictionaries, and imported my settings: everything works. Furthermore, the storage size is about half what is listed on my default browser profile despite having the same dictionaries installed. Is there something else stored in the DB that hasn't been populated? Or maybe deleting doesn't clean up correctly?

Like I said in my original post, I did some funny business when swapping computers to avoid reimporting, which could have caused this. I just tried deleting all dictionaries, but several hundred MB were still being used. Only after clicking "Clear site data" and reimporting all dictionaries in the developer tools, everything seems to be working correctly.