kuroahna
commented
2 years ago I did some extra testing, and I found something very strange. I edited anki-connect's __init__.py making
def isNoteDuplicateOrEmptyInScope(
self,
note,
deck,
collection,
duplicateScope,
duplicateScopeDeckName,
duplicateScopeCheckChildren,
duplicateScopeCheckAllModels
):
return 2
# Returns: 1 if first is empty, 2 if first is a duplicate, 0 otherwise.
# note.dupeOrEmpty returns if a note is a global duplicate with the specific model.
# This is used as the default check, and the rest of this function is manually
# checking if the note is a duplicate with additional options.
if duplicateScope != 'deck' and not duplicateScopeCheckAllModels:
return note.dupeOrEmpty() or 0where the function immediately returns that every card is a duplicate (no database queries) and it's even slower. Scanning a word like すき焼き (has 33 entries with my 32 enabled dictionaries) taking about ~8 seconds to return even though there shouldn't be any database queries. And strangely, it even freezes my anki during these 8 seconds. I try to move the anki window around but it freezes until it resolves the addNote request I assume? Scanning a word like 昨日 instead has 8 entries in the yomichan popup and is much faster but still hangs for ~3 seconds despite not having to do anything.
Making the function return 1 has the same effect as return 2. On the other hand, making the function return 0 makes the duplicate check fast, although takes about ~1 seconds to do the duplicate check for すき焼き and ~0.25 seconds to do the duplicate check for 昨日
I can only assume that this lag I'm experiencing is because we're throwing exceptions when calling createNote ?
duplicateOrEmpty = self.isNoteDuplicateOrEmptyInScope(
ankiNote,
deck,
collection,
duplicateScope,
duplicateScopeDeckName,
duplicateScopeCheckChildren,
duplicateScopeCheckAllModels
)
if duplicateOrEmpty == 1:
raise Exception('cannot create note because it is empty')
elif duplicateOrEmpty == 2:
if allowDuplicate:
return ankiNote
raise Exception('cannot create note because it is a duplicate')
elif duplicateOrEmpty == 0:
return ankiNote
else:
raise Exception('cannot create note for unknown reason')Anki hanging and freezing for ~8 seconds when hovering over a word for checking duplicates is really concerning though, is anki-connect blocking and running on the main thread? Or is it running on a separate thread?
toasted-nutbread
peldas
Perhaps this should be raised in
anki-connectinstead of Yomichan, but if I understand it properly, Yomichan calls anki-connect's APIs to do the duplicate card checking And we do database queries such asselect id from notes where csum=? and mid=?select id from notes where csum=? and id!=?select did from cards where nid=?select id, field_at_index(flds, 0) from notes where csum=? and mid=?if duplicateScope != 'deck' and not duplicateScopeCheckAllModelswhich callsis_duplicate()and does the database queryIt would be nice if we cached all the cards in the anki collection that we know exists and avoid having to query the database to see if it's a duplicate card or not.
Would it be as simple as going through all the notes in the anki collection on
anki-connectstartup, caching the note's first field checksum into a Set, and then when we want to check if a new note is a duplicate or not, we get the checksum of the new note's first field and check if it is in the Set? This will probably slow anki on startup since it has to build the cache, but perhaps this could be an optional option/flag? Also not sure how much memory this will take, but my computer has 16 GB of RAM which should be plenty, and I feel like most computers nowadays ship with 8 GB at leastI heavily use the duplicate card check feature, and when hovering some words/phrases like 今日 or 大体の, it takes about ~3 seconds to show that it is a duplicate in my anki collection. But for other words like 食べる, it takes about ~1 second. I have tried this on both firefox and chrome, and they're both the same speed.
I have ~17.5k cards created so far, and I have 34 dictionaries installed, so Yomichan feels somewhat sluggish.
I can share my anki collection if needed.
EDIT: Yomichan version: 22.4.4.0 Chrome: 102.0.5005.115 (Official Build) (64-bit) Firefox Developer Edition: 102.0b9 (64-bit)