search

Stvad / CrowdAnki

Plugin for Anki SRS designed to facilitate cooperation on creation of notes and decks.
MIT License
533 stars 44 forks source link

Anki crashes on start #205

Open chrislongros opened 10 months ago

chrislongros commented 10 months ago

I recently tried the add-on on Anki 23.12.1, but unfortunately although I could create a json file, snapshots do not work. Could you please update the add-on to be compatible with the latest Anki version? I find it very useful for version control of my changes. I would really be thankful.

Christos Longros

aplaice commented 10 months ago

I'm very sorry to hear that! Thanks for reporting!

However, in my testing (on Linux with Anki 23.12.1 Qt6), manual snapshots (via File > CrowdAnki: Snapshot) work, as do the automated snapshot on opening and closing Anki (when the relevant option is turned on).

(Edit (in case you read the previous version of this comment): snapshots on closing also seem to work — I got confused...)

Could you please:

  1. make sure that CrowdAnki has been updated.
  2. provide more details about your system (e.g. OS? Qt5 or Qt6? etc.) and what (if any) error messages are displayed? (so that I can try to reproduce and/or get a better handle on what's going wrong)

Thanks and sorry again!

chrislongros commented 10 months ago 
Anki 23.12.1 (1a1d4d54)  (ao)
Python 3.9.15 Qt 6.6.1 PyQt 6.6.1
Platform: Linux-6.6.8-arch1-1-x86_64-with-glibc2.38

Traceback (most recent call last):
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/file.py", line 148, in __init__
    fd = os.open(
FileExistsError: [Errno 17] File exists: '/home/chris/.local/share/Anki2/addons21/1788670778/user_files/Chris/MyDeck__Medicine__a_YelloW_deck__The_ABSITE_Review__5th_edition__Chapter_15__Trauma/.git/objects/3c/9a31cef15582487e1dad1fde925062de89958a.lock'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "aqt.progress", line 118, in handler
  File "aqt.main", line 218, in on_window_init
  File "aqt.main", line 317, in setupProfile
  File "aqt.main", line 496, in loadProfile
  File "_aqt.hooks", line 4107, in __call__
  File "anki.hooks", line 34, in runHook
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/archiver_vendor.py", line 42, in snapshot_on_sync
    self.do_snapshot('CrowdAnki: Snapshot on sync')
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/archiver_vendor.py", line 50, in do_snapshot
    self.all_deck_archiver().archive(overrides=self.overrides(),
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/archiver.py", line 23, in archive
    self.deck_archiver_supplier(deck).archive(reason=reason)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/anki_deck_archiver.py", line 22, in archive
    repo.stage_all()
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/dulwich_repo.py", line 32, in stage_all
    self.dulwich_repo.stage(status.untracked + status.unstaged)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/repo.py", line 1374, in stage
    self.object_store.add_object(blob)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/object_store.py", line 940, in add_object
    with GitFile(path, "wb", mask=PACK_MODE) as f:
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/file.py", line 92, in GitFile
    return _GitFile(filename, mode, bufsize, mask)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/file.py", line 154, in __init__
    raise FileLocked(filename, self._lockfilename) from exc
dulwich.file.FileLocked: ('/home/chris/.local/share/Anki2/addons21/1788670778/user_files/Chris/MyDeck__Medicine__a_YelloW_deck__The_ABSITE_Review__5th_edition__Chapter_15__Trauma/.git/objects/3c/9a31cef15582487e1dad1fde925062de89958a', '/home/chris/.local/share/Anki2/addons21/1788670778/user_files/Chris/MyDeck__Medicine__a_YelloW_deck__The_ABSITE_Review__5th_edition__Chapter_15__Trauma/.git/objects/3c/9a31cef15582487e1dad1fde925062de89958a.lock')

===Add-ons (active)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
BetterSearch ['1052724801', 2023-06-15T22:44, 'None', '']
Button Colours Good Again ['2494384865', 2022-09-27T12:18, 'None', '']
CrowdAnki JSON exportimport Edit history Collaborate on deck creation ['1788670778', 2023-10-30T19:57, 'None', mod]
Edit Field During Review Cloze ['385888438', 2023-11-01T05:53, '6.17', mod]
FSRS4Anki Helper ['759844606', 2023-12-29T16:38, 'None', mod]
Highlight Search Results in the Browser ['225180905', 2023-10-21T21:17, 'None', '']
Hint Hotkeys ['1844908621', 2023-04-14T16:13, 'None', '']
More Overview Stats 21 ['738807903', 2021-07-30T21:13, 'None', '']
Review Heatmap ['1771074083', 2022-06-30T03:43, 'None', '']

===IDs of active AnkiWeb add-ons===
1052724801 1771074083 1788670778 1844908621 225180905 2494384865 385888438 738807903 759844606

===Add-ons (inactive)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
AnkiHub ['1322529746', 2023-12-27T21:22, 'None', '']
chrislongros commented 10 months ago

The application just crashes unexpectedly. I managed to only export my data in a JSON file just once. After that when the add-on is enabled, Anki just crashes on start. Auto sync on open/close is enabled. Thanks for your quick response!

aplaice commented 10 months ago

Oh no, this looks like a dulwich/git issue! :( (i.e. it's possible that it's indirectly caused by incompatibility between CrowdAnki and the newest version of Anki, but it might just be bad luck).

The direct issue is that at some point, when creating a snapshot, CrowdAnki (precisely the (otherwise amazing!) dulwich library which we use to handle git operations) did not remove a temporary git lock file (the lock files are there to make sure that all git operations are atomic). (For some as yet unknown reason — hopefully just a very rare timing issue, but possibly not.)

This lock file now persists and prevents CrowdAnki from doing anything with that git repository (because dulwich sees the lock file and believes that there is a stale/unfinished operation).

I believe that it should be safe to just remove the lock file, but just in case it's probably better to move it somewhere — say try (in a terminal):

mv -i /home/chris/.local/share/Anki2/addons21/1788670778/user_files/Chris/MyDeck__Medicine__a_YelloW_deck__The_ABSITE_Review__5th_edition__Chapter_15__Trauma/.git/objects/3c/9a31cef15582487e1dad1fde925062de89958a.lock ~/temporary_lockfile

and then try opening Anki with CrowdAnki enabled. Hopefully this was a one-off issue and snapshots will continue as normal. (If not please complain here!)

chrislongros commented 10 months ago

Followed your instructions and removed the .lock file but another one appears. Keep in mind that my collection is relatively big and the JSON exported file is over 100 Mb, that git normally handles. Would git lfs be a better option? Is it combatible with CrowdAnki?

Anki 23.12.1 (8f77e519) (src) (ao)
Python 3.9.15 Qt 6.6.1 PyQt 6.6.1
Platform: Linux-6.6.8-arch1-1-x86_64-with-glibc2.38

Traceback (most recent call last):
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/file.py", line 148, in __init__
    fd = os.open(
FileExistsError: [Errno 17] File exists: '/home/chris/.local/share/Anki2/addons21/1788670778/user_files/Chris/MyDeck__Medicine__AnKing__Zanki_Step_Decks__Zanki_Neurology__Neuro_Misc/.git/objects/db/de378783376bef535b843dbb9b62c795f4084f.lock'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/chris/anki/ankidev/qt/aqt/progress.py", line 118, in handler
    func()
  File "/home/chris/anki/ankidev/qt/aqt/main.py", line 218, in on_window_init
    fn()
  File "/home/chris/anki/ankidev/qt/aqt/main.py", line 317, in setupProfile
    self.loadProfile()
  File "/home/chris/anki/ankidev/qt/aqt/main.py", line 496, in loadProfile
    gui_hooks.profile_did_open()
  File "/home/chris/anki/ankidev/out/qt/_aqt/hooks.py", line 4107, in __call__
    anki.hooks.runHook("profileLoaded")
  File "/home/chris/anki/ankidev/pylib/anki/hooks.py", line 34, in runHook
    func(*args)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/archiver_vendor.py", line 42, in snapshot_on_sync
    self.do_snapshot('CrowdAnki: Snapshot on sync')
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/archiver_vendor.py", line 50, in do_snapshot
    self.all_deck_archiver().archive(overrides=self.overrides(),
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/archiver.py", line 23, in archive
    self.deck_archiver_supplier(deck).archive(reason=reason)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/anki_deck_archiver.py", line 22, in archive
    repo.stage_all()
  File "/home/chris/.local/share/Anki2/addons21/1788670778/history/dulwich_repo.py", line 32, in stage_all
    self.dulwich_repo.stage(status.untracked + status.unstaged)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/repo.py", line 1374, in stage
    self.object_store.add_object(blob)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/object_store.py", line 940, in add_object
    with GitFile(path, "wb", mask=PACK_MODE) as f:
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/file.py", line 92, in GitFile
    return _GitFile(filename, mode, bufsize, mask)
  File "/home/chris/.local/share/Anki2/addons21/1788670778/dist/dulwich/file.py", line 154, in __init__
    raise FileLocked(filename, self._lockfilename) from exc
dulwich.file.FileLocked: ('/home/chris/.local/share/Anki2/addons21/1788670778/user_files/Chris/MyDeck__Medicine__AnKing__Zanki_Step_Decks__Zanki_Neurology__Neuro_Misc/.git/objects/db/de378783376bef535b843dbb9b62c795f4084f', '/home/chris/.local/share/Anki2/addons21/1788670778/user_files/Chris/MyDeck__Medicine__AnKing__Zanki_Step_Decks__Zanki_Neurology__Neuro_Misc/.git/objects/db/de378783376bef535b843dbb9b62c795f4084f.lock')

===Add-ons (active)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
BetterSearch ['1052724801', 2023-06-15T22:44, 'None', '']
Button Colours Good Again ['2494384865', 2022-09-27T12:18, 'None', '']
CrowdAnki JSON exportimport Edit history Collaborate on deck creation ['1788670778', 2023-10-30T19:57, 'None', mod]
Edit Field During Review Cloze ['385888438', 2023-11-01T05:53, '6.17', mod]
FSRS4Anki Helper ['759844606', 2023-12-29T16:38, 'None', mod]
Highlight Search Results in the Browser ['225180905', 2023-10-21T21:17, 'None', '']
Hint Hotkeys ['1844908621', 2023-04-14T16:13, 'None', '']
More Overview Stats 21 ['738807903', 2021-07-30T21:13, 'None', '']
Review Heatmap ['1771074083', 2022-06-30T03:43, 'None', '']

===IDs of active AnkiWeb add-ons===
1052724801 1771074083 1788670778 1844908621 225180905 2494384865 385888438 738807903 759844606

===Add-ons (inactive)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
AnkiHub ['1322529746', 2023-12-27T21:22, 'None', '']
aplaice commented 10 months ago

JSON exported file is over 100 Mb

I'll try to see if I can reproduce your issue with a large deck — if I can that will make debugging/testing smoother since I won't need to ask you to try things.

that git normally handles

Unfortunately, we don't directly use git, but instead use dulwich which has a ground-up pure-python implementation of git. (It has to be pure-python to be uploadable to AnkiWeb. We can't just "shell out" to git because most users won't have git installed.) If the issue turns out to be a not-easily-workaroundable performance issue in dulwich, then one solution might be to have an option (off-by-default) for CrowdAnki to directly call git (if the user has it installed). (Or possibly use libgit, but that's tricky because Anki doesn't access the "normal" python site packages.)

Would git lfs be a better option?

Possibly. Just allowing normal git to be called would be the first step, though. (To use git lfs we'd need to call the binary anyway, as dulwich doesn't fully support LFS (yet?).)

aplaice commented 10 months ago

To keep you informed, I've tested with heavy decks (3 GB deck.json) and with large-ish amounts of media (10000 (small-ish) randomly generated images), and I'm still unable to reproduce the issue where the lock file is generated.

When I have more time I'll test some more situations (e.g. on Wayland), but for the time being I have no idea. (My best guess is that on your system Anki is (for some reason) killed before the creation of the snapshot is complete, but I have no idea why.)

If/when I have more time I'll also see if adding the option to use git directly might be straightforward and if it might help you.

As an alternative way of keeping a history of your deck you might, perhaps, in the meantime want to try ki. (Not affiliated and haven't actually tried it, but the creator had reached out to me suggesting it for another project, and it looks cool — its main disadvantage is that it's a separate tool.)

chrislongros commented 10 months ago

Thank you very much for your quick response and your time !! :)