athrowaway2021 / comix

Seamlessly download and de-drm comics and manga from Kindle in highest possible quality
MIT License
61 stars 20 forks source link

KeyError: 'YJFragmentList get has multiple matches #56

Open vee-eye-en-kay opened 8 months ago

vee-eye-en-kay commented 8 months ago

This has worked mostly well with the books I have attempted, except for this one: B0BPX4YJ8V

Microsoft Windows [Version 10.0.22631.3085]
(c) Microsoft Corporation. All rights reserved.

D:\comix-main>python unkindle.py B0BPX4YJ8V
Authenticating . . .
Logged in with saved state
Authenticated!
Download size: 124MB
Downloading the book . . . 97%
Extracting . . .
Traceback (most recent call last):
  File "D:\comix-main\unkindle.py", line 188, in <module>
    kindle.download(args.output, args.keep_temp)
  File "D:\comix-main\unkindle.py", line 138, in download
    metadata = self.book.get_metadata()
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\comix-main\third_party\kfxlib\yj_book.py", line 156, in get_metadata
    if self.has_metadata() and self.has_cover_data():
       ^^^^^^^^^^^^^^^^^^^
  File "D:\comix-main\third_party\kfxlib\yj_metadata.py", line 246, in has_metadata
    self.fragments.get(YJFragmentKey(ftype="$258")) is not None)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\comix-main\third_party\kfxlib\yj_container.py", line 322, in get
    raise KeyError("YJFragmentList get has multiple matches for %s: %s" % (repr(key), list_symbols(matches)))
KeyError: 'YJFragmentList get has multiple matches for $258::: $258::, $258::'

D:\comix-main>

I have tried to delete the output folder it created, then retrying the process from the start, but continue to get the same error each time. It seems to always happen after the "download" gets to 97% and begins to extract, then error.

Other books before this one and after, have not received any errors on my end. My script knowledge is extremely limited so I don't know how to troubleshoot.

2023comicjtr commented 8 months ago

I received the same error (on a different book), and like you, the error always appeared after the downloading was complete and the extraction process started. I never found a solution, but I was able to successfully grab the book by using the less elegant method discussed in a different thread (use an Android emulator, import into Calibre, then use the terminal to extract). Obviously that doesn't solve this problem, but I was able to grab the book I was after!

Edit: oops... this thread: https://github.com/noDRM/DeDRM_tools/discussions/440

neckothy commented 8 months ago

I haven't yet encountered this myself. Assuming the above comment is true, I'm not sure why comix would be failing here when KFX Input doesn't. KFX Input's version history mentions addressing similar issues in v1.18.0 and v1.31.0.

In the meantime, if the problem is only with extraction, you should at least be able to avoid setting up an Android emulator. Use the --keep_temp arg mentioned in the README. After downloading and presumably failing extraction, you should have an output folder with the KFX files inside (e.g. comix/output/B0BPX4YJ8V). These can be imported to calibre after adding the device_id from state.json to DeDRM_tools.

SYY-b commented 7 months ago

For what it's worth, I ran into this issue on a few books now. Luckily, on failure, even without --keep_temp, the tool leaves the downloaded files in the temp folder. I was able to import the KFX into Calibre by adding just one of the KFX files in the temp folder.

For each case of the first issue, I hit a second issue using Calibre to convert the KFX due to noDRM/DeDRM_tools#1818. Maybe related? Edit: I realize now Calibre simply does not support converting fixed-layout KFX books directly, i.e. comics.

To work around the Calibre conversion limitations, I was able to use the KFX Input plugin CLI, as the conversion error message suggests.

These were the affected ASIN I have encountered so far:

Weirdly, books in the same series worked fine, so it seems kinda random. Hopefully this context is helpful.

whatveer commented 4 months ago

For what it's worth, I ran into this issue on a few books now. Luckily, on failure, even without --keep_temp, the tool leaves the downloaded files in the temp folder. I was able to import the KFX into Calibre by adding just one of the KFX files in the temp folder.

For each case of the first issue, I hit a second issue using Calibre to convert the KFX due to noDRM/DeDRM_tools#1818. ~Maybe related?~ Edit: I realize now Calibre simply does not support converting fixed-layout KFX books directly, i.e. comics.

To work around the Calibre conversion limitations, I was able to use the KFX Input plugin CLI, as the conversion error message suggests.

These were the affected ASIN I have encountered so far:

  • B01GT6BLX8
  • B00F3HK6TA
  • B071NX66W6

Weirdly, books in the same series worked fine, so it seems kinda random. Hopefully this context is helpful.

can you explain how you did this? I am having the same issue now

SYY-b commented 4 months ago

can you explain how you did this? I am having the same issue now

If you have Calibre and the KFX Input plug-in installed on your computer, then you should be able to use the command line to decrypt the book into any supported format of your choosing.

Take a look at the Command Line Interface section here.

As a tip, you only need to specify any one single kfx, azw, etc. file from the folder that contains the files, and it will properly detect the related files and decrypt them into a single book.