Books are imported as KFX-ZIP and DRM is not removed

[steakhutzeee]

Hi, i'm actually on Windows 10 x64 with latest version of Calibre and DeDrm plugin.

I have installed Kindle application for PC version 1.26 from the link found in the wiki page.

Also latest version of KFX Input and Output installed.

Anyway when i import the books in Calibre they are in KFX-ZIP format and DRM is not removed.

Could you help me understand why this happens?

As a sidenote, if i download the books directly from my Amazon contents the books are imported as AZW3 (as espected in this case, correct me if i'm wrong but downloading the books this way can only output an AZW3 file) and the DRM is correctly removed.

[ElleKayEm]

Not sure if KFX decryption is working yet with pre-release DeDRM 7.0.0b4. Were the books downloaded with the current installation of Kindle for PC on the same machine and user as calibre with DeDRM?

[steakhutzeee]

Yes, everything on the same PC.

[ElleKayEm]

Let's see a log: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md#i-cannot-solve-my-problem-with-the-dedrm-plugin-and-now-i-need-to-post-a-log-how-do-i-do-that

[steakhutzeee]

Sure, here is the log:

calibre Debug log
calibre 5.6 [64bit]  embedded-python: True is64bit: True
Windows-10-10.0.18362 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.18362')
Python 3.8.5
Windows: ('10', '10.0.18362', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 36, 0) && DeDRM (7, 0, 0) && Package KFX (from KFX Input) (1, 36, 0) && Count Pages (1, 11, 0) && Goodreads Sync (1, 14, 3) && Hyphenate This! (0, 1, 3) && KFX metadata reader (from KFX Input) (1, 36, 0) && KFX Input (1, 36, 0) && Set KFX metadata (from KFX Output) (1, 46, 0) && KFX Output (1, 46, 0) && Quality Check (1, 10, 1)
calibre 5.6 [64bit]  embedded-python: True is64bit: True
Windows-10-10.0.18362 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.18362')
Python 3.8.5
Windows: ('10', '10.0.18362', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 36, 0) && DeDRM (7, 0, 0) && Package KFX (from KFX Input) (1, 36, 0) && Count Pages (1, 11, 0) && Goodreads Sync (1, 14, 3) && Hyphenate This! (0, 1, 3) && KFX metadata reader (from KFX Input) (1, 36, 0) && KFX Input (1, 36, 0) && Set KFX metadata (from KFX Output) (1, 46, 0) && KFX Output (1, 46, 0) && Quality Check (1, 10, 1)
Turning off automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 144.0 x 144.0
physicalDpi: 157.8252427184466 x 158.5664739884393
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[1.66] splash screen shown
[1.66] Initializing db...
[1.66] db initialized
[1.66] Constructing main UI...
DEBUG:    0.0 Goodreads Sync::dialogs.py - loading translations
DEBUG:    0.0 GoodreadsSync::action.py - loading translations
DEBUG:    0.9 HttpHelper::__init__: proxy=None
[2.77] main UI initialized...
[2.77] Hiding splash screen
Starting QuickView
Gather KFX-ZIP (from KFX Input) 1.36.0: Importing C:\Users\xxx\Documents\My Kindle Content\B00GNZNEJK_EBOK\B00GNZNEJK_EBOK.azw
Gather KFX-ZIP (from KFX Input): Gathered 4 file(s) as C:\Users\xxx\AppData\Local\Temp\calibre_u1wu3nj0\zkg7lwam.kfx-zip
DeDRM v7.0.0: Trying to decrypt zkg7lwam.kfx-zip
Using Library AlfCrypto DLL/DYLIB/SO
Using Library AlfCrypto DLL/DYLIB/SO
Decrypting KFX-ZIP ebook: zkg7lwam
Got DSN key from database default_key
Error getting PIDs from serial number G000PP12915206X3: string argument without an encoding
Traceback (most recent call last):
  File "calibre_plugins.dedrm.kgenpids", line 306, in getPidList
TypeError: string argument without an encoding
Found 1 keys to try after 0.2 seconds
Decrypting KFX DRM voucher: amzn1.drm-voucher.v1.f02607c1-fff2-4505-abee-51491d1b7653.voucher
DeDRM v7.0.0: Failed to decrypt with error: Failed to decrypt KFX DRM voucher with any key
DeDRM v7.0.0: Looking for new default Kindle Key after 0.2 seconds
searching for kinfoFiles in C:\Users\xxx\AppData\Local
Found K4PC 1.25+ kinf2018 file: C:\Users\xxx\AppData\Local\Amazon\Kindle\storage\.kinf2018
[141.09] splash screen hidden
[141.09] Started up in 141.09 seconds with 12 books
stdout+stderr from file dialog helper: [b'', b'']
piped data from file dialog helper: [b'\xcek\x19@\x15E`Qy\x03\xcemC\x85,\xb9=\xe7h\x8b\xa1\x1b\n\xc8s\x912\x8f\xb4.\xd8\xfc', b'C:\\Users\\xxx\\Documents\\My Kindle Content\\B00GNZNEJK_EBOK\\B00GNZNEJK_EBOK.azw']
Decrypted key file using IDString '1054301041' and UserName 'xxx'
DeDRM v7.0.0: Found 1 new key
Decrypting KFX-ZIP ebook: zkg7lwam
Got DSN key from database default_key_1
Found 1 keys to try after 4.9 seconds
Decrypting KFX DRM voucher: amzn1.drm-voucher.v1.f02607c1-fff2-4505-abee-51491d1b7653.voucher
DeDRM v7.0.0: Ultimately failed to decrypt after 4.9 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Package KFX (from KFX Input) 1.36.0: Packaging C:\Users\xxx\AppData\Local\Temp\calibre_u1wu3nj0\zkg7lwam.kfx-zip
Processing container: B00GNZNEJK_EBOK.azw
KFX metadata reader (from KFX Input) activated for C:\Users\xxx\AppData\Local\Temp\calibre_u1wu3nj0\ip03bl4__add_books\0\B00GNZNEJK_EBOK.kfx-zip
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 534, in KindleMobiDecrypt
  File "calibre_plugins.dedrm.k4mobidedrm", line 238, in GetDecryptedBook
  File "calibre_plugins.dedrm.kfxdedrm", line 42, in processBook
  File "calibre_plugins.dedrm.kfxdedrm", line 82, in decrypt_voucher
Exception: Failed to decrypt KFX DRM voucher with any key

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre\customize\ui.py", line 172, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 627, in run
  File "calibre_plugins.dedrm.__init__", line 577, in KindleMobiDecrypt
calibre_plugins.dedrm.DeDRMError: DeDRM v7.0.0: Ultimately failed to decrypt after 4.9 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Running file type plugin Package KFX (from KFX Input) failed with traceback:
Traceback (most recent call last):
  File "calibre\customize\ui.py", line 172, in _run_filetype_plugins
  File "calibre_plugins.kfx_input.package_filetype", line 21, in run
  File "calibre_plugins.kfx_input.package_filetype", line 32, in run_kfx
  File "calibre_plugins.kfx_input.kfxlib.yj_book", line 83, in convert_to_single_kfx
  File "calibre_plugins.kfx_input.kfxlib.yj_book", line 208, in decode_book
  File "calibre_plugins.kfx_input.kfxlib.yj_book", line 315, in get_container
calibre_plugins.kfx_input.kfxlib.utilities.KFXDRMError: Book container B00GNZNEJK_EBOK.azw has DRM andAdded Il giro di vite (Italian Edition) to db in: 0.2
Added 1 books in 6.0 seconds

[ElleKayEm]

It seems to be able to decrypt the key, but still can't decrypt the book with that key. Have you reinstalled Kindle for PC or had to login in again since downloading this copy of the book? If so, try downloading a fresh copy of the book. If not, then I'd say DeDRM 7 still needs work to handle KFX format. You could go back to DeDRM 6.8 and calibre 4.23. Or wait to try the next DeDRM release.

Alternatively, of course, you can stick with the azw3 format. Either from the website for "download & transfer via usb" for an E-ink Kindle or by disabling KFX downloads in the Kindle app.

[steakhutzeee]

Yes i tried to download a fresh copy but the issue persists.

No issue with Calibre 4.23

So actually importing a KFX book in calibre this way is like importing the exact book i am reading on my Kindle without any modification?

[ismail]

This is just a Python3 problem

Error getting PIDs from serial number G000PP12915206X3: string argument without an encoding
Traceback (most recent call last):
  File "calibre_plugins.dedrm.kgenpids", line 306, in getPidList
TypeError: string argument without an encoding

Python3 port is still in progress, so this is expected. There is no ETA for a stable release for Python3 port.

[steakhutzeee]

This is just a Python3 problem

Error getting PIDs from serial number G000PP12915206X3: string argument without an encoding
Traceback (most recent call last):
  File "calibre_plugins.dedrm.kgenpids", line 306, in getPidList
TypeError: string argument without an encoding

Python3 port is still in progress, so this is expected. There is no ETA for a stable release for Python3 port.

So the whole KFX conversion not working is due to this issue? Or is valid only for the part of the log you mentioned? (the serial of my Kindle to DeDrm files imported after directly downloading from my Amazon content).

[ismail]

This is just a Python3 problem

Error getting PIDs from serial number G000PP12915206X3: string argument without an encoding
Traceback (most recent call last):
  File "calibre_plugins.dedrm.kgenpids", line 306, in getPidList
TypeError: string argument without an encoding

Python3 port is still in progress, so this is expected. There is no ETA for a stable release for Python3 port.

So the whole KFX conversion not working is due to this issue? Or is valid only for the part of the log you mentioned? (the serial of my Kindle to DeDrm files imported after directly downloading from my Amazon content).

This is one issue, there might be more issues in the code path. We had another bug fixed in this code path and now we have another error. I'm hopeful there won't be many more.

[steakhutzeee]

This is one issue, there might be more issues in the code path. We had another bug fixed in this code path and now we have another error. I'm hopeful there won't be many more.

Maybe i could install also Python 2 on the system to circumvent this issue?

[ismail]

This is one issue, there might be more issues in the code path. We had another bug fixed in this code path and now we have another error. I'm hopeful there won't be many more.

Maybe i could install also Python 2 on the system to circumvent this issue?

You should switch to Calibre 4.x and use the DeDRM_Tools 6.x version. That should work.

[ElleKayEm]

DeDRM 6.8 and calibre 4 use Python 2. Pre-release DeDRM 7 is an in progress re-write to Python 3 which is what calibre 5 requires.

From what I understand about KFX format, the version downloaded will have formatting and any images optimized for that app or device.

[srlevine1]

I have the same problem -- the input files are recognized (azw; mbpv2; azw.md; azw.res; .voucher) -- when processed, fail to decrypt properly, and then a kfx-zip file is placed in the library. And when it is opened, throws the drm error dialog.

Since I have 5.12 and 7.10, I think the error is in how the input files are processed.

[ElleKayEm]

@srlevine1 To decrypt KFX format, be sure you're using Kindle for PC/Mac 1.26 or earlier. See this post: https://www.mobileread.com/forums/showthread.php?t=283371

[michaelh99]

fwiw, same issue here. Have latest calibre, de-drm and kindle 1.20 and have a new book that's not being decrypted

[ElleKayEm]

@michaelh99 Have you tried disabling KFX format downloads? See Method 2 at this link: https://www.mobileread.com/forums/showthread.php?t=283371

If you need further assistance, post a debug log of adding a book.

[michaelh99]

@michaelh99 Have you tried disabling KFX format downloads? See Method 2 at this link: https://www.mobileread.com/forums/showthread.php?t=283371

If you need further assistance, post a debug log of adding a book.

Grr. I've known about that workaround for a long time and for whatever reason hadn't made it the last time I installed the kindle app. Thank you for the reminder