search

noDRM / DeDRM_tools

DeDRM tools for ebooks
7.24k stars 323 forks source link

Cryptographic/key problems with Amazon-downloaded books and latest tools/Calibre on Windows? #291

Open ksbrar opened 1 year ago

ksbrar commented 1 year ago

Question / bug report

I downloaded a book directly from Amazon and tried to import it to Calibre. It was published in 2017 and received in the AZW3 format - so it should supported by my understanding.

I tried importing it a couple of times into Calibre and ran into the two following problems: 1) it looks as though there is an import error for certain python cryptographic modules, 2) DeDRM may not be trying the correct keys. I've posted the log I got back from Calibre below and it more directly relates to point (2). After getting a bit frustrated I tried to go back and play with the plugin settings and when I tried to customize it I got this interesting error:

calibre, version 6.13.0
ERROR: Unhandled exception: <b>OSError</b>:Cannot load native module 'Crypto.Cipher._raw_des': Not found '_raw_des.cp310-win_amd64.pyd', Not found '_raw_des.pyd'

calibre 6.13  embedded-python: True
Windows-10-10.0.22623-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22623')
Python 3.10.1
Windows: ('10', '10.0.22623', 'SP0', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 51, 0) && DeDRM (10, 0, 3) && Package KFX (from KFX Input) (1, 51, 0) && BookFusion Plugin (0, 8, 0) && EpubMerge (2, 15, 0) && Find Duplicates (1, 10, 7) && Generate Cover (2, 3, 2) && GetFileName (0, 2, 0) && Goodreads (1, 7, 3) && Goodreads Sync (1, 16, 2) && Job Spy (1, 0, 205) && KFX metadata reader (from KFX Input) (1, 51, 0) && KFX Input (1, 51, 0) && KePub Input (3, 6, 3) && KePub Metadata Reader (3, 4, 3) && KePub Metadata Writer (3, 4, 3) && KePub Output (3, 6, 3) && Kindle hi-res covers (0, 5, 0) && KoboTouchExtended (3, 6, 3) && Library Genesis (1, 2, 0) && LibraryThing Match (0, 2, 1) && Manage Series (1, 5, 0) && Obok DeDRM (10, 0, 3) && Prettify Cover (1, 6, 0) && Resize Cover (1, 2, 0) && Standard Ebooks (1, 0, 0) && The Zlibrary Library (0, 0, 10)
Traceback (most recent call last):
  File "calibre\gui2\preferences\plugins.py", line 301, in customize_plugin
  File "calibre\gui2\preferences\plugins.py", line 379, in modify_plugin
  File "calibre\customize\__init__.py", line 156, in do_user_config
  File "calibre_plugins.dedrm.__init__", line 1079, in config_widget
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\config.py", line 54, in <module>
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\androidkindlekey.py", line 38, in <module>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "bypy-importer.py", line 279, in exec_module
  File "Crypto\Cipher\DES.py", line 42, in <module>
  File "Crypto\Util\_raw_api.py", line 309, in load_pycryptodome_raw_lib
OSError: Cannot load native module 'Crypto.Cipher._raw_des': Not found '_raw_des.cp310-win_amd64.pyd', Not found '_raw_des.pyd'

Another user ran into a similar problem, though it was under Linux and I followed his steps (installing Pycrypto and Crypto again) to no avail (same error at the end).

Finally, I started from scratch and instead downloaded v1.26 of Kindle for PC (and launched it using the no KFX script) and was able to get a file that could be decrypted properly by DeDRM. But I'm surprised this method worked over downloading it straight from Amazon. What is also strange is that I was able to import an Amazon-downloaded book a few weeks ago, and the only thing I can think of that has changed since then is that I upgraded to Calibre 6.13 from 6.12.

Which version of Calibre are you running?

6.13

Which version of the DeDRM plugin are you running?

v10.0.3

If applicable, which version of the Kindle software are you running?

1.26

Log output

ApplicationPaletteChange event ignored
Using calibre Qt style: True
calibre_plugins.librarything_match.fuzzywuzzy.fuzz:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
Traceback (most recent call last):
  File "calibre\gui2\ui.py", line 179, in load_store_plugins
  File "calibre\gui2\ui.py", line 191, in init_istore
  File "calibre\customize\__init__.py", line 709, in load_actual_plugin
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "calibre\customize\zipplugin.py", line 206, in exec_module
  File "calibre_plugins.store_zlibrary.zlibrary_plugin", line 18, in <module>
ModuleNotFoundError: No module named 'urllib3'
EpubMerge: DEBUG: 2023-02-23 01:09:41,146: calibre_plugins.epubmerge.epubmerge_plugin(154): macmenuhack file_path:C:\Users\ksbrar\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
DEBUG: calibre_plugins.epubmerge.epubmerge_plugin - macmenuhack file_path:C:\Users\ksbrar\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
calibre Debug log
calibre 6.13  embedded-python: True
Windows-10-10.0.22623-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22623')
Python 3.10.1
Windows: ('10', '10.0.22623', 'SP0', 'Multiprocessor Free')
Interface language: None
DEBUG:    0.0 GetFileName::__init__.py - loading translations
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 51, 0) && DeDRM (10, 0, 3) && Package KFX (from KFX Input) (1, 51, 0) && BookFusion Plugin (0, 8, 0) && EpubMerge (2, 15, 0) && Find Duplicates (1, 10, 7) && Generate Cover (2, 3, 2) && GetFileName (0, 2, 0) && Goodreads (1, 7, 3) && Goodreads Sync (1, 16, 2) && Job Spy (1, 0, 205) && KFX metadata reader (from KFX Input) (1, 51, 0) && KFX Input (1, 51, 0) && KePub Input (3, 6, 3) && KePub Metadata Reader (3, 4, 3) && KePub Metadata Writer (3, 4, 3) && KePub Output (3, 6, 3) && Kindle hi-res covers (0, 5, 0) && KoboTouchExtended (3, 6, 3) && Library Genesis (1, 2, 0) && LibraryThing Match (0, 2, 1) && Manage Series (1, 5, 0) && Obok DeDRM (10, 0, 3) && Prettify Cover (1, 6, 0) && Resize Cover (1, 2, 0) && Standard Ebooks (1, 0, 0) && The Zlibrary Library (0, 0, 10)
calibre 6.13  embedded-python: True
Windows-10-10.0.22623-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22623')
Python 3.10.1
Windows: ('10', '10.0.22623', 'SP0', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 51, 0) && DeDRM (10, 0, 3) && Package KFX (from KFX Input) (1, 51, 0) && BookFusion Plugin (0, 8, 0) && EpubMerge (2, 15, 0) && Find Duplicates (1, 10, 7) && Generate Cover (2, 3, 2) && GetFileName (0, 2, 0) && Goodreads (1, 7, 3) && Goodreads Sync (1, 16, 2) && Job Spy (1, 0, 205) && KFX metadata reader (from KFX Input) (1, 51, 0) && KFX Input (1, 51, 0) && KePub Input (3, 6, 3) && KePub Metadata Reader (3, 4, 3) && KePub Metadata Writer (3, 4, 3) && KePub Output (3, 6, 3) && Kindle hi-res covers (0, 5, 0) && KoboTouchExtended (3, 6, 3) && Library Genesis (1, 2, 0) && LibraryThing Match (0, 2, 1) && Manage Series (1, 5, 0) && Obok DeDRM (10, 0, 3) && Prettify Cover (1, 6, 0) && Resize Cover (1, 2, 0) && Standard Ebooks (1, 0, 0) && The Zlibrary Library (0, 0, 10)
QPA platform: windows
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 108.91792294807368 x 108.85714285714285
[0.00] Starting up...
[0.00] Initializing db...
[0.08] db initialized
[0.08] Constructing main UI...
DEBUG:    0.0 obok::utilities.py - loading translations
DEBUG:    0.0 obok::dialogs.py - loading translations
DEBUG:    0.0 obok::config.py - loading translations
DEBUG:    0.0 obok::action_err.py - loading translations
prettify cover: registry parse error
Screens currently in system:
{'depth': 32,
 'device_pixel_ratio': 1.0,
 'geometry_in_logical_pixels': {'height': 1440, 'width': 2560, 'x': 0, 'y': 0},
 'index_in_screens_list': 0,
 'manufacturer': '',
 'model': '',
 'name': 'LC27G7xT',
 'serial': '',
 'size_in_logical_pixels': {'height': 1440, 'width': 2560},
 'virtual_geometry': {'height': 1440, 'width': 2560, 'x': 0, 'y': 0}}
Restoring geometry for: jobs_dialog_geometry
{'frame_geometry': {'height': 573, 'width': 836, 'x': 188, 'y': 83},
 'full_screened': False,
 'geometry': {'height': 542, 'width': 836, 'x': 188, 'y': 114},
 'maximized': False,
 'normal_geometry': {'height': 542, 'width': 836, 'x': 188, 'y': 114},
 'screen': {'depth': 32,
            'device_pixel_ratio': 1.0,
            'geometry_in_logical_pixels': {'height': 1440,
                                           'width': 2560,
                                           'x': 0,
                                           'y': 0},
            'index_in_screens_list': 0,
            'manufacturer': '',
            'model': '',
            'name': 'LC27G7xT',
            'serial': '',
            'size_in_logical_pixels': {'height': 1440, 'width': 2560},
            'virtual_geometry': {'height': 1440,
                                 'width': 2560,
                                 'x': 0,
                                 'y': 0}}}
Matching screen: {'depth': 32,
 'device_pixel_ratio': 1.0,
 'geometry_in_logical_pixels': {'height': 1440, 'width': 2560, 'x': 0, 'y': 0},
 'index_in_screens_list': 0,
 'manufacturer': '',
 'model': '',
 'name': 'LC27G7xT',
 'serial': '',
 'size_in_logical_pixels': {'height': 1440, 'width': 2560},
 'virtual_geometry': {'height': 1440, 'width': 2560, 'x': 0, 'y': 0}}
Setting widget geometry to: {'height': 542, 'width': 836, 'x': 188, 'y': 114}
Restoring geometry for: calibre_main_window_geometry
{'frame_geometry': {'height': 960, 'width': 1702, 'x': 164, 'y': 212},
 'full_screened': False,
 'geometry': {'height': 929, 'width': 1702, 'x': 164, 'y': 243},
 'maximized': False,
 'normal_geometry': {'height': 929, 'width': 1702, 'x': 164, 'y': 243},
 'screen': {'depth': 32,
            'device_pixel_ratio': 1.0,
            'geometry_in_logical_pixels': {'height': 1440,
                                           'width': 2560,
                                           'x': 0,
                                           'y': 0},
            'index_in_screens_list': 0,
            'manufacturer': '',
            'model': '',
            'name': 'LC27G7xT',
            'serial': '',
            'size_in_logical_pixels': {'height': 1440, 'width': 2560},
            'virtual_geometry': {'height': 1440,
                                 'width': 2560,
                                 'x': 0,
                                 'y': 0}}}
Matching screen: {'depth': 32,
 'device_pixel_ratio': 1.0,
 'geometry_in_logical_pixels': {'height': 1440, 'width': 2560, 'x': 0, 'y': 0},
 'index_in_screens_list': 0,
 'manufacturer': '',
 'model': '',
 'name': 'LC27G7xT',
 'serial': '',
 'size_in_logical_pixels': {'height': 1440, 'width': 2560},
 'virtual_geometry': {'height': 1440, 'width': 2560, 'x': 0, 'y': 0}}
Setting widget geometry to: {'height': 929, 'width': 1702, 'x': 164, 'y': 243}
Job Spy has begun initialization...
Calibre, and hence Job Spy, was gracefully shut down last time?  True
Last time daemon started:  never
Last time daemon failed:  never
Total daemon starts inception_to_date:  0
Total daemon failures inception-to-date:  0
JS: ===>>> change_gui_alternating_row_colors()
Job Spy has finished initialization...
[1.25] main UI initialized...
[1.25] Started up in 1.25 seconds with 1424 books
Starting QuickView
DeDRM v10.0.3: Trying to decrypt <redacted by me>
Using Library AlfCrypto DLL/DYLIB/SO
MobiDeDrm v1.1.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: <redacted by me>
Got DSN key from database default_key
Found 4 keys to try after 0.1 seconds
Crypto Type is: 2
DeDRM v10.0.3: Failed to decrypt with error: No key found in 4 PIDs tried.
DeDRM v10.0.3: Looking for new default Kindle Key after 0.1 seconds
searching for kinfoFiles in C:\Users\ksbrar\AppData\Local
Found K4PC 1.25+ kinf2018 file: C:\Users\ksbrar\AppData\Local\Amazon\Kindle\storage\.kinf2018
Decrypted key file using IDString '2689983668' and UserName 'ksbrar'
DeDRM v10.0.3: Found 1 new key
MobiDeDrm v1.1.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: <redacted by me>
Got DSN key from database key_0
Found 4 keys to try after 1.8 seconds
Crypto Type is: 2
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 4 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 991, in KindleMobiDecrypt
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 4 PIDs tried.
DeDRM v10.0.3: Ultimately failed to decrypt after 1.8 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 4 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre\customize\ui.py", line 178, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 1056, in run
  File "calibre_plugins.dedrm.__init__", line 1007, in KindleMobiDecrypt
calibre_plugins.dedrm.DeDRMError: DeDRM v10.0.3: Ultimately failed to decrypt after 1.8 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md
GetFileType::run -  C:/Users/ksbrar/Downloads/<redacted by me>
GetFileType::run - Fich name:  C:\Users\ksbrar\AppData\Local\Temp\calibre_egk7bstw\56ywc0chGetFileName\Dict.txt
GetFileType::run - Have file
GetFileType::run - error opening file: Expecting value: line 1 column 1 (char 0)
GetFileType::run-fich
Failed to get text from book_id: 1899 format: AZW3
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 4 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 991, in KindleMobiDecrypt
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 4 PIDs tried.
Running file type plugin DeDRM failed with traceback:

Traceback (most recent call last):

  File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt

  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook

  File "C:\Users\ksbrar\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook

mobidedrm.DrmException: No key found in 4 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "calibre\customize\ui.py", line 178, in _run_filetype_plugins

  File "calibre_plugins.dedrm.__init__", line 1056, in run

  File "calibre_plugins.dedrm.__init__", line 1007, in KindleMobiDecrypt

calibre_plugins.dedrm.DeDRMError: DeDRM v10.0.3: Ultimately failed to decrypt after 1.8 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md

Traceback (most recent call last):

  File "runpy.py", line 196, in _run_module_as_main

  File "runpy.py", line 86, in _run_code

  File "site.py", line 83, in <module>

  File "site.py", line 78, in main

  File "site.py", line 50, in run_entry_point

  File "calibre\utils\ipc\worker.py", line 196, in main

  File "<string>", line 1, in <module>

  File "calibre\db\fts\text.py", line 103, in main

  File "calibre\db\fts\text.py", line 90, in extract_text

  File "calibre\ebooks\oeb\iterator\book.py", line 62, in extract_book

  File "calibre\customize\conversion.py", line 242, in __call__

  File "calibre\ebooks\conversion\plugins\mobi_input.py", line 44, in convert

  File "calibre\ebooks\mobi\reader\mobi8.py", line 85, in __call__

  File "calibre\ebooks\mobi\reader\mobi6.py", line 162, in check_for_drm

calibre.ebooks.DRMError: <redacted by me>
ElleKayEm commented 1 year ago

Did you choose an E-ink Kindle? Do you have it's serial number entered correctly in DeDRM customization?

ksbrar commented 1 year ago

@ElleKayEm Thanks for your help. Yup, I chose an e-ink Kindle. I am still unable to customize the DeDRM plugin due to the Cannot load native module 'Crypto.Cipher._raw_des': Not found '_raw_des.cp310-win_amd64.pyd', Not found '_raw_des.pyd' error

ElleKayEm commented 1 year ago

That's weird. You're not able to customize the E-ink Kindle part? The error you posted at the beginning looks like it's coming up with the Kindle for Android key script.

As far as I know there shouldn't be any need to have Python or Pycryptodome installed with Windows calibre. Linux needs them installed under Wine in order to retrieve keys from programs installed there, but Windows calibre should just use calibre's embedded Python.

I'm using DeDRM 10.0.3 with calibre 6.13 on Windows 10 and not having an issue customizing or with decrypting azw3 from download & transfer.

ksbrar commented 1 year ago

That's weird. You're not able to customize the E-ink Kindle part?

Nope, this is the error that stops me: calibre_NlREfR5Rkl

Windows calibre should just use calibre's embedded Python.

Right, which is what's got me confused.

I'm using DeDRM 10.0.3 with calibre 6.13 on Windows 10 and not having an issue customizing or with decrypting azw3 from download & transfer.

Hmm, only difference is I'm on W11.

ElleKayEm commented 1 year ago

Huh, it seems like I would have seen more issues like this if Windows 11 was causing the problem. But that might be it, and I don't know how to resolve that. Hopefully @noDRM can look into it.