search

apprenticeharper / DeDRM_tools

DeDRM tools for ebooks
14.54k stars 1.52k forks source link

[QUESTION] Amazon website downloaded Kindle azw file fail to convert #2182

Open maxloosmu opened 1 year ago

maxloosmu commented 1 year ago

CheckList

Amazon Kindle azw file fail to convert

My Environment

Calibre: Version 6.10

Kindle: Version 1.39.2 (65383)

DeDRM: Version DeDRM_tools_10.0.3

Log

Log Title ```log ApplicationPaletteChange event ignored Using calibre Qt style: True ApplicationPaletteChange event received calibre Debug log calibre 6.10 embedded-python: True Windows-10-10.0.19044-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.19044') Python 3.10.1 Windows: ('10', '10.0.19044', 'SP0', 'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 50, 0) && DeDRM (10, 0, 3) && Package KFX (from KFX Input) (1, 50, 0) && KFX metadata reader (from KFX Input) (1, 50, 0) && KFX Input (1, 50, 0) calibre 6.10 embedded-python: True Windows-10-10.0.19044-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.19044') Python 3.10.1 Windows: ('10', '10.0.19044', 'SP0', 'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 50, 0) && DeDRM (10, 0, 3) && Package KFX (from KFX Input) (1, 50, 0) && KFX metadata reader (from KFX Input) (1, 50, 0) && KFX Input (1, 50, 0) QPA platform: windows devicePixelRatio: 1.25 logicalDpi: 96.0 x 96.0 physicalDpi: 74.88368522072936 x 74.89965870307167 [0.00] Starting up... [0.00] Showing splash screen... [0.17] splash screen shown [0.17] Initializing db... [0.17] db initialized [0.17] Constructing main UI... Screens currently in system: {'depth': 32, 'device_pixel_ratio': 1.25, 'geometry_in_logical_pixels': {'height': 864, 'width': 1536, 'x': 0, 'y': 0}, 'index_in_screens_list': 0, 'manufacturer': '', 'model': '', 'name': '\\\\.\\DISPLAY1', 'serial': '', 'size_in_logical_pixels': {'height': 864, 'width': 1536}, 'virtual_geometry': {'height': 864, 'width': 1536, 'x': 0, 'y': 0}} Restoring geometry for: calibre_main_window_geometry {'frame_geometry': {'height': 644, 'width': 849, 'x': 442, 'y': 55}, 'full_screened': False, 'geometry': {'height': 614, 'width': 849, 'x': 442, 'y': 85}, 'maximized': False, 'normal_geometry': {'height': 614, 'width': 849, 'x': 442, 'y': 85}, 'screen': {'depth': 32, 'device_pixel_ratio': 1.25, 'geometry_in_logical_pixels': {'height': 864, 'width': 1536, 'x': 0, 'y': 0}, 'index_in_screens_list': 0, 'manufacturer': '', 'model': '', 'name': '\\\\.\\DISPLAY1', 'serial': '', 'size_in_logical_pixels': {'height': 864, 'width': 1536}, 'virtual_geometry': {'height': 864, 'width': 1536, 'x': 0, 'y': 0}}} Matching screen: {'depth': 32, 'device_pixel_ratio': 1.25, 'geometry_in_logical_pixels': {'height': 864, 'width': 1536, 'x': 0, 'y': 0}, 'index_in_screens_list': 0, 'manufacturer': '', 'model': '', 'name': '\\\\.\\DISPLAY1', 'serial': '', 'size_in_logical_pixels': {'height': 864, 'width': 1536}, 'virtual_geometry': {'height': 864, 'width': 1536, 'x': 0, 'y': 0}} Setting widget geometry to: {'height': 614, 'width': 849, 'x': 442, 'y': 85} [1.58] main UI initialized... [1.58] Hiding splash screen Starting QuickView Gather KFX-ZIP (from KFX Input) 1.50.0: Importing C:/Users/Max/Documents/My Kindle Content/B09YR9YT5Q_EBOK/B09YR9YT5Q_EBOK.azw Gather KFX-ZIP (from KFX Input): Gathered 4 file(s) as C:\Users\Max\AppData\Local\Temp\calibre_trp_fezf\p7h9u1df.kfx-zip DeDRM v10.0.3: Trying to decrypt p7h9u1df.kfx-zip Using Library AlfCrypto DLL/DYLIB/SO [3.86] splash screen hidden [3.86] Started up in 3.86 seconds with 0 books Decrypting KFX-ZIP ebook: p7h9u1df Found 0 keys to try after 0.2 seconds Decrypting KFX DRM voucher: amzn1.drm-voucher.v1.b57097b5-e708-489a-a970-641989ab0398.voucher Traceback (most recent call last): File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 105, in decrypt_voucher File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 886, in decryptvoucher File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 762, in pkcs7unpad File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 102, in _assert Exception: Incorrect padding - Wrong key DeDRM v10.0.3: Failed to decrypt with error: Failed to decrypt KFX DRM voucher with any key DeDRM v10.0.3: Looking for new default Kindle Key after 0.2 seconds searching for kinfoFiles in C:\Users\Max\AppData\Local Found K4PC 1.25+ kinf2018 file: C:\Users\Max\AppData\Local\Amazon\Kindle\storage\.kinf2018 Decrypted key file using IDString '504818401' and UserName 'Max' DeDRM v10.0.3: Found 1 new key Decrypting KFX-ZIP ebook: p7h9u1df Got DSN key from database key_0 Found 1 keys to try after 0.6 seconds Decrypting KFX DRM voucher: amzn1.drm-voucher.v1.b57097b5-e708-489a-a970-641989ab0398.voucher Traceback (most recent call last): File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 67, in processBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 111, in decrypt_voucher Exception: Failed to decrypt KFX DRM voucher with any key During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 105, in decrypt_voucher File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 886, in decryptvoucher File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 762, in pkcs7unpad File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 102, in _assert Exception: Incorrect padding - Wrong key Traceback (most recent call last): File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 67, in processBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 111, in decrypt_voucher Exception: Failed to decrypt KFX DRM voucher with any key During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 105, in decrypt_voucher File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 891, in decryptvoucher File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\ion.py", line 102, in _assert Exception: Expected KeySet, got Traceback (most recent call last): File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 67, in processBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 111, in decrypt_voucher Exception: Failed to decrypt KFX DRM voucher with any key During handling of the above exception, another exception occurred: Traceback (most recent call last): File "calibre_plugins.dedrm.__init__", line 991, in KindleMobiDecrypt File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 67, in processBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 111, in decrypt_voucher Exception: Failed to decrypt KFX DRM voucher with any key DeDRM v10.0.3: Ultimately failed to decrypt after 0.6 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md Running file type plugin DeDRM failed with traceback: Traceback (most recent call last): File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 67, in processBook File "C:\Users\Max\AppData\Roaming\calibre\plugins\DeDRM.zip\kfxdedrm.py", line 111, in decrypt_voucher Exception: Failed to decrypt KFX DRM voucher with any key During handling of the above exception, another exception occurred: Traceback (most recent call last): File "calibre\customize\ui.py", line 178, in _run_filetype_plugins File "calibre_plugins.dedrm.__init__", line 1056, in run File "calibre_plugins.dedrm.__init__", line 1007, in KindleMobiDecrypt calibre_plugins.dedrm.DeDRMError: DeDRM v10.0.3: Ultimately failed to decrypt after 0.6 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md Package KFX (from KFX Input) 1.50.0: Packaging C:\Users\Max\AppData\Local\Temp\calibre_trp_fezf\p7h9u1df.kfx-zip Processing container: B09YR9YT5Q_EBOK.azw Running file type plugin Package KFX (from KFX Input) failed with traceback: Traceback (most recent call last): File "calibre\customize\ui.py", line 178, in _run_filetype_plugins File "calibre_plugins.kfx_input.package_filetype", line 21, in run File "calibre_plugins.kfx_input.package_filetype", line 32, in run_kfx File "calibre_plugins.kfx_input.kfxlib.yj_book", line 83, in convert_to_single_kfx File "calibre_plugins.kfx_input.kfxlib.yj_book", line 210, in decode_book File "calibre_plugins.kfx_input.kfxlib.yj_book", line 317, in get_container calibre_plugins.kfx_input.kfxlib.utilities.KFXDRMError: Book container B09YR9YT5Q_EBOK.azw has DRM and cannot be converted KFX metadata reader (from KFX Input) activated for C:\Users\Max\AppData\Local\Temp\calibre_trp_fezf\vvynm28q_add_books\0\B09YR9YT5Q_EBOK.kfx-zip ```
ElleKayEm commented 1 year ago

You'll need an earlier version of Kindle for PC to avoid the latest KFX format. See this post: https://www.mobileread.com/forums/showthread.php?t=283371

smilodon79 commented 1 year ago

Amazon is disallowing Download option for a lot of the new files. Likewise, attempting to download using Kindle version 1.17.44 gives a message that this book can't be downloaded and the kindle needs to be upgraded to download. Request a permanent solution please.

ElleKayEm commented 1 year ago

Unfortunately the last two times that DeDRM was specifically updated to deal with this, Amazon broke the fix within one day.

j-howell commented 1 year ago

Amazon is disallowing Download option for a lot of the new files. Likewise, attempting to download using Kindle version 1.17.44 gives a message that this book can't be downloaded and the kindle needs to be upgraded to download.

Amazon uses their toughest DRM only for some categories of books. The main ones so far are expensive textbooks and books published in India.

If you are seeing this applied to further categories of new books it would be good if you would share that information.

Request a permanent solution please.

Amazon has a large set of DRM algorithms already implemented in their reading apps and devices. Each time one of them has been solved Amazon has just switched to another within a day, rendering the effort involved in defeating that DRM useless.

Someone might be able to solve all of the deployed but unused DRM schemes, but that would be extremely difficult. And in the end the next software releases from Amazon will add more new DRM, locking things down again. So even in the best case that would likely be good for only a few weeks.

smilodon79 commented 1 year ago

The book I’m talking about costs $2.50 and was published by the author originally on 1992. The book link is: Enn? Etharku? Eppadi? (Part -1) (Tamil Edition) https://www.amazon.com/dp/B06XW64F7G?ref_=cm_sw_r_apin_dp_6642A2X3WEGVGGJ0V1SD. So yes, I suspect Amazon might be applying the newer encryption to increasing array of books.

On Mon, Dec 26, 2022 at 11:45 AM John Howell @.***> wrote:

Amazon is disallowing Download option for a lot of the new files. Likewise, attempting to download using Kindle version 1.17.44 gives a message that this book can't be downloaded and the kindle needs to be upgraded to download.

Amazon uses their toughest DRM only for some categories of books. The main ones so far are expensive textbooks and books published in India.

If you are seeing this applied to further categories of new books it would be good if you would share that information.

Request a permanent solution please.

Amazon has a large set of DRM algorithms already implemented in their reading apps and devices. Each time one of them has been solved Amazon has just switched to another within a day, rendering the effort involved in defeating that DRM useless.

Someone might be able to solve all of the deployed but unused DRM schemes, but that would be extremely difficult. And in the end the next software releases from Amazon will add more new DRM, locking things down again. So even in the best case that would likely be good for only a few weeks.

— Reply to this email directly, view it on GitHub https://github.com/apprenticeharper/DeDRM_tools/issues/2182#issuecomment-1365281136, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPFBQB52H7PJQMQNPLGFDDWPHDS5ANCNFSM6AAAAAATH4KA3Q . You are receiving this because you commented.Message ID: @.***>

j-howell commented 1 year ago

(Tamil Edition)

Books in Tamil are are only available in KFX format. Kindle for PC version 1.26 should be able to download that book with older DRM that DeDRM can handle. See Method 4 in Dealing with Kindle for PC/Mac 1.19 (and later) and KFX in calibre.

smilodon79 commented 1 year ago

Appreciate the prompt response. I downloaded Kindle for PC version 1.26 on a clean Windows 11 VM and validated the SHA-256 hash (it was the same as listed in the Method 4 of the link above). I downloaded the books using that Kindle for PC version 1.26 (after disabling Automatic Updates in the App and by deleting the updates folder and replacing with a file called 'updates' as indicated in Method 4). I tried converting the books using Calibre (version 6.10) with DeDRM (version 10.0.3) and KFS-Input (version 1.50.0).

Still getting the same DRM error message. Please advise on what to do next.

On Mon, Dec 26, 2022 at 1:33 PM John Howell @.***> wrote:

(Tamil Edition)

Books in Tamil are are only available in KFX format. Kindle for PC version 1.26 should be able to download that book with older DRM that DeDRM can handle. See Method 4 in Dealing with Kindle for PC/Mac 1.19 (and later) and KFX in calibre https://www.mobileread.com/forums/showthread.php?t=283371.

— Reply to this email directly, view it on GitHub https://github.com/apprenticeharper/DeDRM_tools/issues/2182#issuecomment-1365351705, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPFBQGRTAUI2MKYIOTGCZTWPHQIZANCNFSM6AAAAAATH4KA3Q . You are receiving this because you commented.Message ID: @.***>

j-howell commented 1 year ago

Still getting the same DRM error message. Please advise on what to do next.

Post the log from a failed import.

Also double check that the Kindle software is at the expected version. (Help, About Kindle.)

smilodon79 commented 1 year ago

Appreciate the phenomenal support so far. Just started on Christmas vacation trip. Will respond with the logs once I’m back.

Thanks again for the prompt responses!

On Mon, Dec 26, 2022 at 3:49 PM John Howell @.***> wrote:

Still getting the same DRM error message. Please advise on what to do next.

Post the log from a failed import.

— Reply to this email directly, view it on GitHub https://github.com/apprenticeharper/DeDRM_tools/issues/2182#issuecomment-1365453017, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPFBQANW64DIIP5BAZB3SDWPIAFDANCNFSM6AAAAAATH4KA3Q . You are receiving this because you commented.Message ID: @.***>

j-howell commented 1 year ago

Forgot to mention, each registered install of Kindle for PC has its own encryption key. You need to run calibre in the same VM as Kindle for PC so that it can obtain the key.

smilodon79 commented 1 year ago

Gotcha. That will probably do the trick.

On Mon, Dec 26, 2022 at 6:02 PM John Howell @.***> wrote:

Forgot to mention, each registered install of Kindle for PC has its own encryption key. You need to run calibre in the same VM as Kindle for PC so that it can obtain the key.

— Reply to this email directly, view it on GitHub https://github.com/apprenticeharper/DeDRM_tools/issues/2182#issuecomment-1365496841, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPFBQHNQU7MJOFDP2DMI6LWPIPX7ANCNFSM6AAAAAATH4KA3Q . You are receiving this because you commented.Message ID: @.***>

smilodon79 commented 1 year ago

Hi John,

Just tried it the way you indicated and got the following error:

calibre, version 6.10.0 ERROR: KFX conversion failed:

Cannot convert Enn? Etharku? Eppadi? (Part -2) (Tamil Edition)

Exception('Container entity signature is incorrect (00 45 33 c9)')

I've attached what I assume is the correct log file. Please let me know if I can provide anything else that can help resolve this issue.

Thanks in advance!

On Mon, Dec 26, 2022 at 6:02 PM John Howell @.***> wrote:

Forgot to mention, each registered install of Kindle for PC has its own encryption key. You need to run calibre in the same VM as Kindle for PC so that it can obtain the key.

— Reply to this email directly, view it on GitHub https://github.com/apprenticeharper/DeDRM_tools/issues/2182#issuecomment-1365496841, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPFBQHNQU7MJOFDP2DMI6LWPIPX7ANCNFSM6AAAAAATH4KA3Q . You are receiving this because you commented.Message ID: @.***>

j-howell commented 1 year ago

I've attached what I assume is the correct log file. Please let me know if I can provide anything else that can help resolve this issue.

Sorry, but I do not see any attachment.

smilodon79 commented 1 year ago

_0etg6f7_worker_redirect.log Guess attachments might emails may not get through. Attaching it through the browser this time.

smilodon79 commented 1 year ago

Please disregard my earlier log file. I wanted to try everything again from a clean slate. So I spun up a new VM, installed Kindle and Calibre (plus plugins) from scratch and tried Method 4 and it seems to have worked fine.

Appreciate your handholding on this issue!

j-howell commented 1 year ago

I was going to say that your log showed a serious data corruption problem.

Glad you got it working.

seasaltseasalt commented 1 year ago

Hello: I have cheap ($6) amazon kindle. downloaded azw. Amazon is disallowing Download option for a lot of the new files. Likewise, attempted to download using Kindle for mac version 1.11 and gives a message that this book can't be downloaded and the kindle needs to be upgraded to download. Is there any permanent solution?

ElleKayEm commented 1 year ago

There's never a permanent solution. All currently known workarounds are in this post: https://www.mobileread.com/forums/showthread.php?t=283371