search

apprenticeharper / DeDRM_tools

DeDRM tools for ebooks
14.52k stars 1.51k forks source link

DRM is not being removed: Calibre 4.23/DeDRM6.8/Kindle for Mac 1.26.1/Mac Big Sur #1369

Closed ghost closed 3 years ago

ghost commented 3 years ago

CheckList

Title

DRM is not being removed

My Environment

Calibre: 4.23

Kindle: 1.26.1

DeDRM: 6.8

Mac mini M1 Big Sur image

Log

Removing DRM of Salt to the Sea ``` calibre Debug log calibre 4.23 embedded-python: True is64bit: True Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '') ('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101') Python 2.7.16 OSX: ('10.16', ('', '', ''), 'x86_64') Interface language: None Successfully initialized third party plugins: DeDRM (6, 8, 0) calibre 4.23 embedded-python: True is64bit: True Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '') ('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101') Python 2.7.16 OSX: ('10.16', ('', '', ''), 'x86_64') Interface language: None Successfully initialized third party plugins: DeDRM (6, 8, 0) devicePixelRatio: 1.0 logicalDpi: 72.0 x 72.0 physicalDpi: 92.0000013817 x 91.0000013667 Using calibre Qt style: True [0.00] Starting up... [0.01] Showing splash screen... [0.21] splash screen shown [0.21] Initializing db... [0.22] db initialized [0.22] Constructing main UI... [0.80] main UI initialized... [0.80] Hiding splash screen [8.18] splash screen hidden [8.18] Started up in 8.18 seconds with 253 books DeDRM v6.8.0: Trying to decrypt B00YM34WM8_EBOK.azw Using Library AlfCrypto Python MobiDeDrm v0.42. Copyright © 2008-2017 The Dark Reverser, Apprentice Harper et al. Decrypting Kindle Format 8 ebook: Salt to the Sea Found 0 keys to try after 0.2 seconds Crypto Type is: 2 DeDRM v6.8.0: Failed to decrypt with error: No key found in 0 keys tried. DeDRM v6.8.0: Looking for new default Kindle Key after 0.2 seconds Found k4Mac kinf2018 file: /Users/lisa/Library/Application Support/Kindle/storage/.kinf2018 Couldn't decrypt file. DeDRM v6.8.0: Ultimately failed to decrypt after 0.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md Running file type plugin DeDRM failed with traceback: Traceback (most recent call last): File "site-packages/calibre/customize/ui.py", line 172, in _run_filetype_plugins File "calibre_plugins.dedrm.__init__", line 625, in run File "calibre_plugins.dedrm.__init__", line 575, in KindleMobiDecrypt DeDRMError: DeDRM v6.8.0: Ultimately failed to decrypt after 0.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md Worker Launch took: 0.0718188285828 Job: 0 Convert book 1 of 1 (Salt to the Sea) finished Convert book 1 of 1 (Salt to the Sea) Conversion options changed from defaults: read_metadata_from_opf: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/BhxNOm.opf' verbose: 2 cover: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/f3MaNO.jpeg' output_profile: u'kindle_oasis' input_profile: u'kindle' Resolved conversion options calibre version: 4.23.0 {'asciiize': False, 'author_sort': None, 'authors': None, 'base_font_size': 0.0, 'book_producer': None, 'change_justification': u'original', 'chapter': u"//*[((name()='h1' or name()='h2') and re:test(., '\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', 'i')) or @class = 'chapter']", 'chapter_mark': u'pagebreak', 'comments': None, 'cover': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/f3MaNO.jpeg', 'debug_pipeline': None, 'dehyphenate': True, 'delete_blank_paragraphs': True, 'disable_font_rescaling': False, 'dont_split_on_page_breaks': False, 'duplicate_links_in_toc': False, 'embed_all_fonts': False, 'embed_font_family': None, 'enable_heuristics': False, 'epub_flatten': False, 'epub_inline_toc': False, 'epub_toc_at_end': False, 'epub_version': u'2', 'expand_css': False, 'extra_css': None, 'extract_to': None, 'filter_css': u'', 'fix_indents': True, 'flow_size': 260, 'font_size_mapping': None, 'format_scene_breaks': True, 'html_unwrap_factor': 0.4, 'input_encoding': None, 'input_profile': , 'insert_blank_line': False, 'insert_blank_line_size': 0.5, 'insert_metadata': False, 'isbn': None, 'italicize_common_cases': True, 'keep_ligatures': False, 'language': None, 'level1_toc': None, 'level2_toc': None, 'level3_toc': None, 'line_height': 0.0, 'linearize_tables': False, 'margin_bottom': 5.0, 'margin_left': 5.0, 'margin_right': 5.0, 'margin_top': 5.0, 'markup_chapter_headings': True, 'max_toc_links': 50, 'minimum_line_height': 120.0, 'no_chapters_in_toc': False, 'no_default_epub_cover': False, 'no_inline_navbars': False, 'no_svg_cover': False, 'output_profile': , 'page_breaks_before': u"//*[name()='h1' or name()='h2']", 'prefer_metadata_cover': False, 'preserve_cover_aspect_ratio': False, 'pretty_print': True, 'pubdate': None, 'publisher': None, 'rating': None, 'read_metadata_from_opf': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/BhxNOm.opf', 'remove_fake_margins': True, 'remove_first_image': False, 'remove_paragraph_spacing': False, 'remove_paragraph_spacing_indent_size': 1.5, 'renumber_headings': True, 'replace_scene_breaks': u'', 'search_replace': '[]', 'series': None, 'series_index': None, 'smarten_punctuation': False, 'sr1_replace': None, 'sr1_search': None, 'sr2_replace': None, 'sr2_search': None, 'sr3_replace': None, 'sr3_search': None, 'start_reading_at': None, 'subset_embedded_fonts': False, 'tags': None, 'timestamp': None, 'title': None, 'title_sort': None, 'toc_filter': None, 'toc_threshold': 6, 'toc_title': None, 'transform_css_rules': '[]', 'unsmarten_punctuation': False, 'unwrap_lines': True, 'use_auto_toc': False, 'verbose': 2} InputFormatPlugin: MOBI Input running on /var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_ivwUhU/FurDhx.azw Found KF8 MOBI of type u'standalone' Python function terminated unexpectedly: Salt to the Sea Traceback (most recent call last): File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 187, in main return run_entry_point() File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 121, in run_entry_point return getattr(pmod, func)() File "site-packages/calibre/utils/ipc/worker.py", line 209, in main File "site-packages/calibre/gui2/convert/gui_conversion.py", line 43, in gui_convert_override File "site-packages/calibre/gui2/convert/gui_conversion.py", line 28, in gui_convert File "site-packages/calibre/ebooks/conversion/plumber.py", line 1110, in run File "site-packages/calibre/customize/conversion.py", line 246, in __call__ File "site-packages/calibre/ebooks/conversion/plugins/mobi_input.py", line 47, in convert File "site-packages/calibre/ebooks/mobi/reader/mobi8.py", line 86, in __call__ File "site-packages/calibre/ebooks/mobi/reader/mobi6.py", line 158, in check_for_drm calibre.ebooks.DRMError: Salt to the Sea ```
ElleKayEm commented 3 years ago

Was the book downloaded with the current installation of Kindle for Mac located on the same machine and user as calibre with DeDRM?

ghost commented 3 years ago

Yes. But not the "current" installation of Kindle for Mac. It was version 1.26.1

ElleKayEm commented 3 years ago

For some reason, DeDRM is unable to decrypt the key that it is finding. Possibly something left behind from a previous installation. Try deleting this file: /Users/lisa/Library/Application Support/Kindle/storage/.kinf2018 Then remove and re-add the book.

ghost commented 3 years ago

So I decided to just delete everything and start over. I uninstalled Kindle, Calibre, deDRM plugin. (I also deleted that entire Kindle folder under Application Support (and anywhere else I found Kindle App items). I reinstalled everything in the versions noted above. Tried to import a book from my Amazon library and received the same error. I went to the /Users/lisa/Library/Application Support/Kindle/storage/ folder and that .kinf2018 file was there again. So I deleted it (again). When I went back into Kindle for Mac, my "device registration" was deleted, so I had to re-register that. But it kept crashing. Once I de-registered from Amazon's site, it let me re-register it. I deleted the book and tried one more time to download and import the book into Calibre, and still received the cannot convert error. So I did it all one more time to gather the log. That .kinf2018 keeps recreating. I don't know if that's expected or not.

image

Calibre Log ``` calibre Debug log calibre 4.23 embedded-python: True is64bit: True Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '') ('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101') Python 2.7.16 OSX: ('10.16', ('', '', ''), 'x86_64') Interface language: None Successfully initialized third party plugins: DeDRM (6, 8, 0) calibre 4.23 embedded-python: True is64bit: True Darwin-20.1.0-x86_64-i386-64bit Darwin ('64bit', '') ('Darwin', '20.1.0', 'Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101') Python 2.7.16 OSX: ('10.16', ('', '', ''), 'x86_64') Interface language: None Successfully initialized third party plugins: DeDRM (6, 8, 0) devicePixelRatio: 1.0 logicalDpi: 72.0 x 72.0 physicalDpi: 92.0000013817 x 91.0000013667 Using calibre Qt style: True [0.00] Starting up... [0.02] Showing splash screen... [0.21] splash screen shown [0.21] Initializing db... [0.23] db initialized [0.23] Constructing main UI... [0.81] main UI initialized... [0.81] Hiding splash screen [4.29] splash screen hidden [4.29] Started up in 4.29 seconds with 255 books DeDRM v6.8.0: Trying to decrypt B083J7QQ89_EBOK.azw Using Library AlfCrypto Python MobiDeDrm v0.42. Copyright © 2008-2017 The Dark Reverser, Apprentice Harper et al. Decrypting Kindle Format 8 ebook: The Cipher (Nina Guerrera) Found 0 keys to try after 0.0 seconds Crypto Type is: 2 DeDRM v6.8.0: Failed to decrypt with error: No key found in 0 keys tried. DeDRM v6.8.0: Looking for new default Kindle Key after 0.0 seconds Found k4Mac kinf2018 file: /Users/lisa/Library/Application Support/Kindle/storage/.kinf2018 Couldn't decrypt file. DeDRM v6.8.0: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md Running file type plugin DeDRM failed with traceback: Traceback (most recent call last): File "site-packages/calibre/customize/ui.py", line 172, in _run_filetype_plugins File "calibre_plugins.dedrm.__init__", line 625, in run File "calibre_plugins.dedrm.__init__", line 575, in KindleMobiDecrypt DeDRMError: DeDRM v6.8.0: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md Worker Launch took: 0.0827639102936 Job: 0 Convert book 1 of 1 (The Cipher (Nina Guerrera)) finished Convert book 1 of 1 (The Cipher (Nina Guerrera)) Conversion options changed from defaults: read_metadata_from_opf: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/hFtlGT.opf' verbose: 2 cover: u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/krqMvT.jpeg' input_profile: u'kindle' output_profile: u'kindle_oasis' Resolved conversion options calibre version: 4.23.0 {'asciiize': False, 'author_sort': None, 'authors': None, 'base_font_size': 0.0, 'book_producer': None, 'change_justification': u'original', 'chapter': u"//*[((name()='h1' or name()='h2') and re:test(., '\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', 'i')) or @class = 'chapter']", 'chapter_mark': u'pagebreak', 'comments': None, 'cover': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/krqMvT.jpeg', 'debug_pipeline': None, 'dehyphenate': True, 'delete_blank_paragraphs': True, 'disable_font_rescaling': False, 'dont_split_on_page_breaks': False, 'duplicate_links_in_toc': False, 'embed_all_fonts': False, 'embed_font_family': None, 'enable_heuristics': False, 'epub_flatten': False, 'epub_inline_toc': False, 'epub_toc_at_end': False, 'epub_version': u'2', 'expand_css': False, 'extra_css': None, 'extract_to': None, 'filter_css': u'', 'fix_indents': True, 'flow_size': 260, 'font_size_mapping': None, 'format_scene_breaks': True, 'html_unwrap_factor': 0.4, 'input_encoding': None, 'input_profile': , 'insert_blank_line': False, 'insert_blank_line_size': 0.5, 'insert_metadata': False, 'isbn': None, 'italicize_common_cases': True, 'keep_ligatures': False, 'language': None, 'level1_toc': None, 'level2_toc': None, 'level3_toc': None, 'line_height': 0.0, 'linearize_tables': False, 'margin_bottom': 5.0, 'margin_left': 5.0, 'margin_right': 5.0, 'margin_top': 5.0, 'markup_chapter_headings': True, 'max_toc_links': 50, 'minimum_line_height': 120.0, 'no_chapters_in_toc': False, 'no_default_epub_cover': False, 'no_inline_navbars': False, 'no_svg_cover': False, 'output_profile': , 'page_breaks_before': u"//*[name()='h1' or name()='h2']", 'prefer_metadata_cover': False, 'preserve_cover_aspect_ratio': False, 'pretty_print': True, 'pubdate': None, 'publisher': None, 'rating': None, 'read_metadata_from_opf': u'/var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/hFtlGT.opf', 'remove_fake_margins': True, 'remove_first_image': False, 'remove_paragraph_spacing': False, 'remove_paragraph_spacing_indent_size': 1.5, 'renumber_headings': True, 'replace_scene_breaks': u'', 'search_replace': '[]', 'series': None, 'series_index': None, 'smarten_punctuation': False, 'sr1_replace': None, 'sr1_search': None, 'sr2_replace': None, 'sr2_search': None, 'sr3_replace': None, 'sr3_search': None, 'start_reading_at': None, 'subset_embedded_fonts': False, 'tags': None, 'timestamp': None, 'title': None, 'title_sort': None, 'toc_filter': None, 'toc_threshold': 6, 'toc_title': None, 'transform_css_rules': '[]', 'unsmarten_punctuation': False, 'unwrap_lines': True, 'use_auto_toc': False, 'verbose': 2} InputFormatPlugin: MOBI Input running on /var/folders/j2/hhvbfrxx3dg9ry4xr4xpp0gc0000gn/C/calibre_4.23.0_tmp_mmt1ZU/g4FRD3.azw Found KF8 MOBI of type u'standalone' Python function terminated unexpectedly: The Cipher (Nina Guerrera) Traceback (most recent call last): File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 187, in main return run_entry_point() File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 121, in run_entry_point return getattr(pmod, func)() File "site-packages/calibre/utils/ipc/worker.py", line 209, in main File "site-packages/calibre/gui2/convert/gui_conversion.py", line 43, in gui_convert_override File "site-packages/calibre/gui2/convert/gui_conversion.py", line 28, in gui_convert File "site-packages/calibre/ebooks/conversion/plumber.py", line 1110, in run File "site-packages/calibre/customize/conversion.py", line 246, in __call__ File "site-packages/calibre/ebooks/conversion/plugins/mobi_input.py", line 47, in convert File "site-packages/calibre/ebooks/mobi/reader/mobi8.py", line 86, in __call__ File "site-packages/calibre/ebooks/mobi/reader/mobi6.py", line 158, in check_for_drm calibre.ebooks.DRMError: The Cipher (Nina Guerrera) ```
ElleKayEm commented 3 years ago

And you redownloaded the book after this new installation? And added the fresh copy to calibre?

ghost commented 3 years ago

Yes, I deleted from Calibre, deleted from My Kindle Library and redownloaded... multiple times.

ghost commented 3 years ago

I also have all these same versions installed on a MacBook running Catalina, and it all works fine. I'm suspicious that it has something to do with Big Sur.

ElleKayEm commented 3 years ago

Do you have KFX downloads disabled?

ghost commented 3 years ago

I think so. If you mean, running: chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test

Then yes. If not, then probably not.

ElleKayEm commented 3 years ago

Yes, that's what I meant. I don't know what else to suggest. Seems like it might be something to do with Big Sur. But I have no idea.

ghost commented 3 years ago

Are you one of the developers? If not, are they monitoring this thread? More and more users will be upgrading to Big Sur. Seems like this might be something they'd want to look into. I'm willing to try any other ideas to help debug this including (but not limited to), uninstalling, reinstalling, changing up the order of steps, etc.

ElleKayEm commented 3 years ago

I'm not. As you say, I'm sure there will be more reports if Big Sur is the issue, and @apprenticeharper and company will become aware. I will also ask around myself.

ghost commented 3 years ago

I tried posting to mobileread, but they closed it immediately. I didn't even post instructions, I just spoke about the problem in the abstract. I don't know where else to post to see if others are having the same issue.

ElleKayEm commented 3 years ago

Yeah, I think they were a little overzealous with that. Hopefully we get some responses from the other thread there.

ghost commented 3 years ago

image I decided to look at the customization of deDRM and the Keys are blank! Gonna try copying them from my Catalina Mac...

ghost commented 3 years ago

Saving the keys from my MacBook running Catalina and importing them to my Mac mini running Big Sur did not work: image I still received the "this book is locked by DRM" error.

If I click the plus sign, I get this error: image

So the problem seems to lie with the non-existence of the proper keys. Maybe @apprenticeharper can offer some help?

ElleKayEm commented 3 years ago

DeDRM will only automatically save a key that successfully decrypted a book. So that's why nothing was there at first. I think the error message you got when trying to add the key may be because DeDRM is not able to decrypt the key, not because it can't find one at all. At least that's what the log from importing a book seemed to indicate. And the key for each installation of the Kindle is unique (even a reinstallation on the same machine which requires you to login again can change the key). So that's why the keys from your Catalina don't work on books from the Big Sur Kindle app. I'd be curious to see if you could decrypt a book from the Catalina Kindle app with the Big Sur calibre/DeDRM setup with the Catalina's imported keys.

ghost commented 3 years ago

Ha! So that worked. I downloaded the book on my Catalina MacBook in Kindle for Mac. Then I copied that .azw file from the Catalina Mac to my Big Sur Mac. (Big Sur Mac had the keys from the Catalina Mac.) Catalina did it's import and it removed the DRM and created the .azw3 (and .epub) files!

So how do we get the proper keys for Big Sur???

ElleKayEm commented 3 years ago

Therein lies the problem. It appears that DeDRM can't decrypt the key for the Big Sur installation of the Kindle app. It's beyond my knowledge of this to see why. But at least that might narrow the issue for those who can figure it out.

ghost commented 3 years ago

If I use the "download via usb" method, it all works fine! TBH, that's easier anyway.

ElleKayEm commented 3 years ago

Ah, didn't know you had an E-ink Kindle. This will still need looking at, but at least book decryption seems fine with Big Sur. It's just getting and decrypting the key for Kindle for Mac that is giving trouble.

ghost commented 3 years ago

Thanks for all your help. If and when someone has time to fix this, just @ me or something and I'll be happy to test it.

Augilare commented 3 years ago

Patch kindle for mac catalina and Big Sur with following command, chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test OR sudo chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test, Then right-click kindle book to download. DON'T double click or open it before the drm removing. Convert Kindle KFX to EPUB/PDF Free on macOS Big Sur (Effective)

net-buoy commented 3 years ago

Remember that you have to do this every time the Kindle app is upgraded.

On 11/24/20 9:16 PM, Augilare wrote:

Patch kindle for mac catalina and Big Sur with following command, chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test OR sudo chmod -x /Applications/Kindle.app/Contents/MacOS/renderer-test, Then right-click kindle book to download. DON'T double click or open it before the drm removing. Convert Kindle KFX to EPUB/PDF Free on macOS Big Sur (Effective) https://garydadic.medium.com/convert-kindle-kfx-to-epub-pdf-free-on-mac-10-15-catalina-included-7ea1d7d2b609

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/apprenticeharper/DeDRM_tools/issues/1369#issuecomment-733491539, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWVX7LZDYKDEQFUF2PZNLDSRSOKJANCNFSM4T4K6REA.

-- "There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy. "

ElleKayEm commented 3 years ago

@Augilare The OP did this. The problem here is decrypting the Kindle for Mac key. Not KFX format (which can be decrypted if from K4Mac 1.26 anyway).

ZolaLa9 commented 3 years ago

@Lisa10101 — have you tried this with K4Mac 1.30?

The blogger referenced by @Augilare reports success using Calibre 4.23, DeDRM 6.8, K4Mac 1.30 and Mac mini M1 Big Sur.

ghost commented 3 years ago

@ZolaLa9 I have not tried 1.30, mostly because DeDRM 6.8 states it's not compatible with anything over 1.26. If I give it a try later, I'll report back.

ElleKayEm commented 3 years ago

@Lisa10101 That is in regard to KFX format. If you have KFX downloads disabled (chmod -x ...), you can use 1.30.

ZolaLa9 commented 3 years ago

@Lisa10101 — This post is updated regularly and tends to have the most up-to-date info on K4Mac compatibility: https://www.mobileread.com/forums/showpost.php?p=3819708&postcount=508

Not sure how active you are on MobileRead, but there is also active discussion happening here on DeDRM 7.0 beta and Mac: https://www.mobileread.com/forums/showthread.php?s=32fb948c4aa246ac91444f4a00ca6632&p=4061946#post4061946

ghost commented 3 years ago

I'm not interested in installing things that are "beta" or "in development", so I felt this was the better place to look for compatibility between DeDRM and Calibre: https://apprenticealf.wordpress.com It was just updated today, so I was going off the previous post which says this:

This means that this release now works with version 1.26 and earlier of Kindle for PC and Kindle for Mac.

and this:

Thanks to work by several people, the DeDRM calibre plugin can decrypt KFX format ebooks from Amazon as long as they are downloaded with Kindle for Mac/PC prior to version 1.27.

ghost commented 3 years ago

@ZolaLa9 @Augilare Just tried it using Kindle for Mac 1.30 (and yes, I did the step to disable KFX before downloading the book) and it did not work: image

I'm pretty skeptical that the author of that blog actually tested this on a new Mac mini M1 running Big Sur.

ElleKayEm commented 3 years ago

And the next sentence:

Books which require the later versions of Kindle for PC/Mac will not be able to be decrypted at this time.

But non-KFX formats can be decrypted from the later versions.

ghost commented 3 years ago

@ElleKayEm It doesn't say "But non-KFX formats can be decrypted from the later versions." There is nothing in that post from June 18, 2020 that would lead one to believe you could use DeDRM 6.8 with Kindle for Mac 1.30. Regardless... if all or most of your books are KFX formats, then anything over 1.26 does not work. (I don't want to muddy the waters here by discussing other tests.. but before I switched to Mac mini M1 Big Sur, I was using a MacBook Catalina and I was unable to use 1.30 there (disabling KFX of course) with DeDRM 6.8. I had to downgrade to 1.26 and disable KFX for it to work. )

ElleKayEm commented 3 years ago

Have a look at this old issue #255. It seemed to be a similar issue with DeDRM not being able to decrypt the Kindle for Mac key. Sounds like the key encryption is tied to an ID number from the internal drive. Perhaps something with Big Sur or your particular setup is causing DeDRM to grab the wrong number.

(DeDRM 6.8 addressed two things: decryption of the new Kindle for PC/Mac key which first appeared in version 1.25 and new KFX DRM used in versions 1.25-1.26. A further change to KFX DRM happened beginning with 1.27 which DeDRM does not handle yet. But the DRM on the older formats that you get when you disable KFX format downloads has not changed.)

j-howell commented 3 years ago

DeDRM uses a variety of cryptographic routines that do much of the same thing due to its history as a bunch of separate tools created by different individuals. I believe that the problem with Big Sur is caused by one of those no longer being available or having its named changed. This and others like it could be eliminated in favor of the cryptography library built into calibre.

protochron commented 3 years ago

It looks like the plugin is failing to load libcrypto because of a change in the way that Big Sur handles loading dynamic libraries. The libcrypto.dylib file no longer exists on disk, so if the plugin is running on a Mac then the code in https://github.com/apprenticeharper/DeDRM_tools/blob/v6.8.0/DeDRM_plugin/kindlekey.py#L1184-L1186 needs to try loading /usr/lib/libcrypto.dylib directly when libcrypto is None and handling an exception if it doesn't work after that. I tested this with Big Sur, Kindle for Mac 1.30, DeDRM 6.8.0 and Calibre 4.21 and it was able to handle getting the key just fine.

See https://github.com/saltstack/salt/issues/57787 for more info re: failing to load dylibs in Big Sur.

It does seem like loading Calibre's version of OpenSSL would solve this once and for all, but I'm happy to put in a PR with the fix.

ZolaLa9 commented 3 years ago

@protochron — AWESOME!!! If you would put in a PR, it would be mucho appreciated!

ZolaLa9 commented 3 years ago

before I switched to Mac mini M1 Big Sur, I was using a MacBook Catalina and I was unable to use 1.30 there (disabling KFX of course) with DeDRM 6.8. I had to downgrade to 1.26 and disable KFX for it to work.

@Lisa10101 — Catalina + K4Mac 1.30 + Calibre 4.23 + Alf 6.8.0 = working for me

apprenticeharper commented 3 years ago

I don't have Big Sur, and I'm still working on other Python 3 problems. A pull request that fixed this problem would be much appreciated.

jcs commented 3 years ago

1381 fixes the key finding/loading for me on Big Sur, though it still fails importing an azw file:

DeDRM v7.0.0: Trying to decrypt book.azw
Using Library AlfCrypto Python
Using Library AlfCrypto Python
MobiDeDrm v1.0.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: [title]
Got SerialNumber from database default_key
Got UsernameHash from database default_key
Found 4 keys to try after 0.0 seconds
Crypto Type is: 2
DeDRM v7.0.0: Failed to decrypt with error: No key found in 4 keys tried.
DeDRM v7.0.0: Looking for new default Kindle Key after 0.0 seconds
Found k4Mac kinf2018 file: /Users/jcs/Library/Containers/com.amazon.Kindle/Data/Library/Application Support/Kindle/storage/.kinf2018
Found k4Mac kinf2018 file: /Users/jcs/Library/Application Support/Kindle/storage/.kinf2018
trying username  b'jcs'  on file  /Users/jcs/Library/Containers/com.amazon.Kindle/Data/Library/Application Support/Kindle/storage/.kinf2018
trying IDString: b'bb8ce...'
trying IDString: b'bb8ce...'
trying IDString: b'bd9b4...'
Decrypted key file using IDString 'bd9b4...' and UserName 'jcs'
trying username  b'jcs'  on file  /Users/jcs/Library/Application Support/Kindle/storage/.kinf2018
trying IDString: b'bb8ce...'
trying IDString: b'bb8ce...'
trying IDString: b'bd9b4...'
Decrypted key file using IDString 'bd9b4...' and UserName 'jcs'
DeDRM v7.0.0: Found 1 new key
MobiDeDrm v1.0.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: [title]
Got SerialNumber from database default_key_2
Got UsernameHash from database default_key_2
Found 4 keys to try after 2.3 seconds
Crypto Type is: 2
DeDRM v7.0.0: Ultimately failed to decrypt after 2.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 533, in KindleMobiDecrypt
    book = k4mobidedrm.GetDecryptedBook(path_to_ebook,kindleDatabases,androidFiles,serials,pids,self.starttime)
  File "calibre_plugins.dedrm.k4mobidedrm", line 238, in GetDecryptedBook
    mb.processBook(totalpids)
  File "calibre_plugins.dedrm.mobidedrm", line 476, in processBook
    raise DrmException("No key found in {0:d} keys tried.".format(len(goodpids)))
calibre_plugins.dedrm.mobidedrm.DrmException: No key found in 4 keys tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre/customize/ui.py", line 172, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 626, in run
    decrypted_ebook = self.KindleMobiDecrypt(path_to_ebook)
  File "calibre_plugins.dedrm.__init__", line 576, in KindleMobiDecrypt
    raise DeDRMError("{0} v{1}: Ultimately failed to decrypt after {2:.1f} seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md".format(PLUGIN_NAME, PLUGIN_VERSION,time.time()-self.starttime))
calibre_plugins.dedrm.DeDRMError: DeDRM v7.0.0: Ultimately failed to decrypt after 2.3 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md
TheFreediver commented 2 years ago

I know this is an old thread but I had a similar issue with Catalina, kindle 1.26 and calibre 4 and 5. What Aguilare said not to actually click the book when downloading, helped me. It appears as soon as kindle opens the book it says preparing your book and deletes the azw format. Despite using kindle 1.26, updates disabled, updates folder removed, etc. So conclusion don't open the book in kindle app on mac os until you have imported in calibre.

Then right-click kindle book to download. DON'T double click or open it before the drm removing.