New AZW3 books from Amazon download/transfer to your device do not DeDRM

[smumustang]

I was using Calibre v6.2 but updated to v6.5 just to make sure that wasn't the issue. I also updated the DeDRM to 10.0.3. I still get the message:

I've been using this plugin for years with no problem, and today I did the same thing I normally do: login to Amazon's Manage Your Kindle, select "download/transfer via USB" to download the AZW3 (and it is an AZW3 that it downloaded), and now it won't DeDRM that book. Books that I downloaded this way, say, four weeks ago and are already on my hard drive do DeDRM correctly, so it appears to be something new with Amazon's download.

[ElleKayEm]

Did you choose an E-ink Kindle for the device when downloading from the website? And do you have that Kindle's serial number entered correctly into DeDRM's customization.

[smumustang]

Yes, I did...if you'd like, I can send you the AZW3 and the serial number.

From: ElleKayEm @.> Sent: Tuesday, September 20, 2022 3:45 PM To: noDRM/DeDRM_tools @.> Cc: Shawn Prestridge @.>; Author @.> Subject: Re: [noDRM/DeDRM_tools] New AZW3 books from Amazon download/transfer to your device do not DeDRM (Issue #158)

Did you choose an E-ink Kindle for the device when downloading from the website? And do you have that Kindle's serial number entered correctly into DeDRM's customization.

- Reply to this email directly, view it on GitHubhttps://eur05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FnoDRM%2FDeDRM_tools%2Fissues%2F158%23issuecomment-1252891774&data=05%7C01%7Cshawn%40iar.com%7C29784503a64548c3d15008da9b48f607%7C752d689536a34853a00fc9efc645753e%7C1%7C0%7C637993034917968176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=tgn43l%2FdmgI9eS59GrnRWogyuRyAC2gA4OHG1Bm2Zjk%3D&reserved=0, or unsubscribehttps://eur05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJURKVX55YW7XRANIGXNOUTV7IO4BANCNFSM6AAAAAAQRNAQ3M&data=05%7C01%7Cshawn%40iar.com%7C29784503a64548c3d15008da9b48f607%7C752d689536a34853a00fc9efc645753e%7C1%7C0%7C637993034917968176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=IQCTnQM4vArg238B8XYm53b2xx1ONMu1MnOr16Lo%2B%2F4%3D&reserved=0. You are receiving this because you authored the thread.Message ID: @.**@.>>

[smumustang]

To reiterate my first message, I've been doing this for years: 1.) Go to amazon.com/myk 2.) Select book and click "Download/Transfer via USB" 3.) Select a specific eInk Kindle. Has worked great. Still works on books I downloaded this way a few weeks ago. But on a book I downloaded today, it doesn't work. Curiously, when I go to https://dedrm.com/ and upload the same book that won't DeDRM in Calibre with the same serial number, it will DeDRM the book.

[ElleKayEm]

Just checked and it is still working for me. Double-check device and serial number if you haven't already. If it still won't work, let's take a look at a debug log: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md#i-cannot-solve-my-problem-with-the-dedrm-plugin-and-now-i-need-to-post-a-log-how-do-i-do-that

[smumustang]

Here is the debug log: ApplicationPaletteChange event ignored Using calibre Qt style: True ApplicationPaletteChange event received calibre Debug log calibre 6.5 embedded-python: True Windows-10-10.0.22000-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.22000') Python 3.10.1 Windows: ('10', '10.0.22000', 'SP0', 'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: DeDRM (10, 0, 3) calibre 6.5 embedded-python: True Windows-10-10.0.22000-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.22000') Python 3.10.1 Windows: ('10', '10.0.22000', 'SP0', 'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: DeDRM (10, 0, 3) QPA platform: windows devicePixelRatio: 1.25 logicalDpi: 96.0 x 96.0 physicalDpi: 81.55183946488295 x 81.64285714285714 [0.00] Starting up... [0.00] Showing splash screen... [0.16] splash screen shown [0.16] Initializing db... [0.20] db initialized [0.20] Constructing main UI... [0.77] main UI initialized... [0.77] Hiding splash screen Starting QuickView [5.45] splash screen hidden [5.45] Started up in 5.45 seconds with 805 books Worker Launch took: 0.02 seconds Job: 0 Convert book 1 of 1 (Everydata) finished Convert book 1 of 1 (Everydata) DeDRM v10.0.3: Trying to decrypt 5crw3w09.azw3 Using Library AlfCrypto DLL/DYLIB/SO MobiDeDrm v1.1. Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al. Decrypting Kindle Format 8 ebook: Everydata Found 2 keys to try after 0.1 seconds Crypto Type is: 2 DeDRM v10.0.3: Failed to decrypt with error: No key found in 2 PIDs tried. DeDRM v10.0.3: Looking for new default Kindle Key after 0.1 seconds searching for kinfoFiles in C:\Users\shawnpr\AppData\Local Found K4PC 1.25+ kinf2018 file: C:\Users\shawnpr\AppData\Local\Amazon\Kindle\storage.kinf2018 Decrypted key file using IDString '2858587680' and UserName 'ShawnPr' DeDRM v10.0.3: Found 1 new key MobiDeDrm v1.1. Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al. Decrypting Kindle Format 8 ebook: Everydata Got DSN key from database key_0 Found 4 keys to try after 0.2 seconds Crypto Type is: 2 Traceback (most recent call last): File "calibre_plugins.dedrm.init", line 957, in KindleMobiDecrypt File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook mobidedrm.DrmException: No key found in 2 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 991, in KindleMobiDecrypt
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 4 PIDs tried.
DeDRM v10.0.3: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 2 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre\customize\ui.py", line 178, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 1056, in run
  File "calibre_plugins.dedrm.__init__", line 1007, in KindleMobiDecrypt
calibre_plugins.dedrm.DeDRMError: DeDRM v10.0.3: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md
Conversion options changed from defaults:
  read_metadata_from_opf: 'C:\\Users\\shawnpr\\AppData\\Local\\Temp\\calibre_whru00g1\\y_bx7lgx.opf'
  output_profile: 'kindle_voyage'
  verbose: 2
  cover: 'C:\\Users\\shawnpr\\AppData\\Local\\Temp\\calibre_whru00g1\\7qmuwtn0.jpeg'
Resolved conversion options
calibre version: 6.5.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0.0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': "//*[((name()='h1' or name()='h2') and re:test(., "
            "'\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', "
            "'i')) or @class = 'chapter']",
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': 'C:\\Users\\shawnpr\\AppData\\Local\\Temp\\calibre_whru00g1\\7qmuwtn0.jpeg',
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_compress': False,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': '',
 'fix_indents': True,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x00000264D96E4250>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0.0,
 'linearize_tables': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'mobi_file_type': 'old',
 'mobi_ignore_margins': False,
 'mobi_keep_original_images': False,
 'mobi_toc_at_start': False,
 'no_chapters_in_toc': False,
 'no_inline_navbars': True,
 'no_inline_toc': False,
 'output_profile': <calibre.customize.profiles.KindleVoyageOutput object at 0x00000264D96E7610>,
 'page_breaks_before': "//*[name()='h1' or name()='h2']",
 'personal_doc': '[PDOC]',
 'prefer_author_sort': False,
 'prefer_metadata_cover': False,
 'pretty_print': False,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': 'C:\\Users\\shawnpr\\AppData\\Local\\Temp\\calibre_whru00g1\\y_bx7lgx.opf',
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': '[]',
 'series': None,
 'series_index': None,
 'share_not_sync': False,
 'smarten_punctuation': False,
 'sr1_replace': None,
 'sr1_search': None,
 'sr2_replace': None,
 'sr2_search': None,
 'sr3_replace': None,
 'sr3_search': None,
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': '[]',
 'transform_html_rules': '[]',
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
DeDRM v10.0.3: Trying to decrypt 5crw3w09.azw3
MobiDeDrm v1.1.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Everydata
Found 2 keys to try after 0.1 seconds
Crypto Type is: 2
DeDRM v10.0.3: Failed to decrypt with error: No key found in 2 PIDs tried.
DeDRM v10.0.3: Looking for new default Kindle Key after 0.1 seconds
searching for kinfoFiles in C:\Users\shawnpr\AppData\Local
Found K4PC 1.25+ kinf2018 file: C:\Users\shawnpr\AppData\Local\Amazon\Kindle\storage\.kinf2018
Decrypted key file using IDString '2858587680' and UserName 'ShawnPr'
DeDRM v10.0.3: Found 1 new key
MobiDeDrm v1.1.
Copyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Everydata
Got DSN key from database key_0
Found 4 keys to try after 0.2 seconds
Crypto Type is: 2
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 2 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 991, in KindleMobiDecrypt
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 4 PIDs tried.
DeDRM v10.0.3: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md
Running file type plugin DeDRM failed with traceback:
Traceback (most recent call last):
  File "calibre_plugins.dedrm.__init__", line 957, in KindleMobiDecrypt
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\k4mobidedrm.py", line 261, in GetDecryptedBook
  File "C:\Users\shawnpr\AppData\Roaming\calibre\plugins\DeDRM.zip\mobidedrm.py", line 501, in processBook
mobidedrm.DrmException: No key found in 2 PIDs tried.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre\customize\ui.py", line 178, in _run_filetype_plugins
  File "calibre_plugins.dedrm.__init__", line 1056, in run
  File "calibre_plugins.dedrm.__init__", line 1007, in KindleMobiDecrypt
calibre_plugins.dedrm.DeDRMError: DeDRM v10.0.3: Ultimately failed to decrypt after 0.2 seconds. Read the FAQs at noDRM's repository: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md
InputFormatPlugin: MOBI Input running
on C:\Users\shawnpr\AppData\Local\Temp\calibre_whru00g1\5crw3w09.azw3
Found KF8 MOBI of type 'standalone'
Traceback (most recent call last):
  File "runpy.py", line 196, in _run_module_as_main
  File "runpy.py", line 86, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 215, in main
  File "calibre\gui2\convert\gui_conversion.py", line 38, in gui_convert_override
  File "calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1108, in run
  File "calibre\customize\conversion.py", line 242, in __call__
  File "calibre\ebooks\conversion\plugins\mobi_input.py", line 44, in convert
  File "calibre\ebooks\mobi\reader\mobi8.py", line 85, in __call__
  File "calibre\ebooks\mobi\reader\mobi6.py", line 162, in check_for_drm
calibre.ebooks.DRMError: Everydata

[smumustang]

I did double-check the Kindle that I used to download from amazon.com/myk is correct (and is eInk) and double-checked the Kindle serial number. As I said, Calbire and this same plugin will remove DRM from books I download a few weeks ago, but not books I've downloaded in the last few days.

[ElleKayEm]

Hmm, log just looks like the key that's entered isn't the right one for this copy of the book. I'd try deleting the book from calibre and adding a freshly downloaded one. Also try copy and paste of the serial number from the website and restarting calibre before adding the book. As I said, it is still working for me. And that DeDRM website is likely based on the code from the plugin.

[smumustang]

It's definitely not the wrong serial number in the plugin. Within Calibre, I can remove DRM from any book I downloaded from Amazon 10Aug22 and earlier. The next time I downloaded books was 31Aug22, and nothing after that time will remove DRM. I use the same Kindle in "Download/transfer via USB" for all of the books that I download, so I know that's not the issue, either.

I've tried re-downloading the books from 31Aug22 onward, and I get the same problem:

[smumustang]

On a lark, I added the same serial number into the plugin again...now the serial number is listed twice and it actually works! The curious thing is that the first one looks like it has a blank line above it...

[ElleKayEm]

Hmm, I wonder if the original serial number got messed up during a calibre update or plugin update. @noDRM might need to look into this.

[noDRM]

Interesting, I don't think Calibre updates would do anything with the stored plugin data.

I'd need to see the exact config file - @smumustang if you'd like me to look into that I'd need you to send me the "dedrm.json" file from the Calibre config folder (in Calibre, go to Preferences -> Miscellaneous -> Open calibre configuration folder -> "plugins") and find the dedrm.json file and send that to the email address in my GitHub profile, then I can check to see if I can find what caused this.

My first guess would be that you accidentally copied (a) space(s) or a newline with the serial number when you first added it. I'll try to have the plugin automatically remove spaces from the serial number to prevent that.

[alexbuisse]

I am having the same issue with a book I just bought on Amazon. Just like smumustang, this is a setup that was working fine until recently, and I haven't changed anything. I checked there were no spaces anywhere in the serial number, and even tried retyping it manually (the plugin wouldn't let me add it again, as it said it was already there). I believe this is a separate issue, but since the symptoms are exactly the same, am adding it here.

I ran calibre in debug mode and got the following log:

calibre.txt

[ElleKayEm]

Just to double-check, you chose an E-ink Kindle when downloading and that Kindle's serial number is in DeDRM's customization?

[alexbuisse]

Just to double-check, you chose an E-ink Kindle when downloading and that Kindle's serial number is in DeDRM's customization?

Correct on both counts. Again, this exact setup used to work until recently (I last used it this spring).

[ElleKayEm]

Do you only have the one device on your account? If not, I'd try downloading again to make sure you chose the right one.

Just double-checked myself and this method is still working for me.