search

apprenticeharper / DeDRM_tools

DeDRM tools for ebooks
14.56k stars 1.52k forks source link

DeDRM v7.1.0: Failed to decrypt with error: can't concat str to bytes #1637

Open dtaylor84 opened 3 years ago

dtaylor84 commented 3 years ago

CheckList

DeDRM v7.1.0: Failed to decrypt with error: can't concat str to bytes

Trying to concatenate str to bytes in "calibre_plugins.dedrm.flatxml2html", line 583, in buildTOCEntry.

A hangover from the python 2/3 conversion?

My Environment

Calibre: v5.14.0

Kindle: v3.4.3

DeDRM: v7.1.0

Log

DeDRM v7.1.0: Failed to decrypt with error: can't concat str to bytes ```log calibre, version 5.14.0 ERROR: Loading book failed: Failed to open the book at C:\...\Calibre Library\Unknown\The Science of Cooking-asin B001GQ1 (561)\The Science of Cooking-asin B00 - Unknown.azw1. Click "Show details" for more info. Failed to convert book: C:\...\Calibre Library\Unknown\The Science of Cooking-asin B001GQ1 (561)\The Science of Cooking-asin B00 - Unknown.azw1 with error: DeDRM v7.1.0: Trying to decrypt The Science of Cooking-asin B00 - Unknown.azw1 Using Library AlfCrypto DLL/DYLIB/SO Using Library AlfCrypto DLL/DYLIB/SO Decrypting Topaz ebook: The Science of Cooking Found 4 keys to try after 0.3 seconds Trying: b'3kk4AIUo' Trying: b'rJQCVmM0' Book Key Found! (b847b05f64bc09ef) Processing Section: color . . . . . . . . . . . . . . . . . . . . Processing Section: dict . . . . Processing Section: glyphs . . . . . . . . . . . . . . . . . . . . . . . Processing Section: img . . . . . . . . . . . . . . . . . . . . Processing Section: metadata . . . . Processing Section: other . . . . Processing Section: page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Successfully Extracted Topaz contents Updating to color images if available Creating cover.jpg Processing Dictionary Processing Meta Data and creating OPF Processing StyleSheet [...] DeDRM v7.1.0: Failed to decrypt with error: can't concat str to bytes DeDRM v7.1.0: Looking for new default Kindle Key after 2.8 seconds searching for kinfoFiles in C:\...\AppData\Local Found K4PC 1.25+ kinf2018 file: C:\...\AppData\Local\Amazon\Kindle\storage\.kinf2018 Decrypted key file using IDString '...' and UserName '...' DeDRM v7.1.0: Found 1 new key Decrypting Topaz ebook: The Science of Cooking Got DSN key from database default_key_1 Found 4 keys to try after 13.5 seconds Trying: b'...' Trying: b'...' Trying: b'...' Trying: b'...' DeDRM v7.1.0: Ultimately failed to decrypt after 13.5 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md Failed to run pipe worker with command: from calibre.srv.render_book import viewer_main; viewer_main() Traceback (most recent call last): File "calibre\customize\ui.py", line 428, in get_file_type_metadata File "calibre\customize\builtins.py", line 416, in get_metadata File "calibre\ebooks\metadata\topaz.py", line 378, in get_metadata File "calibre\ebooks\metadata\topaz.py", line 114, in __init__ ValueError: 'C:\...\Calibre Library\Unknown\The Science of Cooking-asin B001GQ1 (561)\The Science of Cooking-asin B00 - Unknown.azw1': Invalid Topaz format - no metadata record Running file type plugin DeDRM failed with traceback: Traceback (most recent call last): File "calibre_plugins.dedrm.__init__", line 538, in KindleMobiDecrypt File "calibre_plugins.dedrm.k4mobidedrm", line 238, in GetDecryptedBook File "calibre_plugins.dedrm.topazextract", line 372, in processBook File "calibre_plugins.dedrm.genbook", line 520, in generateBook File "calibre_plugins.dedrm.flatxml2html", line 806, in convert2HTML File "calibre_plugins.dedrm.flatxml2html", line 739, in process File "calibre_plugins.dedrm.flatxml2html", line 583, in buildTOCEntry TypeError: can't concat str to bytes During handling of the above exception, another exception occurred: Traceback (most recent call last): File "calibre\customize\ui.py", line 172, in _run_filetype_plugins File "calibre_plugins.dedrm.__init__", line 631, in run File "calibre_plugins.dedrm.__init__", line 581, in KindleMobiDecrypt calibre_plugins.dedrm.DeDRMError: DeDRM v7.1.0: Ultimately failed to decrypt after 13.5 seconds. Read the FAQs at Harper's repository: https://github.com/apprenticeharper/DeDRM_tools/blob/master/FAQs.md Traceback (most recent call last): File "runpy.py", line 194, in _run_module_as_main File "runpy.py", line 87, in _run_code File "site.py", line 82, in File "site.py", line 77, in main File "site.py", line 49, in run_entry_point File "calibre\utils\ipc\worker.py", line 197, in main File "", line 1, in File "calibre\srv\render_book.py", line 870, in viewer_main File "calibre\srv\render_book.py", line 861, in render_for_viewer File "calibre\srv\render_book.py", line 838, in render File "calibre\ebooks\oeb\iterator\book.py", line 54, in extract_book File "calibre\ebooks\conversion\plumber.py", line 752, in __init__ ValueError: No plugin to handle input format: azw1 ```
ElleKayEm commented 3 years ago

Give calibre 4.23 and DeDRM 6.8.1 a try.

dtaylor84 commented 3 years ago

I hacked the code to work around the errors (see patch below), and it is now decrypting correctly with Calibre 5.14.0 and DeDRM 7.1.0+patch.

Hacky patch to work around str/byte errors ```diff --- DeDRM_tools_7.1.0/DeDRM_plugin/flatxml2html.py Tue Dec 22 16:37:50 2020 +++ DeDRM_tools_7.1.0.fixed/DeDRM_plugin/flatxml2html.py Fri Apr 2 18:43:10 2021 @@ -440,6 +440,10 @@ classres = '' if pclass : + try: # HACK fix byte vs str errors + pclass = pclass.encode('utf-8') + except (UnicodeDecodeError, AttributeError): + pass classres = ' class="' + pclass.decode('utf-8') + '"' br_lb = (regtype == 'fixed') or (regtype == 'chapterheading') or (regtype == 'vertical') @@ -473,6 +477,10 @@ if (link > 0): linktype = self.link_type[link-1] title = self.link_title[link-1] + try: # HACK fix byte vs str errors + title = title.encode('utf-8') + except (UnicodeDecodeError, AttributeError): + pass if (title == b"") or (parares.rfind(title.decode('utf-8')) < 0): title=parares[lstart:].encode('utf-8') if linktype == 'external' : @@ -579,6 +587,10 @@ linkpage = self.id[4:] if len(alt_title) >= len(title): title = alt_title + try: # HACK fix byte vs str errors + title = title.decode('utf-8') + except (UnicodeDecodeError, AttributeError): + pass if title != '' and linkpage != '': tocentry += title + '|' + linkpage + '\n' lstart = len(parares) @@ -779,6 +791,10 @@ if pclass[3:6] == b'h1-' : tag = 'h4' if pclass[3:6] == b'h2-' : tag = 'h5' if pclass[3:6] == b'h3-' : tag = 'h6' + try: # HACK fix byte vs str errors + pclass = pclass.encode('utf-8') + except (UnicodeDecodeError, AttributeError): + pass hlst.append('<' + tag + ' class="' + pclass.decode('utf-8') + '">') hlst.append(self.buildParagraph(pclass, pdesc, 'middle', regtype)) hlst.append('') ```
ableeker commented 3 years ago

Another user had noticed that this file wasn't completely converted to Python 3. I ran into the same error decrypting the ebook. That's why I have submitted pull request #1615 that has a fix for this particular issue. This fix changes line 562

title = title.rstrip(b'. ')

to

title = title.rstrip(b'. ').decode('utf-8')

After this change the ebook decrypted successfully.

@dtaylor84 could you try decrypting the ebook with this fix?

dtaylor84 commented 3 years ago

That partially fixes it, but it still fails with:

Running file type plugin DeDRM failed with traceback: Traceback (most recent call last): File "calibre_plugins.dedrm.init", line 538, in KindleMobiDecrypt File "calibre_plugins.dedrm.k4mobidedrm", line 238, in GetDecryptedBook File "calibre_plugins.dedrm.topazextract", line 372, in processBook File "calibre_plugins.dedrm.genbook", line 520, in generateBook File "calibre_plugins.dedrm.flatxml2html", line 806, in convert2HTML File "calibre_plugins.dedrm.flatxml2html", line 782, in process AttributeError: 'str' object has no attribute 'decode'

ableeker commented 3 years ago

@dtaylor84 Your ebook has found 4 lines of code with conversion issues, concerning pclass, and title. The content of these variables are read from the ebook, and should be byte arrays, and they are written out as strings. So it seems there are issues in the code that are caused by this conversion. Your hacks rather cleverly solve the issues, but it would be nice if we could find the underlying issue in the code.

The one ebook that I have that throws this error unfortunately doesn't hit line 782. However, looking at the code I noticed lines 773-776:

                            if orig_regtype.endswith(b'.right')     : pclass = 'cl-right'
                            elif orig_regtype.endswith(b'.center')  : pclass = 'cl-center'
                            elif orig_regtype.endswith(b'.left')    : pclass = 'cl-left'
                            elif orig_regtype.endswith(b'.justify') : pclass = 'cl-justify'

These lines create a pclass if there's isn't one, but here it's created as a string. So changing these lines to create byte arrays may fix the error in line 782:

                            if orig_regtype.endswith(b'.right')     : pclass = b'cl-right'
                            elif orig_regtype.endswith(b'.center')  : pclass = b'cl-center'
                            elif orig_regtype.endswith(b'.left')    : pclass = b'cl-left'
                            elif orig_regtype.endswith(b'.justify') : pclass = b'cl-justify'

Could you check if this change fixes the issue?

dtaylor84 commented 3 years ago

@ableeker Yes, with the addition of that change, Calibre imports & displays the book correctly.

ableeker commented 3 years ago

Thanks! I've added the fix to pull request #1615.