Apostrophe in title got converted to weird unicode

[p1nkyy]

Hello, sometimes, a book title that has apostrophe in the title got converted into weird unicode (’) so for example: The Bookseller's Apprentice becomes The Bookseller’s Apprentice. The folder name becomes The BookselleraEUTMs Apprentice - Amelia Mellor. Is this due to Calibre, ADE, the plugin, Windows? Haven't figured out the situation that caused this. Should I send a log when re-processing these titles?

Attached is some screenshot:

[Leseratte10]

Does this happen with all books that have an apostrophe in their name, or only with some?

My first guess would be that that's an issue with the book shop / whoever generates the ACSM file or the ebook. Can you post a debug log of an attempt that creates such a file?

[p1nkyy]

Only some and very random and only started recently (September). OK, will try to send one if I encounter this again.

[p1nkyy]

log 4.txt

Hi, finally it has happened again and here's the log for you. Hope it's useful!

[Leseratte10]

Can you re-do that log with the verbose log setting enabled if you still have the ACSM file? Do you happen to have an install of ADE with the same AdobeID to check if the same error occurs when downloading with ADE?

[p1nkyy]

Sorry for being silly but how do I enable the verbose log? No I have long uninstalled ADE because it's practically unusable. Should I send you this acsm so you can process it? (Clean re-borrowed ACSM that I haven't processed so it won't say already fulfilled)

[JJTech0130]

Sorry for being silly but how do I enable the verbose log?

Open Calibre, then click "Preferences" -> "Restart in debug mode". Then Calibre will restart, then perform the action that causes the error (probably adding the ACSM file), then close Calibre. A text file should then be opened with some more verbose logs.

[p1nkyy]

That was what I did with the log above. If just opening & closing it again then here it is.

log 5.txt

[Leseratte10]

That's just the "normal" log. You can go into the plugin settings in Calibre and check the "Enable verbose debug logging" box and then do the same steps again.

[p1nkyy]

Is it here? If yes, don't see any option to enable verbose debug logging

[mwsundberg]

I have the same issue and can send you an example file, the key/config used for the DeACSM plugin, and a verbose debug log. I'd prefer to share those elsewhere and not publicly upload the files, may I email you? It's also not reading the metadata, yet I don't know if it's an issue with my library's files or the plugin/Calibre. I'm using deACSM v0.0.16 downloaded from the Releases tab, DeDRM 10.0.3 from noDRM's fork, and Calibre 6.7 on Windows 10. For the ADE account I'm using an auto-generated anonymous one.

[p1nkyy]

Yeah, I would like to corresponds via e-mail instead for some of these questions but he was hesitant to share his e-mail, hopefully he'll let us know soon. With yours @mwsundberg is it all titles that has apostrophe converted into unicode or just randomly like mine?

[Leseratte10]

I don't really like publicly sharing my main email (hello bot spam!) so here's a dedicated one for this issue that I can delete again once the issue is solved:

<e-mail deleted, issue is solved>

You can send verbose log files (or even whole authorizations and/or ACSM files if you're comfortable with that) to this address and I'll take a look.

[Leseratte10]

Thanks for your email @mwsundberg , to me this issue looks like it's a bug either in Calibre (more likely) or in the book itself (less likely), not in my plugin. I took the book EPUB file that comes straight from the yourcloudlibrary.com server before it was processed by my plugin, removed all the actual content and data (so it's legal to share now) and just kept title and author as-is.

When I import that book into Calibre I get the same weird book title as you - I attached it here for reference.

ef3a1633-95ce-4752-9c45-6c6905ac178c.epub.zip

Probably because the title uses one of these weird, non-ASCII, Unicode apostrophes - ’ instead of the normal one '. Though, I'm surprised Calibre doesn't like these. I've ran the resulting EPUB through a validation tool and it didn't complain about the book name. Other EPUB reading tools also display the book title correctly.

I have opened a Calibre bug report regarding this issue; that bug is not something that I can fix in my plugin. You can verify that by checking that EPUB I linked with other readers (correct title) and importing it into Calibre (wrong title).

Is it here? If yes, don't see any option to enable verbose debug logging

Also, in case someone has the same question and finds this issue in the future: The verbose log setting is not a global Calibre setting, it's a setting specifically for this plugin, so it'll be under Preferences -> Plugins -> File type -> ACSM Input / DeACSM -> Customize plugin -> Enable verbose debug logging.

[mwsundberg]

Cool, thanks for the quick follow-up and debugging a problem that ultimately had nothing to do with your plugin.

[p1nkyy]

Wow after waiting for quite some time and it's resolved finally haha. OK I don't need to send in my ACSM then. Thanks for the answers @Leseratte10 and thanks for your input @mwsundberg

[Leseratte10]

@p1nkyy I checked your email / ACSM as well and it seems to be the exact same issue with the unicode apostrophe, so once that Calibre bug is fixed the issue should be gone.

[Leseratte10]

The bug in Calibre has now been fixed with kovidgoyal/calibre@ad34b0ea3b2d0ba514e93d17b060a3d9af1247eb and I confirmed that both EPUB files now display the apostrophe correctly with that version. The bugfix will be included in the next Calibre update which I assume will happen in two weeks. I'm going to close this issue for now.

Thanks to @p1nkyy and @mwsundberg for providing test files and logs.