search

Sigil-Ebook / Sigil

Sigil is a multi-platform EPUB ebook editor
GNU General Public License v3.0
5.92k stars 578 forks source link

EPUBs not compatible with iBooks on iPhone #341

Closed daniel-j closed 6 years ago

daniel-j commented 6 years ago

UPDATE: I made a Sigil output plugin that fixes this: https://github.com/daniel-j/sigil-plugin-ibooksfix

iBooks on iPhone fails to render the TOC generated by Sigil properly. After some experimentation trying to isolate the issue I've found out that if you put all XHTML files in the OEBPS directory (next to the OPF metadata file), iBooks's TOC and syncing on iPhone works as expected. When a working EPUB is opened and saved in Sigil, it breaks, since Sigil puts all XHTML files in a separate directory.

Here is what happens in iBooks on iPhone when it fails to render the built-in TOC:

Steps to reproduce:

  1. Download Moby-Dick from IDPF EPUB3 samples.
  2. Add it to iBooks on macOS and sync it to the iPhone
  3. Open the downloaded ebook in Sigil, open content.opf and edit dc:title and dc:identifier to other values. iBooks on iPhone will think it's the same book otherwise.
  4. Save As and add it to iBooks like in step 2.
  5. Open the first ebook on the iPhone and open the TOC menu. Page numbers are visible and clicking items works.
  6. Open the other ebook that you saved with Sigil and open the TOC menu. No page numbers are visible and clicking any item opens the cover image.

Devices tested (by me):

Sigil version used: 0.9.9 (Linux and macOS)

iBooks on macOS and iPad does not have this issue (for me at least), but on iPhones it can be replicated 100% of the time. This seems to be an issue with Apple's iBooks. Having an option in Sigil to not generate a Text directory or keep directory structure unmodified would be nice. Until then, I'll make my own Sigil plugin or external script to convert already authored ebooks. You may close this as wontfix, but I leave this here for others with the same issue to find. I love Sigil! :D

Similar issues: #79 #80

kevinhendricks commented 6 years ago

This is definitely a "wont fix". Sigil produces as an epub structure that exactly meets the specifications for an epub from the IDPF. Historically, Apple IBooks seems to have had a hard time reading/understanding the epub2 and epub3 spec. They have broken spec in a number of ways over the years.

So this is definitely an Apple bug in iBooks for phones. Sigil builds a spec directory structure for epub. The problem is entirely in iBooks for not properly grokking the spec.

You should be filing your bug report with them.

Closing as this is not a bug in Sigil.

On Mar 12, 2018, at 10:55 AM, djazz notifications@github.com wrote:

iBooks on iPhone fails to render the TOC generated by Sigil properly. After some experimentation trying to isolate the issue I've found out that if you put all XHTML files in the OEBPS directory (next to the OPF metadata file), iBooks's TOC and syncing on iPhone works as expected. When a working EPUB is opened and saved in Sigil, it breaks, since Sigil puts all XHTML files in a separate directory.

Here is what happens in iBooks on iPhone when it fails to render the built-in TOC:

• The TOC shows no page numbers, even though it has scanned the book. Current page number is visible in reading mode as usual. • Syncing reading progress between devices does not work. • Clicking an item in the TOC loads the cover page. The only way to go to a specific chapter is to use the HTML TOC (if any). Steps to reproduce:

• Download Moby-Dick from IDPF EPUB3 samples. • Add it to iBooks on macOS and sync it to the iPhone • Open the downloaded ebook in Sigil, open content.opf and edit dc:title and dc:identifier to other values. iBooks on iPhone will think it's the same book otherwise. • Save As and add it to iBooks like in step 2. • Open the first ebook on the iPhone and open the TOC menu. Page numbers are visible and clicking items works. • Open the other ebook that you saved with Sigil and open the TOC menu. No page numbers are visible and clicking any item opens the cover image. Devices tested (by me):

• iPhone 8 with iOS 11.2.6, iBooks version unknown • MacBook Pro with macOS 10.13.3 High Sierra, iBooks v1.13 (1458) • Latest iPad with iOS 11.2.6, iBooks version unknown Sigil version used: 0.9.9 (Linux and macOS)

iBooks on macOS and iPad does not have this issue (for me at least), but on iPhones it can be replicated 100% of the time. This seems to be an issue with Apple's iBooks. Having an option in Sigil to not generate a Text directory or keep directory structure unmodified would be nice. Until then, I'll make my own Sigil plugin or external script to convert already authored ebooks. You may close this as wontfix, but I leave this here for others with the same issue to find. I love Sigil! :D

Similar issues: #79 #80

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

dougmassay commented 6 years ago

Your sample epub is an EPUB3. Is this issue limited to EPUB3, or does it happen with EPUB2 as well?

I ask because I'm wondering if merely moving the NAV doc (the built-in TOC in an EPUB3, as opposed to EPUB2's NCX) to the same level as the OPF would be enough (rather than all the xhtml files). If that's the case, an output plugin to accommodate iBooks' insufficiencies would be pretty simple.

daniel-j commented 6 years ago

I haven't tested EPUB2 as much, I only do EPUB3 (that are backwards-compatible), but I can give it a try. I haven't had the same issue with EPUB2 with NCX, so I think it's only for EPUB3 ebooks in iBooks.

kevinhendricks commented 6 years ago

Also, does iBooks on iPhone for the TOC use the epub3 nav or the ncx if one is available? When you load an epub3 into Sigil, it creates a template toc.ncx file so that the epub can be used on epub2 only readers as well.

This toc.ncx needs to be manually updated from the Nav in epub3.

So in your step 3 above, before saving the epub in Sigil, please run: Tools->Epub3 Tools-> Generate NCX from Nav

This will fill in that template ncx file in case it is actually being used by iBooks for iPhone in place of the standard epub3 nav document.

daniel-j commented 6 years ago

iBooks uses the HTML nav TOC in EPUB3 ebooks and NCX in EPUB2 ebooks. Modifying/removing the NCX in an EPUB3 ebook has no effect.

kevinhendricks commented 6 years ago

You have verified that on iBooks for iPhone?

daniel-j commented 6 years ago

Yes.

daniel-j commented 6 years ago

I ask because I'm wondering if merely moving the NAV doc (the built-in TOC in an EPUB3, as opposed to EPUB2's NCX) to the same level as the OPF would be enough (rather than all the xhtml files). If that's the case, an output plugin to accommodate iBooks' insufficiencies would be pretty simple.

You are right! Just moving the TOC XHTML file to the OEBPS dir works too! I just tested it.

kevinhendricks commented 6 years ago

Then the bug is purely in iBooks for iPhone. Our epub3 Nav document meets the exact spec. But to make an epub3 be backwards compatible for epub2 only readers it definitely helps to add an ncx and properly fill it in so that it matches the nav exactly.

dougmassay commented 6 years ago

Is the vestigial ncx still in these epubs? Just wondering if eliminating that would have any effect? I have an output plugin that removes the ncx (both the file and it's OPF entries).

daniel-j commented 6 years ago

iBooks does not use the NCX file at all in EPUB3 ebooks. The TOC shown is from the XHTML TOC only. Removing it and its references has no effect.

daniel-j commented 6 years ago

For completeness sake, here are some screenshots. First image is with iBooks' broken rendering (missing page numbers, clicking items opens cover page) and the second one is a "fixed" epub. img_0002 img_0001

kevinhendricks commented 6 years ago

Your "page numbers" are confusing. There are no "page numbers" in your moby-dick.epub. There is no epub:type="page-list" in the Nav at all in that epub freshly downloaded before Sigil is ever opened.

So are these "pages" something like the horrible Kindle style standard size blocks of nonsense pages, or actual Nav with a real page-list section?

dougmassay commented 6 years ago

iBooks does not use the NCX file at all in EPUB3 ebooks. The TOC shown is from the XHTML TOC only. Removing it and its references has no effect.

Thanks for checking. Sounds like an output plugin (to be used after all Sigil edits are final) that moves and rejiggers the NAV would be the way to work around this iBooks problem.

daniel-j commented 6 years ago

These page numbers are generated dynamically by iBooks and is calculated after the number of pageturns you have to do to finish the book.

kevinhendricks commented 6 years ago

Thanks for clarifying that. Basically those numbers are nonsense just like on the Kindle as a "page number" on one iphone device will be different to an ipad, Desktop, or other iphone with a different screen size and/or font sized used. I thought you were referring to a set of real page numbers generated from a specific print edition which can be added as a separate page-list section in the Nav. I guess I just spend too much time with academic epubs where real page numbers are needed and used for academic citations.

I agree, a short plugin could easily be built to walk the opf manifest checking out the manifest properties to properly identify the name of the "nav" and then moving that file up to sit beside the content.opf, changing its manifest href entry to match and then walk each and every anchor in that nav document changing the ".../Text" or "../Image" to remove the "../" for the proper href and for all landmarks and page-list sections as well.

kevinhendricks commented 6 years ago

You may also have to adjust links to external stylesheets sometimes used in the Nav as well.

daniel-j commented 6 years ago

Made a Sigil output plugin, works nicely! https://github.com/daniel-j/sigil-plugin-ibooksfix It rewrites all hrefs in all xhtml files pointing to the toc, updates the ncx navPoint if any, updates the EPUB2 guide element if any, and updates all href/src attributes in the xhtml toc itself. Tried it with my existing Sigil-authored ebooks and they all validate with epubcheck and the toc works on iOS iBooks!

dougmassay commented 6 years ago

Would you be interested in publishing your plugin on MobileRead (you'd need to create a free account) where the bulk of third-party plugins are discussed? There's a dedicated subforum for plugins and an index for users to find them. That's the where most Sigil users go for Sigil/plugin support.

https://www.mobileread.com/forums/forumdisplay.php?f=268

daniel-j commented 6 years ago

Yes I have planned to do that, will do when I get time.

pekingli commented 6 years ago

My OS is Mac High Sierra I tried your plugin but get the following error message:

objc[32282]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fffa83e4c90) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x112937cd8). One of the two will be used. Which one is undefined.

dougmassay commented 6 years ago

Please ask for help with this 3rd-party plugin on it's GitHub repository. This issue tracking database is for Sigil problems.

kevinhendricks commented 6 years ago

That is a notification, not an error. You can simply ignore it.

On May 29, 2018, at 3:37 PM, pekingli notifications@github.com wrote:

I tried your plugin but get the following error message:

objc[32282]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fffa83e4c90) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x112937cd8). One of the two will be used. Which one is undefined.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.