Flexbox in CSS prevents EPUB from opening on some e-readers

[denisftw]

It seems that in this change the embedded CSS was modernized to use flexbox. However, it makes EPUB unreadable on some old e-readers, i.e on my PocketBook Pro 912 (with latest firmware). I'm not sure what was the motivation behind this change, but I don't see any attempts to use flexbox in EPUB-books produced by professional book publishers, i.e O'Reilly or No Starch. Not surprisingly, I can easily open their books but not the ones produced with pandoc.

Maybe it's worth reconsidering giving that not opening the book is already pretty bad. Alternatively, I would appreciate if there was a way to alter the inline CSS during generation.

Using 2.19.2 of pandoc on vanilla Arch:

$ pandoc -s -t epub2 --metadata title="EPUB Test" test.md -o test.epub

where test.md consists of one paragraph of random text without any Markdown-markup.

[jgm]

@denisftw have you tried using --css to pass in custom CSS and override these?

I'm surprised that the reader fails to open the epub at all just because of some CSS it can't interpret!

@tarleb any thoughts? I note that data/epub.css (which is used by default if they don't use --css) includes some things that duplicate what is added in styles.html, including the old-style declaration for div.column! We should remove those duplications. Or, maybe we should add in the old-style declarations for div.column and div.columns, to override the flexbox stuff for EPUBs, given the difficulties older readers have with flexbox.

[jgm]

@denisftw - could you try using --css epub.css with the following epub.css?

/* This defines styles and classes used in the book */
body { margin: 5%; text-align: justify; font-size: medium; }
h1, h2, h3, h4, h5, h6 { text-align: left; }
/* For title, author, and date on the cover page */
h1.title { }
p.author { }
p.date { }
nav#toc ol,
nav#landmarks ol { padding: 0; margin-left: 1em; }
nav#toc ol li,
nav#landmarks ol li { list-style-type: none; margin: 0; padding: 0; }
a.footnote-ref { vertical-align: super; }
em, em em em, em em em em em { font-style: italic;}
em em, em em em em { font-style: normal; }
q { quotes: "“" "”" "‘" "’"; }
/* We don't use flexbox for columns due to lack of support in older readers. */
/* This means that only two columns are supported.  Removing the following */
/* two lines will cause flexbox to be used for columns, allowing more than */ 
/* two columns at the expense of support for older readers. */
div.columns{ display: block; }
div.column{ display: inline-block; vertical-align: top; width: 50%; }

@media screen { /* Workaround for iBooks issue; see #6242 */
  .sourceCode {
    overflow: visible !important;
    white-space: pre-wrap !important;
  }
}

[tarleb]

I fail to reproduce this with pandoc 2.19.2. This was a bug in 2.19.1, but it is fixed in the latest release as far as I can see.

[tarleb]

Scratch that, I do see this.

[tarleb]

Minor side note: my goal with #8275 is to make it possible to work around such an issue with a custom writer:

local zip = pandoc.zip

function ByteStringWriter (doc, opts)
  local epub = pandoc.write(doc, 'epub3', opts)
  local archive = zip.toarchive(epub)
  archive.entries:map(function (entry)
      return zip.Entry(
        entry.path,
        entry:contents():gsub('\n%s*div.columns?%b{}', '')
      )
  end)
  return archive:tobytestring()
end

[jgm]

If the --css option described above doesn't work, another option would be using a custom template that replaces the styles.html partial.

In the end, though, we need to consider whether we should be producing output by default that will work on a wider variety of readers.

[denisftw]

Thanks for a quick response!

I tried overriding the styles with -css option, but it didn't help me. Apparently the reader trips at the moment it sees something flex-related and not when it consolidates styles. If I open the file with Sigil and manually replace the two inline CSS lines with styles from your file:

div.columns{ display: block; }
div.column{ display: inline-block; vertical-align: top; width: 50%; }

then it works.

Changing templates also works, and indeed, I was able to produce the desired EPUB by specifying a different template. The easiest option was to simply put an adjusted (as above) version of styles.html into ~/.pandoc/templates. In this case, the resulting EPUB will have the updated styles and I don't even need to do anything special in command line :)

Thanks for this suggestion!

[tarleb]

The only fix I can think of is to introduce a css2 variable and to use that to switch between the two versions for column handling. The variable would be set when converting to HTML4.

[denisftw]

I think that any sort of command-line flag would be fine as long as it allows to fix behavior locally without the need for updating user or system-wide settings. Even a flag to completely disable inline CSS would be fine (Leanpub, for example, doesn't embed CSS); it just means that the user needs to ensure that the result looks decent.

[jgm]

@denisftw are you targeting EPUB v2 or v3?

[jgm]

We don't want to completely disable inline CSS, as we rely on it for some formatting to appear correctly. But @tarleb's solution seems a good one if it is okay to use flexbox when targeting EPUB v3 (which uses XHTML5).

[denisftw]

@jgm Yeah, I also thought about making a distinction between EPUB v2 and v3, but surprisingly, in my case, it didn't matter. Without flexbox, the reader can open either v2 or v3, but if flexbox is used, it cannot open any.

In my view, having an option to skip inline CSS altogether would be a pretty useful feature by itself even if it didn't address any real problem, and I would expect that someone might ask for it in the future.

[jgm]

One option would be to remove the $styles.html()$ partial from the EPUB templates and put the necessary CSS in epub.css. This would make it easier to override -- you'd just need to supply your own --css. There's really no good reason to use inline CSS in EPUB.

The drawback would be that if we add new things to styles.html(), perhaps to handle new things the HTML writer is doing, we'd have to remember them to port them to the default epub.css. [EDIT: I suppose we could even replace epub.css with a CSS template that includes styles.html().]

[tarleb]

That seems like a better approach than the CSS2 variable.

[jgm]

Looking at this more carefully, here's another drawback of this approach:

Currently the styles.html() contains a bunch of variables for things like line-height, font-size, margin. These would no longer work if we just put an instantiation of this template into epub.css.

Options:

1. make epub.css into a template, as mentioned above, so that it can contain variables that are replaced
2. create a separate styles.epub to be used in the epub templates; this could be more trimmed-down, just including the things that use variables, and everything else could be put into epub.css
3. just don't worry about the variables; just create a plausible epub.css and let people make the changes there

Thoughts?

[jgm]

Leaning towards 3.

[jgm]

OK, I've made a stab at 3. I separated out the citeproc-specific CSS, and we still use an inline style for this, but only on the page that requires it. This needs to be in a template partial, because the CSS depends on variables set by the writer.

Try this out and let me know if we need further changes.

I also realized that our previous setup would include two copies of the highlighting CSS!

[denisftw]

Do I need to re-install something or I can simply replace my files at /usr/share/pandoc with the new ones?

[jgm]

Not sure -- it might work just to replace these, so you can try that first. Alternatively, use a nightly build, which has data files baked into the executable; this won't interfere with your existing installation.

[denisftw]

@jgm Just checked it using the nightly build.

Absolutely brilliant work, thank you very much!

[jnaklaas]

I'm stuck trying to fix this. The --css option doesn't fix it. How can I install the nightly build? I'm having this issue on mac osx with pandoc 2.19.2 homebrew install.

[tarleb]

You can download a nightly from https://github.com/jgm/pandoc/actions/runs/3562784440 (downloads are at the bottom of that page).

[gevhaz]

Thanks for your work. But are you sure this fixed it? I've compiled from the master branch as of today and still see the same problem. This line exists in the css if I unzip my epub so I think I've installed it right.

Using --css with the style sheet you provided works though.

Edit: Using a Kobo Libra H20, if it helps. I only tried the --css flag with my build from master.

[jgm]

@gevhaz thanks for testing. Can you try with a nightly instead of building yourself from source? That will give us one more data point.

[gevhaz]

Tested with the Linux artifact for the last nightly (https://github.com/jgm/pandoc/actions/runs/3811239175). Still getting the same issue.

[jgm]

@gevhaz would you be able to provide the epub you've built so we can inspect it? Or a simpler one that reproduces the issue on your reader?

[gevhaz]

Sure! Here is a link to it: https://tempfile.io/en/i0mNXgixowuffRV/file I wish GitHub would allow me to attach it here but this is the best I can do.

I created bad-ebook.md with the content "test". Then I convert it to epub like so:

~/Downloads/pandoc-nightly/pandoc-nightly-linux-2022-12-31/pandoc -f markdown -t epub -o bad-ebook.epub bad-ebook.md --metadata title="test title"

I sent it to my Kobo Libra H2O. If I try to open it, I get a blank page (no "test" text and no title page). If I try to go to the next page, the book closes as if I had pressed "next" from the last page and the book is marked as finished.

[denisftw]

When I was testing this, I actually looked inside the generated EPUB using Sigil.

I just downloaded the latest pandoc nightly build and it still works perfectly fine with the --css option.

I do see flex in the generated default CSS, but wasn't this the intention? Use it by default but allow the user to override the default styles?

[gevhaz]

Using --css and providing the css that @jgm posted earlier, everything works fine for me.

Like you say, the issue is probably that there is flex stuff in the CSS included in the fix. This line from the fix seems to indicate that you can choose between those two options, but I'm not sure how that's meant to work since later statements just overwrite earlier statements in CSS. Maybe the reader would ignore the line it can't read? But it doesn't work on my reader (updated 2022 for the record).

I don't know how Kobo messed up interpreting flex stuff, but I do think that styling should be left as much as possible to the reading software. The point of epub is to hold the content without enforcing the way it is presented like in PDF.

[jgm]

I think the intent was that those lines should be commented out! I'll fix.

[gevhaz]

For the record, pandoc-produced epubs work fine on Kobo Libra H2O with no overriding of CSS with --css now. Thanks @jgm!