What is the pagination strategy in Readium-2?

rkwright commented 8 years ago

This is a Discussion

What is the pagination strategy?
CSS columns have several problems
Per platform or cross-platform?

rkwright commented 8 years ago

From Jean-Marie: Probably per-platform. On iOS it would be great to get some insurance from Apple that the paginationMode that has become private in WkWebview (but can be used) will not disappear… Is this realistic?

On Android the situation is more complex ;-)

rkwright commented 8 years ago

From Jean-Marie:

Here is the code snippet that does the trick:

<body style="overflow:-webkit-paged-x; overflow:paged-x;">

It flies!!! Tested on huge pages on Android devices. Instant display, blazing-fast horizontal scroll.

It was implemented in 2012, I didn't know that. https://trac.webkit.org/changeset/126343

rkwright commented 8 years ago

From Juan Corona: I'm for the the per-platform approach. But one of my concerns is with the increased overhead supporting multiple strategies for serializing/deserializing page locations, figuring out visible page ranges, and all the related code we wrote in the cfi_navigation_logic monolith. Although if we can keep things modular and with well designed interfaces I can see this burden being eased.

rkwright commented 8 years ago

From Jean-Marie:

seems to work like a charm (very fast) on Android Chromium (available since 4.4 and even before for those who embed a Chromium-based WebView) and on iOS too. Any known drawback? My first tests cumulating this pagination and multi columns (are not very positive (it works but thare are small scrolling issues on iOS and viewport issues on Android), but we can dig [into them] if nobody already has negative conclusions. Edit: I see that this option was discussed back in 2014...

rkwright commented 8 years ago

From Juan Corona: I wasn't part of the discussion back in 2014, but if this is a viable strategy for Android then it sounds great.

rkwright commented 8 years ago

From Juan Corona: CSS scroll snap points provide a nice scrolling experience, as if flipping through pages. https://www.chromestatus.com/feature/5721832506261504 It’s supported everywhere but Chrome and IE (but is shipped in Edge!) Demo (vertical, but can be horizontal): https://blog.hospodarets.com/demos/scroll-snap-full-screen

rkwright commented 8 years ago

Using overflow-paged sounds very promising. However, to be viable it needs to work on all platforms we support (if at all possible). It is much more complex to have one solution for MOST platforms and one or two others on other platforms. We need some research into how works across different platforms and versions. Also, is this a feature that the browser vendors will continue to support or will they drop it just as we finish R2?

dauwhe commented 8 years ago

I've made little demos using overflow: -webkit-paged-x with scroll snapping. It's a lovely experience, requiring three lines of CSS:

overflow: -webkit-paged-x;
-webkit-scroll-snap-type: mandatory;
-webkit-scroll-snap-points-x: repeat(100%);

But AFAIK it's only working publicly in Desktop Safari. Sadly, I haven't heard of interest from other browsers for this.

rkwright commented 8 years ago

@dauwhe Can you share your demos?

dauwhe commented 8 years ago

http://dauwhe.github.io/epub-zero/acme-publishing/MobyDickPaged/

jmgeffroy commented 8 years ago

Right. Theoretically it should work on all webkit-based webviews... It seems to be present at least since 2012 in WebKit (See this description). We should probably define a fallback strategy however, for browsers that don't support it or don't support it well... PS: - Just for fun - While doing some random "googling" on this topic I found a funny paper to read later... They transform the page with XSLT to break the page into pages, so preserving the CFIs with this technique is probably a little bit tricky ;-) But this is probably an entry point to the world of of pagination, and the References section contains some interesting stuff (...for insomniacs ;-))

rkwright commented 8 years ago

@dauwhe Thanks. I see the pagination layout but how is the snap-navigation supposed to work?Seems to simply lay out the "pages" then you have to manually move the viewport so a given page is in view. Tested in Safari 10.0 on OSX.

jmgeffroy commented 8 years ago

@dauwhe According to my tests, the paged-x works fine on my Android Webkit-based webviews and on iOS. Snap points don't.

dauwhe commented 8 years ago

@rkwright if I do the horizontal two-finger gesture on my trackpad, the "page" slides and then sticks to the snap point. It's very much like page-swipe gestures in many EPUB reading systems.

One bad thing is that, at least in my file, when I go to a page via a TOC link, it doesn't automatically snap to the page boundary. I have to nudge it.

HadrienGardeur commented 8 years ago

Works fine for me on macOS with Safari, no snap points with Chrome though.

rkwright commented 8 years ago

@dauwhe Ah so. Yes, that works. But clearly there are a bunch of wrinkles to this approach. But if it can be made to work it would be a great boon. And after all, WebKit is open source.

rkwright commented 8 years ago

Shock! It doesn't work in Edge. And last time I looked, Edge wasn't open source...

jmgeffroy commented 8 years ago

I thought it was, look :-D

jmgeffroy commented 8 years ago

Interesting reference about Edge

rkwright commented 8 years ago

@jmgeffroy Huh. I hadn't stumbled on that. But I don't see the source code. Still, interesting.

jmgeffroy commented 8 years ago

@rkwright The source code isn't there but it contains interesting stuff and pointers. It's on this Github repo that I found the link to the Edge Platform Status page, for example. It's also probably interesting to look at the tests. It's reading the Webkit tests that I discovered the overflow: paged-x, for example. At least, it's easier to read than the complete source code in a first approach...

llemeurfr commented 8 years ago

At the W3C TPAC, CSS Houdini was cited as a possible great solution for our pagination problem. It would be a DIY solution, where we create a pagination CSS polyfill in JS (more that a polyfill, a real new thing).

More specifically, I was advised by A.Stearns (Adobe), co-chair of the CSSWG, that:

readium architecture folks should consult https://drafts.css-houdini.org/css-typed-om/ and http://wicg.github.io/CSS-Parser-API/ and give feedback on whether they would be useful to you.

The new liaison between the publishing industry and the W3C could help us getting more from the browser vendors.

rkwright commented 8 years ago

Of course, there is also: http://sorotokin.com/adaptive-layout/ The CSS working group reviewed it several years ago. Some liked it, some didn't. But it is very nice, very full-featured and works.

dauwhe commented 8 years ago

@rkwright Vivliostyle has used Peter's work as the basis for their stuff. https://github.com/vivliostyle

llemeurfr commented 8 years ago

Note that I met Florian Rivoal from Vivliostyle at TPAC, we talked about their work, achieved from sorotokin's work. They have some open-source software they could share.

olivierkorner commented 8 years ago

FYI the CSS paged-x property doesn't seem to work in Safari on iOS (version 10).

HadrienGardeur commented 8 years ago

You should always include both paged-x and -webkit-paged-x in your test.

olivierkorner commented 8 years ago

Well support looks buggy: I was indeed able on iOS to "page" some content by adding the CSS to a simple HTML page, but @dauwhe's example displays a very long iframe on my iPad: paged-x-bug-ipad-ios10-screenshot

HadrienGardeur commented 8 years ago

I've also created a demo for paged-x: https://hadriengardeur.github.io/webpub-manifest/examples/pagination/overflow-paged-x/

Just a few notes about the demo:

it detects clicks on left/right on desktop and Chrome Android in order to navigate to the next page
this is still buggy on mobile Safari (page turn), will need some extra work from me
since taps do not work yet in mobile Safari, scrolling is still allowed in this demo

My main concern regarding -webkit-paged-x is that I can't find a single way to set a top and bottom margin on overflowed content (page 2+).

jmgeffroy commented 8 years ago

Top and bottom margin can easily be outside the overflowed area. 3 obvious "pros" of this solution wrt multi-column are simplicity, speed and the fact that it doesn't break multi-columned layouts. Note that I'm still studying other solutions, so I'm not pushing it as the only way to go. Peter Sorotokin's adaptive layout is impressive (as always) but I don't know how fast it can be. BTW more generally I think that pagination strategy is probably one of the areas that should be normalized as much as possible across all implementations and integrations if we still want to fulfill the promise of "providing a unified Epub 3 rendering"...

HadrienGardeur commented 8 years ago

@jmgeffroy that means introducing a synthetic margin (top/bottom) that won't respect the publication's CSS. Sure that works, but it's slightly inconsistent with the author's intent.

BTW more generally I think that pagination strategy is probably one of the areas that should be normalized as much as possible across all implementations and integrations if we still want to fulfill the promise of "providing a unified Epub 3 rendering"...

Do you mean normalized across apps on the same platform or normalized across different platforms?

rkwright commented 8 years ago

Peter's adaptive layout was (is) very performant. The more serious problem was that it had to essentially take over CSS layout from the browser. Even so it was performant (Peter at his best) but IIRC it did interfere with accessibility (or at least was never optimized to support accessibility properly).

My larger concern in this area (and with paged-x in particular) is having a consistent solution across the supported platforms.

HadrienGardeur commented 8 years ago

I've created a more advanced demo (limited to desktop Chrome/Safari/Opera and Chrome on Android for now) that does the following:

pagination both within an HTML and between documents
offline through a Service Worker
three tap areas (previous page, menu and next page)

It's entirely based on "overflow: paged-x" with very little JS/CSS injected in the iframe.

It's available at https://hadriengardeur.github.io/webpub-manifest/examples/paged-viewer

HadrienGardeur commented 8 years ago

Just a quick update based on something that @rkwright said yesterday and that @olivierkorner mentioned earlier in this thread:

"overflow: paged-x" works fine if applied to a document directly on iPad, you can try it at https://hadriengardeur.github.io/webpub-manifest/examples/pagination/overflow-paged-x/
but when applied on an iframe, the content is not properly paginated: the width is stretched to the proper size, but the text is not divided into pages/columns as you can see on @dauwhe example (https://dauwhe.github.io/epub-zero/acme/main.html?manifest=MobyDick/manifest.json) or my recent experiment (https://hadriengardeur.github.io/webpub-manifest/examples/paged-viewer)

Overall, this means that while "overflow: paged-x" works really quite well on Webkit-based browsers on either desktop (Chrome, Opera, Safari) or on Android (Chrome), it's not supported at all in Edge/Firefox and doesn't behave properly in an iframe on iOS.

In order to continue these tests with various techniques for pagination, I'd like to set up a test environment that would have the following components:

a basic viewer written in JS (I've been using https://github.com/HadrienGardeur/webpub-viewer) with the ability to quickly switch between pagination modules
a set of pagination modules, each module has one JS and one CSS that is injected in the content document
a collection of EPUB files that we can use to test these modules (should we use the standard EPUB3 test documents?)
a server component that will serve the resources and the manifest of these EPUB documents in HTTP/HTTPS

I've also listed the following things that IMO have an impact on pagination and aren't mentioned as often in our discussions:

Do we use an iframe? Sandboxed?
Can we inject the JS+CSS directly in the content document without using an iframe?
What's the impact on author and user stylesheets when we inject such JS/CSS?
How does the accessibility options of each platform behave?
How do we capture the current position in a publication? How do we navigate back to this location? (This should also cover the use cases for annotations and search)

There are also a number of pagination solutions for which we can't do the implementation strictly in JS: using multiple webviews, private APIs on WKWebview etc. Should we also test some of these solutions? Do we have a good idea how various RS actually implement pagination (iBooks, Kindle, Kobo, Google, Scribd...)?

winniequinn commented 8 years ago

I'm just starting to play with the idea of not using iframes at all. They cause issues with paged-x (at least on mobile), they can get in the way when using VoiceOver or TalkBack (as the browser sees the iframe as another container to be navigated into), et cetera.

The whole point of using an iframe, at least as far as I know, is to be able to maintain an outer container for the reading system with the iframe being responsible for holding the contents of the book. This certainly makes sense if the goal is to be able to move between different chapters of a book all within the browser as is presently the case with Readium.

At least on mobile though, I think another solution is possible. We could get rid of the iframe, make the browser responsible only for showing a single chapter, and then load up other chapters in different web views that are then swapped into view as appropriate by the application-layer code written in Java, Swift, et cetera. This approach would seem to eliminate the need for an iframe, would make precaching adjacent spine items very simple, and would allow more work to be lifted out of the land of JavaScript and brought into the application layer.

It may well be possible to use the same approach on the desktop as well. Electron has a web view component that presumably could be used to the same effect as WKWebKit and android.webkit.WebView. I need to look into this a bit further.

rkwright commented 8 years ago

@winniequinn This is not a bad idea (at all). Iframes are certainly not ideal. However, trying to put separate spine items in separate containers has been tried more than once. See the prefetch branch in shared-js. One of the major problems encountered is NOT playing audio and video (as well as scripting) before the web-view is "brought into view". Both EvidentPoint and Bluefire experimented with it, but were unable to resolve it satisfactorily. But YMMV... It WOULD be a good approach if the problems can be resolved.

HadrienGardeur commented 8 years ago

This feels like two separate discussions:

what the reader/container is and how we inject CSS/JS
how we handle navigation between spine items

I'm also in favour of either dropping the iframe or offering an alternative, as I've said before in https://github.com/readium/readium-2/issues/12#issuecomment-255015051

How we actually handle different spine items is another question, probably worth having its own issue here.

danielweck commented 8 years ago

@winniequinn this is how other "i"-named reading systems do it.

In Readium-1 we have reader.html that effectively bootstraps the entire readium-shared-js subsystem (rendering engine), which lays out the EPUB content iframe hosts, and is responsible for injecting behaviours into spine items (Media Overlays, link hijacking, keyboard / input device intercept and forwarding, CSS styling, epubReadingSystem, MathJax, etc.).

Using native webviews instead of iframes would mean writing a large portion of the rendering code in native per-platform code, which I think is acceptable in the context of Readium-2.

Note that we did look into using Electron's special web frame, but this is basically an extended iframe with special features (still in the renderer process though).

danielweck commented 8 years ago

Continued iframe discussion: https://github.com/readium/readium-2/issues/15

HadrienGardeur commented 8 years ago

During our weekly-call, @JCCR volunteered to create a prototype based on CSS-columns in order to continue to explore and compare various options for pagination.

jccr commented 7 years ago

Added the example for CSS-columns on develop: https://github.com/readium/readium-2/tree/develop/examples/pagination/columns

I see that @HadrienGardeur did the same for overflow: paged-x: https://github.com/readium/readium-2/tree/develop/examples/pagination/overflow-paged-x

HadrienGardeur commented 7 years ago

Thanks @JCCR that's very helpful.

I'll try to adapt my WebPub Viewer code as soon as possible to provide the ability to quickly switch between pagination modes. I'd like to roll out a full test suite where we can test a number of EPUB live and switch between different pagination modes.

dauwhe commented 7 years ago

Note I filed a bug on the webkit iframe issue: https://bugs.webkit.org/show_bug.cgi?id=165304

HadrienGardeur commented 7 years ago

@dauwhe thanks Dave !

In other news, I'd like to discuss the APIs available for the pagination module in the next call. No matter what we decide to use in order to paginate, we could have a number of APIs that work across all implementations (even for native code).

Let's start with some basic stuff that we can all agree on and expand from there.

danielweck commented 7 years ago

@HadrienGardeur started that page: https://github.com/readium/readium-2/tree/master/doc/pagination

bluefirepatrick commented 7 years ago

The location of the document seems to have changed. This works for me: https://github.com/readium/readium-2/tree/master/pagination

JayPanoz commented 7 years ago

For the (track) record, explored overflow: paged-x as soon as we started working on pagination for Readium CSS and it was quite unreliable. For some reason, blink’s and webkit’s implementations differ a lot, with Blink being more complete.

There are simply too many issues at the moment:

background-color doesn’t work in Safari if the element has overflow: paged-x declared, which is problematic for reading modes, while it works in Chrome;
Webkit doesn’t normalize scrollWidth while Chromes does. So only chrome sets the scrollWidth of the element as a multiple of the viewport width;
margins and paddings are completely screwed up in both implementations (i.e. they’re not based on page containers);
you get text cut-off in some cases;
you still need columns to emulate two-page spread;
I had to resolve to left: value to make it scroll everywhere, scrollLeft didn’t work with columns for some reason.

It is also my understanding that Opera’s intent was to use it to solve a lot of the column-box model issues and limitations (padding, margin, overflow, position: fixed|absolute etc.). I’m taking the Presto’s implementation (Opera Reader) as a reference there since it was feature complete.

So they used it with columns anyway, it was not designed as a replacement but more like a sanitizer for paged media.

HadrienGardeur commented 7 years ago

By the way @JayPanoz which page should we reference officially for the pagination strategy in Readium CSS?

JayPanoz commented 7 years ago

@HadrienGardeur Well, stylesheets are currently in branches (mapped to the project’s phases) as I can’t necessarily PR parts of the CSS as they come along e.g. pagination impacts user settings and vice versa. Besides, we still have to deal with scroll at the moment, which once again is impacting/impacted by user settings.

In other words, I’ll be able to PR when we have a fundamental system designed and then improve details.

But I can edit the Readium CSS wiki for reference in the meantime, especially as we now know how we’ll inject contents. That was on my TODO list and I’ll have to wait for basic user settings to be implemented in the iOS proto anyway so I can prioritize that tomorrow.

BTW, which would be the best repo to discuss the iOS proto issue about odd number of columns when two-column view? I may have an idea, which could also save us a lot of layout trashing, but Daniel could probably give a lot of insights about it as it would impact CFI at first sight.

JayPanoz commented 7 years ago

@HadrienGardeur So I guess you can reference this Wiki page as it sums up injection and pagination approaches.

I’ll add references to technical docs as soon as we push the stylesheets on master.

readium / architecture

What is the pagination strategy in Readium-2? #10

This is a Discussion