Sound effects not playing and related animations not functioning as expected

[terrillt]

General Instructions Sound effects not playing and related animations not functioning as expected

Reproducibility Rate: 100%

This issue is a Bug

Expected Behaviour

Expected Results: Sounds effects play. Animations work as expected.

Observed behaviour

Actual Results: Sound effects do not play. Animations do not work as expected.

Steps to reproduce

Steps to reproduce: Open title in NOOK for iOS reader. (Note that book has no voice track, only sound effects.) Advance to first page of text. Sound effects are not playing. Page through book. Sound effects are not playing. Animations throughout do not work as expected: are not working at all, are jerky, or are incorrect. (See attached .mov files).

Separately: For animation issues, see attached .mov files for a comparison of NOOK animations vs. iBooks.

Test file(s)

Ric, I will coordinate to send you the test file via a non-public method.

Product

• Native application (Readium SDK C++)
  • NOOK for iOS
  • iOS 9/10/11 (using WkWebView)

Additional information

Comments from the NOOK development team:

Zak Hsieh added a comment - 08/18/2017 This should be a bug in WKWebView with iFrame. Some standard CSS 2D transformation won't be applied correctly when content is inside iFrame. Not sure how much I could deal with this browser engine bug, would try to find some work arounds for this.

Otis Chen added a comment - 09/21/2017 - edited According to the flowing article. Some CSS attribute will cause this issue. I try to remove those CSS attribute than the animation in 001 video works. But it also broke Readium reader. https://stackoverflow.com/questions/26124981/z-ind (ex-issue-with-youtube-iframe-in-full-screen-mode

Zak Hsieh added a comment - 10/11/2017 Hi Tom Terrill, after investigation, this should be a bug in WKWebView with iframe. It only happens on iOS, Android does not have such problem. So far we don't have a good solution for fixing this bug, the only way may resolve this is remove iframe usage in readium-shared-js. But I won't suggest going through that way by following reasons:

1. All Readium js implementations rely on iframe and remove iframe is a big architecture changing in readium-shared-js. Once we change by this way, it will be very hard to upgrade/merge new changing from Readium.org.
2. Even though it works fine for fixed layout contents, but it will break functions for reflowable contents and readium plugins (like highlights plugin).
3. So far as we know only this book occurs this problem, remove iframe may generate other issues potentially. I don't think it is worth for doing such big architecture changing only for this book.

There is another approach, maybe we could ask Readium folks to see is there any other solution to work around this problem.

Please let me know your feedback for this.

Tom Terrill added a comment - 10/11/2017 Zak Hsieh I am curious does iBooks have the same issue?

Zak Hsieh added a comment - 10/12/2017 iBooks does not have such issue, but again, it does not the same architecture as ours and it does not use WKWebView with iframe supposedly. I could reproduce it by Readium SDKLauncher-iOS and iOS Safari. Please refer to the screenshots.

Reference NOOK for iOS JIRA: IOS-13236

[terrillt]

video_example_files.zip

[terrillt]

Apple iOS safari

[terrillt]

Readium-sdklauncher-ios