w3c / epub-tests

Test repository for EPUB3, maintained by the EPUB3 WG
https://w3c.github.io/epub-tests/
Other
22 stars 20 forks source link

Tests for manifest item fallback, unclear expected outcome #165

Open danielweck opened 2 years ago

danielweck commented 2 years ago

https://github.com/w3c/epub-tests/blob/9a023cea6d7a34fdce86b28e8f443a8c913f0fd5/tests/pub-foreign_bad-fallback/EPUB/package.opf#L20-L27

In this case, the resulting reading order (i.e. computed spine, once the fallback chain is resolved) is empty if the reading system follows the cascade of unsupported media types. In some reading systems this may cause a crash / broken user interface, in others this may result in an empty viewport ... what consitutes a pass vs. fail?

danielweck commented 2 years ago

The expectation is clear with the test that has a specific XML content type:

https://github.com/w3c/epub-tests/blob/9a023cea6d7a34fdce86b28e8f443a8c913f0fd5/tests/pub-foreign_xml-suffix-spine/EPUB/package.opf#L20-L27

... but conversely the "plain" application/xml test is problematic because XHTML itself is XML, so the description of the expected outcome needs to be clarified:

https://github.com/w3c/epub-tests/blob/9a023cea6d7a34fdce86b28e8f443a8c913f0fd5/tests/pub-foreign_xml-spine/EPUB/package.opf#L20-L27

More specifically:

If the reading system does not support XML, it should display the HTML.

https://github.com/w3c/epub-tests/blob/9a023cea6d7a34fdce86b28e8f443a8c913f0fd5/tests/pub-foreign_xml-spine/EPUB/package.opf#L8-L11

danielweck commented 2 years ago

Similarly with the JSON test, I struggle to understand the criteria for fail vs. pass:

https://github.com/w3c/epub-tests/blob/9a023cea6d7a34fdce86b28e8f443a8c913f0fd5/tests/pub-foreign_json-spine/EPUB/package.opf#L20-L27

Let's say my reading system claims to support application/json by displaying its raw contents in the publication viewport, thereby ignoring the fallback XHTML. Is that a pass? In the absence of clear content processing model for JSON, it's hard to assert success.

(Naturally, in the real world I would expect most reading systems to follow the fallback chain to XHTML, but that's beside the point)

gautierchomel commented 1 year ago

see related discussion #245

Regarding _pub-foreignbad-fallback what happens is that Thorium open system dialog and ask where to save the file. I find it's a useful feature :)

I consider it "n/a" (no fallback is needed as access to the file is provided).

Same for _pub-foreignjson-spine and _pub-foreignxml-spine