w3c / epub-tests

Test repository for EPUB3, maintained by the EPUB3 WG
https://w3c.github.io/epub-tests/
Other
22 stars 20 forks source link

Add fxl equivalents for the media overlays tests #274

Closed mattgarrish closed 1 year ago

mattgarrish commented 1 year ago

This is a quick conversion of the MO tests so they can be checked on reading systems that only support playback with fixed layouts.

I've added a rendition:layout/pre-paginated declaration to each package document and a viewport declaration to each xhtml document (except the nav docs which aren't in the spine). I also added some minor styling to increase the font size to improve the readability a bit (probably could have bumped down the viewport size, but given time constraints whatever works, right?).

I didn't modify the tests result reporting. I assume we'll just tell testers to use the version that works with the reading system they're testing at this point.

I did some checking of the files in ibooks just to verify that the publications will open and you can initiate playback and they seem fine.

mattgarrish commented 1 year ago

These changes are based on the assumption that these tests are in addition of today's tests

No, I would see these as duplicates for reading systems that only accept FXLs. The isntructions for testing should just say to use the appropriate test document based on the results of the -synchronization tests. Those two tests check support for reflowable vs. fxl, so you shouldn't need to duplicate all the other tests. You can figure out from that one whether media overlays generally are supported in one, the other, or both layout formats.

iherman commented 1 year ago

These changes are based on the assumption that these tests are in addition of today's tests

No, I would see these as duplicates for reading systems that only accept FXLs. The isntructions for testing should just say to use the appropriate test document based on the results of the -synchronization tests. Those two tests check support for reflowable vs. fxl, so you shouldn't need to duplicate all the other tests. You can figure out from that one whether media overlays generally are supported in one, the other, or both layout formats.

Ok, so then my comments above stand for all tests on the opf and also on the necessity to add the test references to the RS spec.

Thanks!

I.

mattgarrish commented 1 year ago

and also on the necessity to add the test references to the RS spec.

You mean not to add, correct? Since they're not testing anything new, why would they have different identifiers? You pick the file to use for the test and add your result. The format you pick is irrelevant.

iherman commented 1 year ago

and also on the necessity to add the test references to the RS spec.

You mean not to add, correct? Since they're not testing anything new, why would they have different identifiers? You pick the file to use for the test and add your result. The format you pick is irrelevant.

Hm. We may not understand one another. The way I understand we plan to do it (which is also reflected in the way the reporting mechanism works) is as follows. We have, to take an example, two tests:

This means we have two folders in the tests folder with those two names, and we have two epub files:

From the reporting point of view these are two independent tests. They are differentiated via their names and the pattern is that the pub-id values in the respective package files are identical to the test name on the file system. The reporting mechanism will, essentially, duplicate the table rows having them both there, so will the reports. And, finally, the references from the RS spec must also be duplicated.

The reporting system works by picking up all the test files in the directory. If you "just" put it there without getting them into the admin framework, then there will be some mess I presume.

(Ideally, we would use multiple rendition :-)

mattgarrish commented 1 year ago

Doesn't this lead to duplicate tests in the report even though the tests are checking the same thing and will have the same description?

I was assuming when you harvest the tests that you'd skip the duplicates.

iherman commented 1 year ago

Doesn't this lead to duplicate tests in the report even though the tests are checking the same thing and will have the same description?

I was assuming when you harvest the tests that you'd skip the duplicates.

Hm. That is not what I had in mind, but we can do this, too. I would have to put the skipped items into the program itself.

But if this is the way we want to go, you should write something into the description entry of the metadata. That item goes into the "Description" column of https://w3c.github.io/epub-tests/#sec-media-overlays-data and it should make it clear that we have two versions of each of those.

Just to be on the safe side, can you give me a list of the "duplicates", ie, the folder names I would have to skip?

This PR should not be merged before I make and commit the change in the program...

Ivan

mattgarrish commented 1 year ago

But if this is the way we want to go, you should write something into the description entry of the metadata.

Ya, that makes sense, but probably means I have to update all the tests and rebuild them. :(

I'll see what I can put together.

mattgarrish commented 1 year ago

Shutting down this PR as it looks like it's no longer needed.