search

w3c / imsc-hrm-tests

IMSC-HRM tests
Other
0 stars 1 forks source link

Consider making spec version-specific subfolders #4

Closed nigelmegitt closed 11 months ago

nigelmegitt commented 11 months ago

I discovered while doing some implementation work (still in progress) that none of the tests in this repo are EBU-TT-D conformant, even though they almost all could be with some edits (which I've done locally).

The tests that can not be EBU-TT-D conformant may also not be conformant to some versions of IMSC, for example support for #textShadow was introduced in v1.1, so is not applicable to v1.0.1; in that context an IMSC-HRM validator built on bindings created for IMSC 1.0.1 would not be able to process pass/dur013-pass.ttml for example, but should still count as an implementation for CR exit.

I wonder if we can structure the tests in the repo to make it easier or possible for implementations predicated on a specific profile to choose just the tests that apply to them? I'd really like to include EBU-TT-D in that set as well.

As part of this we could replace the non-EBU-TT-D conformant tests with ones that do conform, or add alternative variants that are EBU-TT-D conformant. The main differences are:

They're all substantively the same content and IMSC conformant, so should process with the same results.

himorin commented 11 months ago

I have some concern about organizing test contents, like maintenance over similar files if we duplicate test contents into multiple packages for 1.0.1, 1.1, or EBU-TT-D. Some possible ways exists, like dividing into base directory (e.g. for 1.0.1) with extensions (for 1.1), having files of list per target environment, but I'd have some secured plan before moving forward...

One question, which I believe answer is no, but is there any side-effect, such as making non-conformant in some other environment, with replacing the non-EBU-TT-D conformant tests?

nigelmegitt commented 11 months ago

One question, which I believe answer is no, but is there any side-effect, such as making non-conformant in some other environment, with replacing the non-EBU-TT-D conformant tests?

I don't think we can say for sure. It's always possible that someone has come up with a constrained subset of IMSC that isn't compatible with EBU-TT-D, but I don't think there are any in the TTML Profiles Registry.

nigelmegitt commented 11 months ago

I have some concern about organizing test contents, like maintenance over similar files if we duplicate test contents into multiple packages for 1.0.1, 1.1, or EBU-TT-D. Some possible ways exists, like dividing into base directory (e.g. for 1.0.1) with extensions (for 1.1), having files of list per target environment, but I'd have some secured plan before moving forward...

I agree, both with the concern and the need to have a plan.

palemieux commented 11 months ago

I suggest two complementary approaches:

Since the IMSC HRM is intended to be compatible with all IMSC versions, so I do not see a clear benefit in signaling which version of IMSC a document claims to conform with.

nigelmegitt commented 11 months ago

There aren't any additional EBU-TT-D tests that I'm aware of. I will open a pull request to make the existing tests EBU-TT-D conformant where they can be, without changing the computed values or document structure.

Since the IMSC HRM is intended to be compatible with all IMSC versions, so I do not see a clear benefit in signaling which version of IMSC a document claims to conform with.

Right, but the tests aren't all compatible with all IMSC versions, which is where the issue arises.

nigelmegitt commented 11 months ago

In #6 I simply added XML comments to the tests to indicate those parts that are not compatible with EBU-TT-D or IMSC 1.0.1, which affected the dur012 and dur013 tests in their pass and fail variants.

nigelmegitt commented 11 months ago

The tests are now EBU-TT-D conformant as much as possible, and include an XML comment explaining when that isn't possible, for both IMSC 1.0.1 and EBU-TT-D. I think this issue has now been adequately addressed. Closing.