[meta] Presentation API Testing

[anssiko]

When doing a larger change spanning across components it is often helpful to have a single issue for tracking the bigger change. This is such a "meta issue" for tracking the development of the test suite for the Presentation API.

Resources:

• w3c/web-platform-tests/presentation-api/ - GitHub repo for tests
• w3c-test.org/presentation-api/ - published tests
• Second_Screen/Work_Mode/#Testing - how to contribute

Tasks and related sub-issues:

• [x] Land initial web-platform-tests/presentation-api PR: w3c/web-platform-tests/pull/2432
• [x] Split interface Presentation IDL into partials: #230
• [x] Update web-platform-tests/presentation-api to match the ED spec
• [x] Annotate IDL blocks with idl-controlling-ua and/or idl-receiving-ua: #268
• [ ] Improve test coverage
• [x] Publish test results (to be updated as test coverage increases)
  • https://w3c.github.io/test-results/presentation-api/controlling-ua/all.html
  • https://w3c.github.io/test-results/presentation-api/receiving-ua/all.html
• [x] https://github.com/w3c/web-platform-tests/issues/4168

All - @louaybassbouss is your test facilitator for the Presentation API spec and will be responsible for coordinating the test suite development. Please work with @louaybassbouss by contributing new test cases and reviewing existing tests. See the Work Mode > Testing wiki for more information on how to setup your own test environment.

Happy testing!

[markafoltz]

That sounds good to me.

[louaybassbouss]

I just submitted a PR with the new test results of Presentation API including WebIDL test results. Three Browsers are tested Chrome for Android, Chrome Desktop and Firefox for Android. Thx for @aleygues for helping in running the tests.

[markafoltz]

From https://www.w3.org/2017/11/06-webscreens-minutes.html#x11:

ACTION: @mfoltzgoogle to investigate gaps in the test coverage for Chrome

[markafoltz]

In Chrome, we are tracking progress on WPT here.

https://bugs.chromium.org/p/chromium/issues/detail?id=705170

We are tracking improving coverage in these two issues. The former run over Blink + content layer, and the latter run over the whole browser.

https://bugs.chromium.org/p/chromium/issues/detail?id=505664 https://bugs.chromium.org/p/chromium/issues/detail?id=678472

[tomoyukilabs]

@mfoltzgoogle @schien FYI, I have made detailed reports of WPT results: [controlling UA][receiving UA]. I'd be happy if these reports would be helpful for you.

Also, I have submitted a couple of PRs to the WPT repository to fix several bugs: https://github.com/w3c/web-platform-tests/pull/9046, https://github.com/w3c/web-platform-tests/pull/9009 (merged).

[markafoltz]

Thanks for the updated results, the detailed reasons for failure help with debugging. I commented on the pending PR as well.

[tomoyukilabs]

The GitHub repo for the Presentation API test suite was moved to: https://github.com/web-platform-tests/wpt/tree/master/presentation-api