search

webmachinelearning / webnn

🧠 Web Neural Network API
https://www.w3.org/TR/webnn/
Other
393 stars 47 forks source link

Documenting implementation status #223

Closed anssiko closed 8 months ago

anssiko commented 3 years ago

As discussed at TPAC, implementation work for the WebNN API is reaching a point web developers can start to experiment with the feature in near future using a native implementation that is integrated into browsers starting with Chromium-based ones. And already today, experimentation can be done with the polyfill implementation that tracks the spec progress. Similarly, we learned at TPAC the Model Loader API implementation work is progressing in parallel.

As a learning from work on other Web APIs, I'm suggesting the group to start an Implementation Status document for the collective WG and CG specs to provide web developers a place to go to to answer their most common questions such as "can I use feature X of Y API in ...". We can also use such a page to document implementation quirks developers should be aware of that do not belong to the specs, as well as instructions how to enable the feature.

For discussion:

First, we should discuss whether this is a good idea. Assuming it is, we should discuss what is the MVP dataset to start with and how to make it easy to maintain by implementers as well as community members, to ensure this remains a useful resource. It is only useful if it is maintained.

One example that has been well received by web developers is the Web Bluetooth Implementation Status page. I think we can borrow ideas from it. We could host such a page at https://webmachinelearning.github.io/ possibly.

Later on, when the code starts to land behind a flag, we can complement this with caniuse.com entries that provide a high-level cross-browser support matrix quite popular among web developers, and link back to the more detailed Implementation Status page.

dontcallmedom commented 2 years ago

a suggestion I wanted to bring up: I think ultimately we will want to have the data integrated into MDN Browser Compat Data (which in turn gets used by e.g. caniuse).

If so, it might be useful to consider adopting the browser-compat-data conventions to document the implementation status. The schema includes structure to document flags/origin-trials as well as a way to document any limitation an implementation might have.

anssiko commented 2 years ago

@dontcallmedom, I think that is a great idea. Which spec we should look at for a good example of such an integration in action? Preferably a spec that is still evolving similarly to WebNN API to get an idea how the flow is for incremental updates.

We are interested in more web developer feedback for our work and I think the suggested MDN integration would help us meet web developers where they are.

dontcallmedom commented 2 years ago

at a high level, adding a spec / set of features in browser-compat-data requires the said feature to be implemented in at least one browser - once that's done, adding the data is a matter of constructing the right JSON file (following the schema described above) and submitting it as a PR. Data gets updated through pull requests as well.

To have the data showing up on MDN, there needs to be reference pages matching the various API entries.

anssiko commented 2 years ago

For Wasm, there's a feature-level compat table (data: features.json) as well as MDN compat table (data: https://github.com/mdn/browser-compat-data/tree/main/javascript/builtins/webassembly). These seem complementary.

For a feature that's early, I think we need both a compat table and a way to document browser- and OS-specific quirks, notes etc. similar to WebBT.

I believe our goal is the MDN reference pages replace the custom implementation status page as things get more stable and quirks ironed out, but such a page will be a helpful resource while the spec and implementations are in active development.

Note to self: see also MDN style guide and Compatibility tables for ideas.

anssiko commented 2 years ago

A few quick updates:

See also open and merged patches, #webml hashtag is used by both the APIs.

anssiko commented 1 year ago

Current status:

This broader issue will soon become more topical for the WebNN API that is expected to enter Dev Trial in the near future. This issue provides a great opportunity for someone, possibly a new contributor, to help integrate the implementation status data into MDN Browser Compat Data as suggested in https://github.com/webmachinelearning/webnn/issues/223#issuecomment-976592549

Please get in touch if you're interested.

Related, we've also made substantial progress with WebNN API web-platform-tests, also tracked in #240. This will help with this task at hand.

anssiko commented 1 year ago

Implementation Status of WebNN Operations: https://webmachinelearning.github.io/webnn-status/

Future work is to interop with browser-compat-data.

ibelem commented 8 months ago

The initial PR was submitted @https://github.com/mdn/browser-compat-data/pull/22569 @anssiko PTAL

anssiko commented 8 months ago

Thanks @ibelem!

@dontcallmedom is in the best position to review the BCD contribution, or delegate to someone who is.

ibelem commented 8 months ago

@dontcallmedom @anssiko The first version of WebNN BCD data has been merged in https://github.com/mdn/browser-compat-data/pull/22569 and available in BCD release v5.5.16.

E.g. https://caniuse.com/?search=conv2d https://caniuse.com/?search=MLGraphBuilder

anssiko commented 8 months ago

Great work! I assume https://webmachinelearning.github.io/webnn-status/ will soon also make use of BCD data.

(Related: I suggested https://github.com/Fyrd/caniuse/issues/6098#issuecomment-2003858074 as an improvement idea for caniuse.com.)

ibelem commented 8 months ago

@anssiko

Thanks for the improvement idea for caniuse.com, indeed that's a great point!

When comparing the information that https://webmachinelearning.github.io/webnn-status/ needed and the BCD data structure, I am afraid I still need to maintain the original webnn_status.json file instead of using BCD data completely.

Regardless of whether I use BCD directly or not, I need to maintain the additional json file, so I will keep both sides updated for the time being. WDYT?

anssiko commented 8 months ago

@ibelem thanks for reminding me of the important additional data we're tracking. It is reasonable to expect we need to maintain alongside the BCD data also our own webnn_status.json.

We should consider BCD to be the upstream and keep it up to date as a first priority. Only data that does not fit into the BCD schema we should maintain in our own repo. We should structure this in a way we know what's the canonical source for which data.

To enable broader contributions it'd be good to document the WebNN implementation status contribution mechanics. For inspiration, see https://github.com/Fyrd/caniuse/blob/main/README.md

Can you take the first stab on this? Thanks!

ibelem commented 8 months ago

We should consider BCD to be the upstream and keep it up to date as a first priority.

Definitely, this is our intention.

@anssiko Please review the https://github.com/webmachinelearning/webmachinelearning.github.io/pull/67 for the WebNN implementation status contribution mechanics.

anssiko commented 8 months ago

Thanks to sustained major contributions by @ibelem we now have:

Thank you @ibelem for your work on this! I want to also extend my thank you to people working on BCD @Elchi3 who helped review our initial contribution in a timely manner and @dontcallmedom for all the guidance on this journey.

I can't wait to see what web developers will build with this web technology now equipped with better developer documentation to aid them. 🚀

With this summary, I feel good closing this issue as completed. Maintenance and improvement work continues and further contributions are welcome.