search

webmachinelearning / webnn

🧠 Web Neural Network API
https://www.w3.org/TR/webnn/
Other
397 stars 48 forks source link

Support multiple op sets / builders #714

Open zolkis opened 5 months ago

zolkis commented 5 months ago

This is an experimental space to consider what it would take to allow multiple types of builders / op sets in order to evaluate the points in e.g. #453, #573 etc.

All builders would have a build() method producing an MLGraph, and specify a chosen op set that is allowed when building the graph.

A builder class hierarchy is also possible, with the base class supporting input(), constant() and build() - and an eventual core op set.

Shared ops can be just references to a single place of definition, e.g. in MLGraphBuilder.

Different enough op sets could be specified in separate documents, following the structure of e.g. the Generic Sensor API. That would also unload the core structure of the Web NN API from the mass of the op set spec.

Also, this would allow experimenting with different op sets, while keeping MLGraphBuilder unchanged / evolve separately.

[ It's a side question whether would it make sense to also expose a compile() method on MLContext that would encapsulate/consolidate individual build() peculiarities on a full graph build. ]

reillyeon commented 5 months ago

This implies that operator availability for a particular context will tend to be organized into consistent sets. Do you think that is the case or should feature detection be done on an operator-by-operator basis?

zolkis commented 5 months ago

If we would do it like in Generic Sensor, the op set(s) will be defined in the specification(s).

I'd prefer this explicit/consistent approach, but first let's see how the op sets would actually differ and whether those could be merged / factorized.

Based on that exercise, we can decide if testing for supported operators is sufficient, while being a leaner approach.

The purpose of this proposal is to enable experimenting with contributions on different op sets, without interfering with the main spec.

zolkis commented 5 months ago

Clarification: I think there are 2 use cases collated here.

One is to support a lower level op set, such as tensor ops, referred in #453. Separating these op sets would allow experimenting with them without interfering with the main spec. Such op sets would need a separate specification each.

The other is to expose only a supported subset of ops for a given context, out of a standardized pool of possible ops. This includes e.g. the core + additional op set mentioned in #573. In principle, this approach would enable exposing arbitrary subsets of ops, but it will become complicated.

I think it would be better to specify/standardize the op sets that may be supported (that would pass as Web NN compliant), like in Generic Sensor each sensor API specializes the generic sensor interface. Eventually we'd give them standard names, and expose the list supported names in the context object.

So a dynamic approach could be that contexts would expose the (consistent) op sets supported in the context.

One of these supported op sets (name or object) needs to be passed to the builder constructor. There could be a default set for a context (by spec).

(Just thinking aloud).