Graph with no input

[philloooo]

Hi, it looks like currently we don't prohibit graph without any input. I was working on Chromium CoreML backend and realized that CoreML doesn't support model without inputs.

It doesn't seem to be useful to have a model that takes no input, as that will just generate the same result every time. So I'd propose to let builder.build error out for no-input graph. What do you think?

example no input graph:

const context = await navigator.ml.createContext();
const builder = new MLGraphBuilder(context);
const operand = builder.mul(builder.constant(2), builder.constant(3));
const graph = await builder.build({'output': operand});

[a-sully]

Is this a dup of #614? Or are you saying that graphs built with one or more MLGraphBuilder.constant() operands but no MLGraphBuilder.input() operands is also not valid in CoreML?

FWIW the question of whether WebNN should be specified to allow no-op graphs (which is what #614 is asking) or no-input (constant-only) graphs seems distinct from the question of how we'd implement that on Mac. If it's true that constant-only graphs are not supported by CoreML, presumably the implementation could work around that by e.g. passing constant() operands as inputs to CoreML under the hood?

[inexorabletash]

Or are you saying that graphs built with one or more MLGraphBuilder.constant() operands but no MLGraphBuilder.input() operands is also not valid in CoreML?

Yes, @philloooo gave more details offline. To be explicit for future readers: the issues are related, but #614 is about graphs where one of the graph input or constant operands is specified as an output, i.e. a single node graph. This issue about a graph with no inputs, of arbitrary complexity:

614 asks if this should be allowed:

const context = await navigator.ml.createContext();
const builder = new MLGraphBuilder(context);
const input = builder.input('A', {dataType:"float32"});
const graph = await builder.build({'output': input});

This issue asks if this should be allowed:

const context = await navigator.ml.createContext();
const builder = new MLGraphBuilder(context);
const operand = builder.mul(builder.constant(2), builder.constant(3));
const graph = await builder.build({'output': operand});

If it's true that constant-only graphs are not supported by CoreML, presumably the implementation could work around that by e.g. passing constant() operands as inputs to CoreML under the hood?

Yes, so it's not a blocker, but worth discussing. We can add a note to implementers to the spec with this hint if we settle on allowing it.

[huningxin]

This issue asks if this should be allowed: const context = await navigator.ml.createContext(); const builder = new MLGraphBuilder(context); const operand = builder.mul(builder.constant(2), builder.constant(3)); const graph = await builder.build({'output': operand});

Just tested this sample code in Chromium prototype, it is allowed by both XNNPACK backend and DirectML backend.

It doesn't seem to be useful to have a model that takes no input, as that will just generate the same result every time.

This is true. But we observed some models use random value tensor generator op, for example RandomNormalLike in Stable Diffusion 2.1 VAE Encoder. WebNN may want to support the random op in the future. Then a model take takes no inputs may generate different result with the random op.

[wacky6]

Purely as a thought experiment: I think graphs without an input is still be useful. I don't see a reason to them from a compute primitive perspective (doing compute on a constant is still a well defined computation).

Building on Ningxin's comment in https://github.com/webmachinelearning/webnn/issues/615#issuecomment-2016342649

In Stable Diffusion, for example, I want to use random latent to see what's learned by the model (e.g. keep generating random images without a text prompt).

Assuming WebNN will include random() (which generates random numbers of a given shape and data type). I can define a WebNN graph like this (at a high level):

const builder = new MLGraphBuilder(context)
const latent = builder.random(/* shape of latent */)
const output = /* subgraph that decodes latent to an image */ subgraph(/*input=*/ latent)
const graph = await builder.build({'output': output})

[philloooo]

thanks for the feedback! I've added a stub node for the CoreML implementation to support empty inputs.

For the spec language, maybe we can add a side note: WebNN allows creating a MLGraph without input operands. If the underlying platform doesn't support that, implementations can work around by adding a stub input or passing constants as inputs to the graph.

?

[inexorabletash]

Nit: "WebNN allows" → "The WebNN API allows" or "The MLGraphBuilder API allows"

Nit: "a MLGraph" → "an MLGraph" (English is weird, sorry!)

Nit: "work around by" → "work around this limitation by"

Otherwise LGTM!

[fdwr]

Purely as a thought experiment: I think graphs without an input is still be useful.

@wacky6 : Yeah, I encountered the same issue with fillValueSequence (now known as a constant overload) where I couldn't even test the operator by itself because empty inputs were not supported. So I had to add some dummy ops to the graph just to satisfy that.