What is purpose of unbound tensor?

[zoeoz]

Any bound tensor can cast to a generic tensor, and any tensor can cast to an unbound tensor.

Aside from this extra level of casting, it seems generic tensor and unbound tensor are the same.

Is there specific reason why generic tensor is not sufficient and NNEF requires the unbound tensor syntax?

[gyenesvi]

As for casting, only bound tensors can be cast to generic tensor type (see section on type casting), because the parser should be able to decide whether a cast is valid without looking at a larger context, and for unbound tensors that info would not be available on the place of invocation.

NNEF has a very limited notion of generics (to keep parsing simple); only a single generic symbol exists (the ?, no symbolic type identifiers) and that allows you to have a generic operation with a single type parameter, which must be the same wherever you use it, so it enables binding of tensor types. For example, it can express that an operation's two inputs have the same data-type, or that the output inherits the input data-type.

The declaration

fragment op<?>( x: tensor<?>, y: tensor<?> ) -> ( z: tensor<?> )

expresses that all of x, y and z has the same data-type. This is enough in most situations, however, not always. It is possible to conceive operations that have generic tensor parameters, but there's also another argument that need not be bound to the same data-type, and that the data-type does not really matter; this is where unbound tensors come into play. An example could be an operation that reshapes a tensor to the shape of another reference tensor, whose data-type does not matter, since only it's shape is used:

fragment reshape_like<?>( input: tensor<?>, reference: tensor<> ) -> ( output: tensor<?> )

where the output data-type is bound to the input data type, but the data-type of the reference tensor is not bound to them.

[zoeoz]

Thank you, that clears it up for me.