Make operand data type and rank validation table-driven

[inexorabletash]

Improve readability of input operand data type and rank by introducing a table within each method definition that lines the restrictions for positional arguments and options. Steps are updated to reference the table columns.

---

Preview | Diff

[zolkis]

I'd prefer this style than the one in #646

[inexorabletash]

A few questions to consider:

dataType

• For most "secondary" operands, the type is constrained to be the same as the primary operand. The relevant steps could be written as either:
  • "If mean’s dataType is not one of its allowed data types, then throw a TypeError." - more consistent.
  • "If mean’s dataType is not the same as input's dataType, then throw a TypeError." - clearer.

rank

• In most cases there's only one allowed rank. So do we prefer:
  • "If mean’s rank is not its allowed rank, then throw a TypeError." - more consistent.
  • "If mean’s rank is not 1, then throw a TypeError." - clearer.
• If we prefer the former (reference the table), what about #659 which proposes simplifying some validation e.g. "If mean’s shape is not equal to « input’s shape[options.axis] »..." — this implicitly validates the rank. If we had that, would we still want a step reading "If mean’s rank is not ..." ?
• If any rank is allowed, do we still want to include the step, for consistency?

General

• Should we add a "validate the dataType and rank of an operand" helper algorithm? The steps for each operands could then be collapsed to just: "If validating the dataType and rank of mean returns false, then throw a TypeError." - less text, but more is hidden behind the algorithm.
• When there are multiple input operands, do we prefer separate steps or combined steps? At an extreme we could even write: "If validating the dataType and rank of any of mean, variance, options.scale (if it exists) or options.bias (if it exists) returns false, then throw a TypeError."

[zolkis]

IMHO the "consistent" option is clear enough in this case. :)

[inexorabletash]

Other notes:

• For the op groups (element-wise binary/unary/logical, pooling, reduction) a table is not given and the inline steps are retained. Any better suggestions?
• Do we keep a rank validation step if there's a subsequent shape validation step? i.e. keep both of these lines:
  1. If mean’s rank is not its allowed rank, then throw a TypeError.
  2. If mean’s shape is not equal to « input’s shape[options.axis] », then throw a TypeError.

[zolkis]

For the op groups (element-wise binary/unary/logical, pooling, reduction) a table is not given and the inline steps are retained.

Looking at the preview, I think that makes sense.

Do we keep a rank validation step if there's a subsequent shape validation step?

Shape should include rank... I'd leave rank out and let impl optionally handle that as a quick check, if it makes sense somewhere.

[inexorabletash]

Quick update here -

It seems like folks are happy with this direction. I've been keeping this PR up-to-date and it is theoretically good to merge any time, but given that it will interact a lot with eventual spec text for #463 (opSupportLimits()) I'm holding off until we're closer to specifying that, so we can avoid churn.

[fdwr]

I'm holding off until we're closer to specifying that, so we can avoid churn.

Understood.

I've been keeping this PR up-to-date

It will be beautiful.

[inexorabletash]

Marking this as "ready for review" - it looks like it didn't get very stale, and opSupportLimits() doesn't seem to require integration as currently written (the actual behavior of that method isn't specified in detail). @fdwr & @huningxin - please take a look?

[inexorabletash]

It seems some operators are not covered, such as argMin/Max. Do you plan to do them in a separate CL?

My approach was to very mechanically migrate explicit rank/shape validation steps from the prose steps that contain the constraints to the table format where the prose steps reference the table. That has these implications:

• If an op didn't have either explicit data type or rank validation steps, the table wasn't added since nothing would reference it. So e.g. argMin/Max didn't get a table.
• If an op's data type and/or rank was provided as parameters to the algorithm steps, the steps were not touched, and the table wasn't added for that op.
• If there wasn't an explicit step validating the data type (e.g. gather), the data type in the table is given as: any data type/any
• If there wasn't an explicit step validating the rank (e.g. gemm), or that validate the rank against something other than an explicit number (e.g. layerNormalization), the rank in the table is given as: any rank/N
  • Note that steps that validate the shape (e.g. where it must be broadcastable to something) are considered to not validate the rank.

In other words, only introduce the table where it will be normatively referenced.

Obviously we can change this!

• We can include the table for all ops, which would have "any" / "N" for the allowed data types/ranks
  • argMin/argMax, cast, clamp, concat, element-wise binary, element-wise logical, expand, pad, reshape, slice, split, transpose
  • 🤔 Do we also add steps to the algorithm that reference the table, even though it's a no-op?
• We can add in tables for the cases where data type/rank are passed in to the algorithm
  • element-wise unary, averagePool2d, l2Pool2d, maxPool2d, reduceL1, reduceL2, reduceLogSum, reduceLogSumExp, reduceMax, reduceMean, reduceMin, reduceProduct, reduceSum, reduceSumSquare
  • 🤔 This implies we'd have a table per op (e.g. one for abs, one for ceil, one for cos, etc), and modify the algorithms to pass/take both data types and ranks. That's... a lot of tables.
• We can expand the definition of "allowed ranks" beyond "any" or an explicit number to include:
  • a reference to other parameters, e.g. "same as input"
  • a range
  • 🤔 Do we introduce the implied duplicate steps? e.g. for layerNormalization, do we have steps that validate rank against the table (i.e. scale is in [0, input rank]) and a specific value (i.e. scale is equal to options.axes's size) ?

I am not sure whether it should point to the allowed data types table of batchNorm. It currently points to the allowed data types definition.

Yeah, I wasn't sure how much effort to put into explicitly linking things. Currently nothing actually links to the tables; it's implied that if a step mentions "allowed data types" or "allowed ranks" then the reader should find the nearby table and look up the appropriate value for the named input operand. It might be better to link those phrases in steps to the table cells.

[huningxin]

@inexorabletash

In other words, only introduce the table where it will be normatively referenced.

Makes sense to me! Thanks for the explanation!

We can include the table for all ops, which would have "any" / "N" for the allowed data types/ranks

• argMin/argMax, cast, clamp, concat, element-wise binary, element-wise logical, expand, pad, reshape, slice, split, transpose

logicalNot may need steps to validate input data type is "uint8"? (and logicalAnd, logicalOr, logicalXor in @fdwr 's wave3 proposal)

And wave3 will introduce "int4"/"uint4" data types, I assume "any" data types may not apply for some ops, we may need "anyDataTypesAtLeast8Bits" (kAllDataTypesAtLeast8bits in Chromium prototype)

Of course we can handle the wave3 ops / data types after its PR landed.

• 🤔 Do we also add steps to the algorithm that reference the table, even though it's a no-op?

I think it's unnecessary to add no-op steps.

We can add in tables for the cases where data type/rank are passed in to the algorithm

• element-wise unary, averagePool2d, l2Pool2d, maxPool2d, reduceL1, reduceL2, reduceLogSum, reduceLogSumExp, reduceMax, reduceMean, reduceMin, reduceProduct, reduceSum, reduceSumSquare
• 🤔 This implies we'd have a table per op (e.g. one for abs, one for ceil, one for cos, etc), and modify the algorithms to pass/take both data types and ranks. That's... a lot of tables.

Yes, it's a lot. Can we group some tables? Like a table for float-pointing element-wise unary, including ceil, floor, cos, sin, erf, exp, log, reciprocal, tan etc.,. However it may require linking the "allowed data types" to a particular table which is another discussion.

We can expand the definition of "allowed ranks" beyond "any" or an explicit number to include:

• a reference to other parameters, e.g. "same as input"
• a range
• 🤔 Do we introduce the implied duplicate steps? e.g. for layerNormalization, do we have steps that validate rank against the table (i.e. scale is in [0, input rank]) and a specific value (i.e. scale is equal to options.axes's size) ?

Duplicating steps is not the intention. I just feel user should not expect validation failure if the supplied tensor is allowed according to "allowed ranks". We have "same as input" for "allowed data types". Could we have something like "maximum to input rank" for "allowed ranks"?

[inexorabletash]

It seems some operators are not covered, such as argMin/Max.

f61292d64b86ca4088d9a4e886d634ea65634fce adds tables for all ops (or op categories)

[inexorabletash]

• 🤔 This implies we'd have a table per op (e.g. one for abs, one for ceil, one for cos, etc), and modify the algorithms to pass/take both data types and ranks. That's... a lot of tables.

Yes, it's a lot. Can we group some tables? Like a table for float-pointing element-wise unary, including ceil, floor, cos, sin, erf, exp, log, reciprocal, tan etc.,. However it may require linking the "allowed data types" to a particular table which is another discussion.

For now, in f61292d64b86ca4088d9a4e886d634ea65634fce I left it at one table per "op category" and introduced the phrase "specified as part of operation steps" in the table.

[inexorabletash]

I just feel user should not expect validation failure if the supplied tensor is allowed according to "allowed ranks".

Makes sense. Added in 4bf14f6504ba0587d33ee4439cc5e05f7f081085 for the cases you called out - I didn't audit the ops for more, though. How does that look?

[inexorabletash]

As always, having the table contain "specified as part of operation steps" is an intentionally minimal change. There are a few more things we could do:

• Have separate tables; e.g. for element-wise logical we could have a generic table with "any" and a table specific to logicalNot; element-wise unary would need 3 (signed types, float types, and any for identity); pooling would need 2; reduction ops would need 3
• Keep a single table, but expand the text with casual developer-focused commentary, e.g. for element-wise logical it could read: "specified as part of operation steps; most element-wise logical ops support any data type, but logicalNot() only supports uint8"