Opinfo testing for python frontend

NVIDIA / Fuser

A Fusion Code Generator for NVIDIA GPUs (commonly known as "nvFuser")

Other

250 stars 49 forks source link

Opinfo testing for python frontend #432

Open rdspring1 opened 1 year ago

rdspring1 commented 1 year ago

This issue tracks opinfo testing for various operations in the python frontend. The goal is to find any corner cases and test for error conditions.

Unary

[x] ceil, floor, frac, neg, bitwise_not, trunc, reciprocal, round

Bool

[x] isfinite, isinf, isnan, isneginf, isposinf, isreal, signbit

Float

[x] abs, acos, acosh, asin, asinh, atan, atanh, cos, cosh, erf, erfc, erfcinv, erfinv, exp, exp2, expm1, lgamma, log, log10, log1p, log2, reciprocal, rsqrt, sign, sin, sinh, sqrt, tan, tanh

Binary

Float

[x] atan2, truediv

Float-Only

[x] nextafter

Cast-Integers-to-Float

[x] add, ceildiv, div, fmod, mod, mul, pow, remainder, sub

Bitwise

[x] and, or, xor

Int-Only

[x] bitwise_left_shift, bitwise_right_shift, logical_right_shift, gcd

Compare

[x] eq, ge, gt, le, lt, ne

Ternary

[ ] lerp

Binary - Alpha

[ ] add-alpha, sub-alpha

Ternary-Alpha

[ ] add-cmul

Ternary - Conditional

[x] where

Threshold

[ ] clamp
[ ] threshold

Tensor Creation

[ ] full
[ ] iota

Transformation

[ ] cast

Shape

[x] broadcast
[x] broadcast-in-dims
[x] cat
[x] gather
[x] index-select
[x] pad
[x] permute
[x] reshape
[x] slice
[x] take-along-axis

Reduction

[x] max
[x] min
[x] prod
[x] sum

Normalization

[x] batch-norm
[x] variance
[x] variance-mean

Random

[ ] uniform
[ ] normal

Matrix-Multiplication

NN, NT, TN, TT

rdspring1 commented 1 year ago

Known Issues:

[ ] https://github.com/NVIDIA/Fuser/issues/427
[ ] define_tensor does not adhere to nvfuser max tensor size. The limit is currently 8 dimensions.
[ ] Do not downcast back to original dtype after upcasting to float32 from bfloat16 or float16
[ ] broadcast_in_dim error check: a[idx] == 1 OR a[idx] == output_shape[new_idx]
[ ] gather and take-along-axis error checks:
1. Index tensor can only be int or long dtypes.
2. Out-of-bounds index value.
3. Expected dimension of index tensor to be smaller than input tensor except for specified axis.
4. Input and Indices tensors should have the same number of dimensions.
[ ] https://github.com/NVIDIA/Fuser/issues/521
[x] For the pad operation, check that the tensor argument and pad value have the same dtype.