(1) injective (one-to-one map, e.g., add), (2) reduction (e.g., sum), (3) complexout-fusible (can fuse element-wise map to output, e.g., conv2d), and (4) opaque (cannot be fused, e.g., sort).
generic rules to fuse these operators
Multiple injective operators can be fused into another injective operator. A reduction operator can be fused with input injective operators (e.g., fuse scale and sum). Operators such as conv2d are complex-out-fusible, fuse element-wise operators to its output.
Intro
OSDI 2018, PDF
Operator Fusion
recognize four categories of graph operators:
(1) injective (one-to-one map, e.g., add), (2) reduction (e.g., sum), (3) complexout-fusible (can fuse element-wise map to output, e.g., conv2d), and (4) opaque (cannot be fused, e.g., sort).
generic rules to fuse these operators
Multiple injective operators can be fused into another injective operator. A reduction operator can be fused with input injective operators (e.g., fuse scale and sum). Operators such as conv2d are complex-out-fusible, fuse element-wise operators to its output.