There are three computational graph optimization opportunities in circuits:
grouping multiple circuit layers into a single more efficient one, e.g., Kronecker + Dense -> Tucker.
grouping multiple parameterization nodes into a single more efficient one, e.g., Kronecker over categorical's weights + Sum over all variables -> Einsum
ungrouping circuit layers with particular parameterization into more than one layer that are more memory efficient, e.g., Dense parameterized by a Kronecker -> a composition of Dense
The key idea for implementing this is having:
I. a registry of optimization rules, which can be possibly extended;
II. a sub-graph matcher that grounds the optimization rules;
III. an engine that orders the grounded optimization rules, applies them, and returns the optimized circuit.
There are three computational graph optimization opportunities in circuits:
The key idea for implementing this is having: I. a registry of optimization rules, which can be possibly extended; II. a sub-graph matcher that grounds the optimization rules; III. an engine that orders the grounded optimization rules, applies them, and returns the optimized circuit.