When materializing a symbolic circuit, we have to enable the user to choose whether to fold the tensorized circuit or not. This is crucial for efficiency (e.g., when showcasing the library in notebooks).
For now, folding will copy tensors and materialize a new circuit, so it will lose the references to parameters within the circuit computational graph (e.g., the parameters of the circuits on which we compute a product). This would be the fastest way to implement folding in time for the April's workshop.
Later on, we can implement an "einsum symbolic engine" as discussed with Rickey, which will keep track of the parameters in the circuit computational graph and will handle folding accordingly.
What do we need in practice for this issue:
Folded versions of layers, perhaps by sharing the einsum definition in most layers as we just need to append a fold dimension.
Let's discard padding parameters as to fold layers with different arity, i.e., we can restrict the library to folding layers having matching arity.
Implement the folded construction via book-keeping (see old version for this).
When materializing a symbolic circuit, we have to enable the user to choose whether to fold the tensorized circuit or not. This is crucial for efficiency (e.g., when showcasing the library in notebooks).
For now, folding will copy tensors and materialize a new circuit, so it will lose the references to parameters within the circuit computational graph (e.g., the parameters of the circuits on which we compute a product). This would be the fastest way to implement folding in time for the April's workshop.
Later on, we can implement an "einsum symbolic engine" as discussed with Rickey, which will keep track of the parameters in the circuit computational graph and will handle folding accordingly.
What do we need in practice for this issue: