Optimized Circuit Product Template (for reference)

[IrwinChay]

Initial version of circuit product

TODOs:

• product with different scope
• support tucker
• circuits with different dimensions (num_units)
• tests, especially with values in exp reparam
• some edge cases: input channel > 1 with normal distribution partition function...

Discussions:

• To support product of circuits with different num_units, the symbolic layer needs to log more than one num_units, one way to support this is to change its type into Union[List[int], int]. And the reparam materialize() function also changed accordingly. But this changes a lot so its better to discuss
• product_partition.py, should it be implemented within the Cat / Normal layer itself?
• should we support different inputs x for each of the 2 circuits?
• Discuss design choices of doing product with different scope

[lkct]

Closed in favour of #180

Branch renamed to bak_circuit_product