search

pymc-devs / symbolic-pymc

Tools for the symbolic manipulation of PyMC models, Theano, and TensorFlow graphs.
https://pymc-devs.github.io/symbolic-pymc
Other
61 stars 8 forks source link

Graph Normalization and miniKanren Goals for TensorFlow Graphs #19

Open brandonwillard opened 5 years ago

brandonwillard commented 5 years ago

After TensorFlow graph support is added (i.e. #4), we will need to adapt the existing miniKanren relations to TF[P] objects.

This might require a core set of linear/tensor-algebraic graph normalization/canonicalization relations, as well, since those were previously provided by Theano.

FYI: It will probably be easier to create new relations specific to TF[P]—compared to generalizing the meta objects enough to cover both. Regardless, we should keep both options in mind.

josephwillard commented 5 years ago

What would be the best way to approach this issue?

brandonwillard commented 5 years ago

We can start by creating a goal, normalizeo, and make it able to normalize according to all the standard scalar group axioms under a simple ordering like lexicographical path ordering.

For example, the standard, reduced normalization rewrite-rules for scalars: 

inv(e) -> e
x * e -> x
e * x -> x
inv(inv(x)) -> x
x * inv(x) -> e
inv(x) * x -> e
x * (inv(x) * y) -> y
inv(x) * (x * y) -> y
(x * y) * z -> x * (y * z)
inv(x * y) -> inv(y) * inv(x)

Next, we'll need to add some rewrite rules for operator properties (e.g. linearity), tensors, and their interactions with scalars. We might be able to get away with some small combination of axioms.

After that, we can designate specific TF operators (e.g. RandomStandardNormal, Add, etc.) as having said properties using kanren.facts.