Efficiently handle Relay's simplified-for-inference batch norms

gussmith23 / glenside

A pure, low-level tensor program representation enabling tensor program optimization via program rewriting. See the web demo at https://gussmith23.github.io/glenside-web-demo/

71 stars 10 forks source link

Efficiently handle Relay's simplified-for-inference batch norms #55

Open gussmith23 opened 4 years ago

gussmith23 commented 4 years ago

Relay can run the SimplifyInference pass to simplify batch norm operators into a collection of primitive ops. Ideally, Glenside would intake these primitive ops and do efficient codegen for them. I don't think this is on the critical path for ISCA right now, and our workaround is to just take in opaque "BatchNormInference" nodes.