search

jiazhihao / TASO

The Tensor Algebra SuperOptimizer for Deep Learning
Apache License 2.0
692 stars 90 forks source link

Why needs build-in GraphXfers in substitution.h? #77

Closed jiahuiyang closed 3 years ago

jiahuiyang commented 3 years ago

Hi,

I found there are some build-in GraphXfers in substitution.h. Why do you need these build-in functions? Is it because they can't be generated automatically? Did you try to generate conv+bn->conv substitution by generator?

And I observe that op_enlarge only have one input in generator.cc, while it needs two inputs in substitution.cc.

jiazhihao commented 3 years ago

Hi @jiahuiyang, you are right the current implementation includes some built-in substitutions for batchnorm. The reason is because batchnorm was not implemented in the previous TASO release but we did have several use cases involving batchnorm. So a temporary solution is to add batchnorm related substitutions as built-in. They will be removed by auto generated substitutions when we have added batchnorm into the generator.

jiahuiyang commented 3 years ago

Thanks