search

jiazhihao / TASO

The Tensor Algebra SuperOptimizer for Deep Learning
Apache License 2.0
692 stars 90 forks source link

How many rules are there in TASO now? #51

Open puddingfjz opened 4 years ago

puddingfjz commented 4 years ago

Hi, now how many transformation rules are used in TASO?

jiazhihao commented 4 years ago

@puddingfjz Thanks for your interests in TASO. TASO currently uses around 700 default substitutions, and you can easily customize the substitutions you want to explore by running https://github.com/jiazhihao/TASO/blob/master/src/generator/generator.cc. Line 1731-1783 specifies the set of operators you want TASO to use when generating substitutions.

puddingfjz commented 4 years ago

When I try to optimize BERT (use the example provided by TASO), I find the final optimization sequence only has 24 steps. The GPU I use is NVIDIA Tesla P100. Besides, I also want to ask whether current TASO jointly optimize the data layout as mentioned in the paper?