search

jiazhihao / TASO

The Tensor Algebra SuperOptimizer for Deep Learning
Apache License 2.0
693 stars 90 forks source link

Graph partition #37

Open ShouyangDong opened 4 years ago

ShouyangDong commented 4 years ago

Hello, I have seen your video of taso on youtube. In order to accelerate the speed of your searching process, you will do graph partition at first. So is that means your subtitions result will greatly depends on the graph partition. Since resblock and inception module can be easily divided into sungraphs. But on other tasks, this process will be complicate. Will graph partition be the bottleneck of this graph substitution method? Look forward to your answer. Thank you!

jiazhihao commented 4 years ago

@SYangDong Thanks for your interest in our work. I guess you were looking at the MetaFlow talk at SysML'19 this year. The graph partitioning/splitting support hasn't been integrated into TASO yet, and we plan to do in the next couple months.

In the current version, TASO directly optimizes the entire computation graph and you can use alpha and budget to guide the search. More specifically, alpha controls the pruning condition (as specified in the MetaFlow paper), and budget denotes the number of candidate graphs TASO explores before exit.

ShouyangDong commented 4 years ago

Thank you very much. Look forward to your future work.