jiazhihao / TASO

The Tensor Algebra SuperOptimizer for Deep Learning
Apache License 2.0
693 stars 90 forks source link

Larger search depth exhausts memory space #57

Open anhtuanhsgs opened 4 years ago

anhtuanhsgs commented 4 years ago

Hi, I tried generating substitutions with a larger search space with depth=4 (MAX_NUM_OPS=4) in generator.cc. The algorithm examined around 300 mil graphs which take up all of memory (128 GB main memory with 28 GB swap space) then crashed.

Would it be possible to run the algorithm with larger search space if more memory is given? Would optimizing with substitutions with depth of 4 produce a substantially better speed than the default setting in TASO at the moment?