jason9693 / MusicTransformer-tensorflow2.0

implementation of music transformer with tensorflow-2.0 (ICLR2019)
MIT License
353 stars 81 forks source link

Refactor to `@tf.function` #60

Open khatchad opened 1 month ago

khatchad commented 1 month ago

We converted several eager execution function to hybrid execution as part of an assessment of our refactoring tool. The transformations produced were completely automated. We have some preliminary evidence that this improves the run-time performance:

Test Python version TensorFlow version Epochs Before accuracy After accuracy Before loss After loss Before elapsed time (s) After elapsed time (s) Speedup
MusicTransformer-tensorflow2.0/train.py 3.10.0 2.9.3 5 0.0233113606 0.02381204049 4.956358294 4.923633542 1330.180665 919.3014389 1.446947224

The above analysis was repeated five times for each program version (before and after the refactoring) and the values were averaged. The increased speedup is ~1.45 with seemingly negligible loss of accuracy. We would appreciate any feedback you may have to help us as assess our refactoring approach. Thank you for your time!