Torch compile support (for LLaMA and Gemma): add the flag --torch_compile to the data args. Improves throughput by approx 1.5x in a benchmark for LLaMA 13b
Incremental save/load: resume prompt creation from an intermediate checkpoint if the run is interrupted, set --overwrite_output_dir=False and --save_steps to your choice of save steps
Adds:
--torch_compileto the data args. Improves throughput by approx 1.5x in a benchmark for LLaMA 13b--overwrite_output_dir=Falseand--save_stepsto your choice of save steps