Code is extremely slow!

pytorch-labs / gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

BSD 3-Clause "New" or "Revised" License

5.36k stars 485 forks source link

Code is extremely slow! #78

Open yafehlis opened 6 months ago

yafehlis commented 6 months ago

I am using AMD MI210s. After loading the models, the following steps are extremely slow (see screenshot). It turned out the Compilation time is 270 seconds. Could you please help me fix it and make it run faster? Thanks, Yao Fehlis (AMD) Capture

yanboliang commented 5 months ago

@yafehlis Can you run the following script to collect the sys env?

wget https://raw.githubusercontent.com/pytorch/pytorch/main/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py