Update PyTorch to 2.2 - Githubissues

MDK8888 / GPTFast

Accelerate your Hugging Face Transformers 7.6-9x. Native to Hugging Face and PyTorch.

Apache License 2.0

677 stars 64 forks source link

Update PyTorch to 2.2 #12

Closed AndreaPi closed 4 months ago

AndreaPi commented 6 months ago

Very interesting work! I see you pinned torch==2.1.2 . PyTorch 2.2 promises a 2x improvement to scaled_dot_product_attention and a few torch.compile improvements:

https://pytorch.org/blog/pytorch2-2/

I don't think using PyTorch 2.2 will result in a big speed increase for GPTFast, but still it may be nice to try. What do you think?

MDK8888 commented 6 months ago

I definitely agree! Apologies for the late response-I will probably upgrade it in the next version, which will be out soon.

MDK8888 commented 4 months ago

Hey, the new version does indeed have torch==2.2! Closing this now :)