search

NVIDIA / FasterTransformer

Transformer related optimization, including BERT, GPT
Apache License 2.0
5.87k stars 891 forks source link

Ada FP8 support? ( for GPT with FP8 (Experimental)) #442

Open oscarbg opened 1 year ago

oscarbg commented 1 year ago

Title says it all..

byshiue commented 1 year ago

Ada is not supported yet.

oscarbg commented 1 year ago

@byshiue with cuda 12.1 now publicily released Ada FP8 is supported.. should be enough, right? Edit: or been delayed to cuda 12.2?

masterkni6 commented 1 year ago

@byshiue with cuda 12.1 now publicily released Ada FP8 is supported.. should be enough, right? Edit: or been delayed to cuda 12.2?

Release notes for cuda 12.1 doesn't have any updates for the cublas library

mgrankin commented 1 year ago

CUDA 12.1 update 1, FP8 matrix multiples are now supported on Ada chips

https://forums.developer.nvidia.com/t/ada-geforce-rtx-4090-fp8-cublaslt-performance/250737