Open matthewchung74 opened 8 months ago
I am not sure if cuda 12.0 is yet supported. -> that's was the error says as well
I do see that, but when I do this
base) [ec2-user@ip-172-16-30-64 notebooks]$ nvidia-smi
Thu Oct 26 00:27:41 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.85.12 Driver Version: 525.85.12 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 |
it looks like coda 12.0 is installed
Thank you for the excellent Blogs!
When running https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/instruction-tune-llama-2-int4.ipynb
I am trying to enable flash attention in a Sagemaker Notebook using ml.g5.2xlarge and nvidia-smi tells me I am on CUDA Version: 12.0 but
gives this error
Is this something you've seen?