The latest pytorch 2.1.0 requires CUDA 12.1 by default. This is to update the images for both AWS and GCP to support the latest cuda driver.
We should get https://github.com/skypilot-org/skypilot/pull/2788 in at the same time.
Tested:
[x] sky launch serve-openai-api.yaml -c vllm-dbg3 --env HF_TOKEN -i30 --down without rolling back the vllm commit
[x] sky launch serve-openai-api.yaml -c vllm-dbg-aws2 --gpus A10g --env HF_TOKEN -i30 --down without rolling back the vllm commit
[x] sky launch --cloud gcp --gpus L4 pip install torch==1.12.0+cu116 torchvision==0.13.0+cu116 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu116 cuda has good backward compatibility
The latest pytorch 2.1.0 requires CUDA 12.1 by default. This is to update the images for both AWS and GCP to support the latest cuda driver. We should get https://github.com/skypilot-org/skypilot/pull/2788 in at the same time.
Tested:
sky launch serve-openai-api.yaml -c vllm-dbg3 --env HF_TOKEN -i30 --downwithout rolling back the vllm commitsky launch serve-openai-api.yaml -c vllm-dbg-aws2 --gpus A10g --env HF_TOKEN -i30 --downwithout rolling back the vllm commitsky launch --cloud gcp --gpus L4 pip install torch==1.12.0+cu116 torchvision==0.13.0+cu116 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu116cuda has good backward compatibility