search

skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.54k stars 466 forks source link

[show-gpus] Show non-common GPUs by default when querying specific clouds #3828

Open romilbhardwaj opened 1 month ago

romilbhardwaj commented 1 month ago

Running sky show-gpus --cloud cudo shows only V100 as the option:

$ sky show-gpus --cloud cudo
COMMON_GPU  AVAILABLE_QUANTITIES
V100        1, 2, 4, 8

Hint: use -a/--all to see all accelerators (including non-common ones) and pricing.

Which makes me think V100 is the only supported GPUs. The hint to add -a is easy to mentally dismiss as a extra flag for showing detailed pricing.

We should show all GPUs (COMMON_GPU and OTHER_GPU) when running sky show-gpus for a specific cloud.

wizenheimer commented 1 week ago

Hey @romilbhardwaj, Attempted a fix for this here. Please have a look.

Before

sky show-gpus --cloud cudo
COMMON_GPU  AVAILABLE_QUANTITIES
V100        1, 2, 4, 8

Hint: use -a/--all to see all accelerators (including non-common ones) and pricing.

After

sky show-gpus --cloud cudo
COMMON_GPU  AVAILABLE_QUANTITIES
V100        1, 2, 4, 8

OTHER_GPU  AVAILABLE_QUANTITIES
A40        1, 2, 4, 8
RTX3080    1
RTXA4000   1, 2
RTXA4500   1, 2
RTXA5000   1, 2
RTXA6000   1, 2

GPU  QTY  CLOUD  INSTANCE_TYPE            DEVICE_MEM  vCPUs  HOST_MEM  HOURLY_PRICE  HOURLY_SPOT_PRICE  REGION
A40  1    Cudo   ice-lake-a40_4x1v2gb     48GB        -      4GB       $ 0.808       -                  se-stockholm-1
A40  1    Cudo   ice-lake-a40_8x1v4gb     48GB        -      8GB       $ 0.827       -                  se-stockholm-1
A40  1    Cudo   ice-lake-a40_24x1v12gb   48GB        -      24GB      $ 0.900       -                  se-stockholm-1
A40  2    Cudo   ice-lake-a40_16x2v8gb    48GB        -      16GB      $ 1.654       -                  se-stockholm-1
A40  2    Cudo   ice-lake-a40_32x2v16gb   48GB        -      32GB      $ 1.727       -                  se-stockholm-1
A40  2    Cudo   ice-lake-a40_48x2v24gb   48GB        -      48GB      $ 1.801       -                  se-stockholm-1
A40  4    Cudo   ice-lake-a40_64x4v32gb   48GB        -      64GB      $ 3.454       -                  se-stockholm-1
A40  4    Cudo   ice-lake-a40_96x4v48gb   48GB        -      96GB      $ 3.602       -                  se-stockholm-1
A40  8    Cudo   ice-lake-a40_128x8v64gb  48GB        -      128GB     $ 6.909       -                  se-stockholm-1
A40  8    Cudo   ice-lake-a40_192x8v96gb  48GB        -      192GB     $ 7.203       -                  se-stockholm-1

GPU      QTY  CLOUD  INSTANCE_TYPE                       DEVICE_MEM  vCPUs  HOST_MEM  HOURLY_PRICE  HOURLY_SPOT_PRICE  REGION
RTX3080  1    Cudo   intel-broadwell-rtx-3080_4x1v2gb    12GB        -      4GB       $ 0.082       -                  ca-montreal-1
RTX3080  1    Cudo   intel-broadwell-rtx-3080_8x1v4gb    12GB        -      8GB       $ 0.094       -                  ca-montreal-1
RTX3080  1    Cudo   intel-broadwell-rtx-3080_24x1v12gb  12GB        -      24GB      $ 0.143       -                  ca-montreal-1
RTX3080  1    Cudo   intel-broadwell-rtx-3080_48x1v24gb  12GB        -      48GB      $ 0.216       -                  ca-montreal-1

GPU       QTY  CLOUD  INSTANCE_TYPE                   DEVICE_MEM  vCPUs  HOST_MEM  HOURLY_PRICE  HOURLY_SPOT_PRICE  REGION
RTXA4000  1    Cudo   epyc-rome-rtx-a4000_4x1v2gb     16GB        -      4GB       $ 0.308       -                  no-luster-1
RTXA4000  1    Cudo   epyc-milan-rtx-a4000_4x1v2gb    16GB        -      4GB       $ 0.318       -                  se-smedjebacken-1
RTXA4000  1    Cudo   epyc-rome-rtx-a4000_8x1v4gb     16GB        -      8GB       $ 0.326       -                  no-luster-1
RTXA4000  1    Cudo   epyc-milan-rtx-a4000_8x1v4gb    16GB        -      8GB       $ 0.337       -                  se-smedjebacken-1
RTXA4000  1    Cudo   epyc-rome-rtx-a4000_24x1v12gb   16GB        -      24GB      $ 0.397       -                  no-luster-1
RTXA4000  1    Cudo   epyc-milan-rtx-a4000_24x1v12gb  16GB        -      24GB      $ 0.410       -                  se-smedjebacken-1
RTXA4000  1    Cudo   epyc-milan-rtx-a4000_48x1v24gb  16GB        -      48GB      $ 0.521       -                  se-smedjebacken-1
RTXA4000  2    Cudo   epyc-milan-rtx-a4000_16x2v8gb   16GB        -      16GB      $ 0.674       -                  se-smedjebacken-1
RTXA4000  2    Cudo   epyc-milan-rtx-a4000_32x2v16gb  16GB        -      32GB      $ 0.747       -                  se-smedjebacken-1
RTXA4000  2    Cudo   epyc-milan-rtx-a4000_48x2v24gb  16GB        -      48GB      $ 0.821       -                  se-smedjebacken-1
RTXA4000  2    Cudo   epyc-milan-rtx-a4000_96x2v48gb  16GB        -      96GB      $ 1.042       -                  se-smedjebacken-1

GPU       QTY  CLOUD  INSTANCE_TYPE             DEVICE_MEM  vCPUs  HOST_MEM  HOURLY_PRICE  HOURLY_SPOT_PRICE  REGION
RTXA4500  1    Cudo   sky-lake-a4500_4x1v2gb    20GB        -      4GB       $ 0.478       -                  gb-london-1
RTXA4500  1    Cudo   sky-lake-a4500_8x1v4gb    20GB        -      8GB       $ 0.495       -                  gb-london-1
RTXA4500  1    Cudo   sky-lake-a4500_24x1v12gb  20GB        -      24GB      $ 0.566       -                  gb-london-1
RTXA4500  1    Cudo   sky-lake-a4500_48x1v24gb  20GB        -      48GB      $ 0.671       -                  gb-london-1
RTXA4500  2    Cudo   sky-lake-a4500_16x2v8gb   20GB        -      16GB      $ 0.990       -                  gb-london-1
RTXA4500  2    Cudo   sky-lake-a4500_32x2v16gb  20GB        -      32GB      $ 1.061       -                  gb-london-1
RTXA4500  2    Cudo   sky-lake-a4500_48x2v24gb  20GB        -      48GB      $ 1.131       -                  gb-london-1
RTXA4500  2    Cudo   sky-lake-a4500_96x2v48gb  20GB        -      96GB      $ 1.342       -                  gb-london-1

GPU       QTY  CLOUD  INSTANCE_TYPE                   DEVICE_MEM  vCPUs  HOST_MEM  HOURLY_PRICE  HOURLY_SPOT_PRICE  REGION
RTXA5000  1    Cudo   epyc-milan-rtx-a5000_4x1v2gb    24GB        -      4GB       $ 0.568       -                  se-smedjebacken-1
RTXA5000  1    Cudo   epyc-milan-rtx-a5000_8x1v4gb    24GB        -      8GB       $ 0.587       -                  se-smedjebacken-1
RTXA5000  1    Cudo   epyc-milan-rtx-a5000_24x1v12gb  24GB        -      24GB      $ 0.660       -                  se-smedjebacken-1
RTXA5000  1    Cudo   epyc-milan-rtx-a5000_48x1v24gb  24GB        -      48GB      $ 0.771       -                  se-smedjebacken-1
RTXA5000  2    Cudo   epyc-milan-rtx-a5000_16x2v8gb   24GB        -      16GB      $ 1.174       -                  se-smedjebacken-1
RTXA5000  2    Cudo   epyc-milan-rtx-a5000_32x2v16gb  24GB        -      32GB      $ 1.247       -                  se-smedjebacken-1
RTXA5000  2    Cudo   epyc-milan-rtx-a5000_48x2v24gb  24GB        -      48GB      $ 1.321       -                  se-smedjebacken-1
RTXA5000  2    Cudo   epyc-milan-rtx-a5000_96x2v48gb  24GB        -      96GB      $ 1.542       -                  se-smedjebacken-1

GPU       QTY  CLOUD  INSTANCE_TYPE                  DEVICE_MEM  vCPUs  HOST_MEM  HOURLY_PRICE  HOURLY_SPOT_PRICE  REGION
RTXA6000  1    Cudo   epyc-rome-rtx-a6000_4x1v2gb    48GB        -      4GB       $ 0.798       -                  no-luster-1
RTXA6000  1    Cudo   epyc-rome-rtx-a6000_8x1v4gb    48GB        -      8GB       $ 0.816       -                  no-luster-1
RTXA6000  1    Cudo   epyc-rome-rtx-a6000_24x1v12gb  48GB        -      24GB      $ 0.887       -                  no-luster-1
RTXA6000  1    Cudo   epyc-rome-rtx-a6000_48x1v24gb  48GB        -      48GB      $ 0.994       -                  no-luster-1
RTXA6000  2    Cudo   epyc-rome-rtx-a6000_16x2v8gb   48GB        -      16GB      $ 1.631       -                  no-luster-1
RTXA6000  2    Cudo   epyc-rome-rtx-a6000_32x2v16gb  48GB        -      32GB      $ 1.702       -                  no-luster-1
RTXA6000  2    Cudo   epyc-rome-rtx-a6000_48x2v24gb  48GB        -      48GB      $ 1.774       -                  no-luster-1
RTXA6000  2    Cudo   epyc-rome-rtx-a6000_96x2v48gb  48GB        -      96GB      $ 1.987       -                  no-luster-1

GPU   QTY  CLOUD  INSTANCE_TYPE                    DEVICE_MEM  vCPUs  HOST_MEM  HOURLY_PRICE  HOURLY_SPOT_PRICE  REGION
V100  1    Cudo   intel-broadwell-v100_4x1v2gb     16GB        -      4GB       $ 1.008       -                  us-santaclara-1
V100  1    Cudo   intel-broadwell-v100_8x1v4gb     16GB        -      8GB       $ 1.027       -                  us-santaclara-1
V100  1    Cudo   intel-broadwell-v100_24x1v12gb   16GB        -      24GB      $ 1.100       -                  us-santaclara-1
V100  2    Cudo   intel-broadwell-v100_16x2v8gb    16GB        -      16GB      $ 2.054       -                  us-santaclara-1
V100  2    Cudo   intel-broadwell-v100_32x2v16gb   16GB        -      32GB      $ 2.127       -                  us-santaclara-1
V100  2    Cudo   intel-broadwell-v100_48x2v24gb   16GB        -      48GB      $ 2.201       -                  us-santaclara-1
V100  4    Cudo   intel-broadwell-v100_64x4v32gb   16GB        -      64GB      $ 4.254       -                  us-santaclara-1
V100  4    Cudo   intel-broadwell-v100_96x4v48gb   16GB        -      96GB      $ 4.402       -                  us-santaclara-1
V100  8    Cudo   intel-broadwell-v100_128x8v64gb  16GB        -      128GB     $ 8.509       -                  us-santaclara-1
V100  8    Cudo   intel-broadwell-v100_192x8v96gb  16GB        -      192GB     $ 8.803       -                  us-santaclara-1