iterative / cml

♾️ CML - Continuous Machine Learning | CI/CD for ML
http://cml.dev
Apache License 2.0
3.99k stars 333 forks source link

How to setup multiple GPU for CML runner in GCP #1430

Closed RTae closed 9 months ago

RTae commented 10 months ago

Summary / Background

I would like to use multiple GPU to training my model using CML runner but I could't find a method to set number of GPU. Moreover, when I select a instance with 2 GPU the TPI will rise error about should set number of accelerator to two, but it does't have a argument to set it

Scope

CptCaptain commented 9 months ago

In case you haven't solved it yet, I create my runners like this:

          cml runner launch \
              --cloud=gcp \
              --cloud-type=n1-highmem-2+nvidia-tesla-t4*2 \
              --labels=cml-gpu

That allows for specifiyng any viable number of GPUs.

RTae commented 9 months ago

@CptCaptain Thank you, it work !!