bentoml / bentoctl

Fast model deployment on any cloud 🚀
https://bentoml.com
Other
175 stars 30 forks source link

Create machines with GPU using bentoctl Google Compute Engine Operator (new template) #188

Open math-sasso opened 1 year ago

math-sasso commented 1 year ago

I am trying to create a machine with GPU using bentoctl with Google Cloud Engine Operator but I believe the way terraform calls docker run does not consider the --gpus all tag. I think that the solution should be in the module "gce-container" inside the terraform template. The code could be found below:

module "gce-container" {
  # https://registry.terraform.io/modules/terraform-google-modules/container-vm/google/latest
  source         = "terraform-google-modules/container-vm/google"
  cos_image_name = "cos-stable-77-12371-89-0"
  container = {
    image = "${data.google_container_registry_image.bento_service.image_url}:${var.image_version}"
    env = [
      {
        name  = "BENTOML_PORT"
        value = "3000"
      },
    ]
  }

  restart_policy = "Always"
}

To verify if a GPU really exists in the GCP Virtual Machine I am printing a counter using Tensorflow:

print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))

This is related to this terraform documentation below, but I dont see any GPU related param.