terraform-google-modules / terraform-google-sql-db

Creates a Cloud SQL database instance
https://registry.terraform.io/modules/terraform-google-modules/sql-db/google
Apache License 2.0
265 stars 427 forks source link

backup stopped working because missing permission storage.objects.delete #599

Closed rverenich closed 6 months ago

rverenich commented 6 months ago

https://github.com/terraform-google-modules/terraform-google-sql-db/blob/fc37d6e6a7c37625ea95770d386e4b3033926926/modules/backup/main.tf#L140-L145

received today stopped working because this permission appeared to be needed

(previously role = "roles/storage.objectCreator" was enough)

  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
    "status": {
      "code": 7,
      "message": "gs://.../: Access denied for service account ...@gcp-sa-cloud-sql.iam.gserviceaccount.com, permissions missing are: [storage.objects.delete]"
    },
    "authenticationInfo": {
      "principalEmail": "...@...",
      "serviceAccountDelegationInfo": [
        {
          "firstPartyPrincipal": {
            "principalEmail": "...@..."
          }
        }
      ],
      "principalSubject": "serviceAccount:...@..."
    },
    "requestMetadata": {
      "callerIp": "...",
      "requestAttributes": {
        "time": "2024-04-24T04:30:00.879795Z",
        "auth": {}
      },
      "destinationAttributes": {}
    },
    "serviceName": "cloudsql.googleapis.com",
    "methodName": "cloudsql.instances.export",
    "authorizationInfo": [
      {
        "resource": "projects/...",
        "permission": "cloudsql.instances.export",
        "granted": true,
        "resourceAttributes": {
          "service": "sqladmin.googleapis.com",
          "name": "...",
          "type": "sqladmin.googleapis.com/Instance"
        },
        "permissionType": "DATA_READ"
      }
    ],
    "resourceName": "...",
    "request": {
      "instance": "...",
      "project": "...",
      "body": {
        "exportContext": {
          "uri": "gs://....sql.gz",
          "offload": false
        }
      },
      "@type": "type.googleapis.com/google.cloud.sql.v1.SqlInstancesExportRequest"
    }
  },
  "insertId": "...",
  "resource": {
    "type": "cloudsql_database",
    "labels": {
      "database_id": "...",
      "region": "europe-west1",
      "project_id": "..."
    }
  },
  "timestamp": "2024-04-24T04:30:00.705844Z",
  "severity": "ERROR",
  "logName": "projects/.../logs/cloudaudit.googleapis.com%2Fdata_access",
  "receiveTimestamp": "2024-04-24T04:30:01.565360563Z"
}
rverenich commented 6 months ago

https://www.googlecloudcommunity.com/gc/Databases/Export-with-gcloud-sql-export-but-service-role-with-storage/m-p/666282

To export data from Cloud SQL to Cloud Storage, the service account should have the roles/storage.legacyBucketWriter role on the bucket. This role allows writing objects to the bucket but doesn't allow listing them

Note: The roles/storage.legacyBucketWriter role is a legacy role and might not be recommended for all use cases. For more granular control, consider using roles like roles/storage.objectCreator or roles/storage.objectAdmin.

ps-occrp commented 6 months ago

Based on this export workflow does not delete exports in bucket, it only exports to bucket so this might be something else

rverenich commented 6 months ago

after a while, this workflow started woking again without permissions missing are: [storage.objects.delete] . This issue may be closed for now