Closed mwarkentin closed 1 month ago
I manually unlocked the slices I've seen this on so far:
iam/salt/us-replay on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) via 💠 default on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/google versions matching "4.72.0"...
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentryio-terraform/regions/multi-tenant/iam/salt/us-replay/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514899941359
│ Path: gs://sentryio-terraform/regions/multi-tenant/iam/salt/us-replay/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:19.74162501 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0014] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-replay/.terragrunt-cache/F8LsNaWO0OGE7Haf9rqWGDGBCsU/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-replay]
ERRO[0014] 1 error occurred:
* exit status 1
iam/salt/us-replay on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) via 💠 default on ☁️ michael@sentry.io(us-central1) took 14s
❯ tg force-unlock 1714514899941359
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-replay]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0004] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-replay]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624416901301894350
[WARN] - (starship::utils): Executing command "/opt/homebrew/bin/git" timed out.
[WARN] - (starship::utils): You can set command_timeout in your config to a higher value to allow longer-running commands to keep executing.
iam/salt/us-replay on OPS-6598-autogrow-permission-all-projects [$] via 🐍 v3.11.7 (.ops-venv) via 💠 default on ☁️ michael@sentry.io(us-central1) took 7s
❯ ..
multi-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd us-meps
iam/salt/us-meps on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) via 💠 default on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentryio-terraform/regions/multi-tenant/iam/salt/us-meps/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514893864749
│ Path: gs://sentryio-terraform/regions/multi-tenant/iam/salt/us-meps/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:13.770326922 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0003] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-meps/.terragrunt-cache/hjP3-bWuSFp9xpGsVjWezWvqt_c/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-meps]
ERRO[0003] 1 error occurred:
* exit status 1
iam/salt/us-meps on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) via 💠 default on ☁️ michael@sentry.io(us-central1) took 3s
❯ tg force-unlock 1714514893864749
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-meps]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0002] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/us-meps]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624417701356578331
iam/salt/us-meps on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) via 💠 default on ☁️ michael@sentry.io(us-central1) took 3s
❯ ..
multi-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 8
drwxr-xr-x 7 mwarkentin staff 224B 17 Jan 09:12 ./
drwxr-xr-x 15 mwarkentin staff 480B 12 Jun 08:56 ../
drwxr-xr-x 5 mwarkentin staff 160B 18 Jun 07:59 de/
-rw-r--r-- 1 mwarkentin staff 114B 26 Oct 2023 service.hcl
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:59 us/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:59 us-meps/
drwxr-xr-x 7 mwarkentin staff 224B 18 Jun 08:14 us-replay/
multi-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd de
iam/salt/de on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-eu-west3-terraform/regions/multi-tenant/iam/salt/de/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514897134834
│ Path: gs://sentry-eu-west3-terraform/regions/multi-tenant/iam/salt/de/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:16.803535684 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0004] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/de/.terragrunt-cache/hm_u3_aQyWPukuT0CH_Lbu4V8EE/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/de]
ERRO[0004] 1 error occurred:
* exit status 1
iam/salt/de on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 4s
❯ tg force-unlock 1714514897134834
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/de]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0004] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/multi-tenant/iam/salt/de]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624418041974120366
iam/salt/de on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 5s
❯ ..
multi-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 8
drwxr-xr-x 7 mwarkentin staff 224B 17 Jan 09:12 ./
drwxr-xr-x 15 mwarkentin staff 480B 12 Jun 08:56 ../
drwxr-xr-x 5 mwarkentin staff 160B 18 Jun 07:59 de/
-rw-r--r-- 1 mwarkentin staff 114B 26 Oct 2023 service.hcl
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:59 us/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:59 us-meps/
drwxr-xr-x 7 mwarkentin staff 224B 18 Jun 08:14 us-replay/
multi-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ..
regions/multi-tenant/iam on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ..
terragrunt/regions/multi-tenant on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ..
ops/terragrunt/regions on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd single-tenant
terragrunt/regions/single-tenant on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 80
drwxr-xr-x 55 mwarkentin staff 1.7K 18 Jun 07:28 ./
drwxr-xr-x 6 mwarkentin staff 192B 12 Jun 08:56 ../
drwxr-xr-x 5 mwarkentin staff 160B 24 Jul 2023 _template/
drwxr-xr-x 3 mwarkentin staff 96B 2 Nov 2023 alloydb/
drwxr-xr-x 3 mwarkentin staff 96B 12 Jun 08:56 bigquery/
drwxr-xr-x 13 mwarkentin staff 416B 12 Jun 08:56 buckets/
drwxr-xr-x 13 mwarkentin staff 416B 12 Jun 08:56 cloudarmor/
-rwxr-xr-x 1 mwarkentin staff 374B 12 Jun 08:56 create-customer-service.sh*
drwxr-xr-x 5 mwarkentin staff 160B 12 Jun 08:56 db/
-rw-r--r-- 1 mwarkentin staff 872B 12 Jun 08:56 disney.hcl
drwxr-xr-x 5 mwarkentin staff 160B 27 Nov 2023 dns/
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 gce-notifier/
-rw-r--r-- 1 mwarkentin staff 870B 12 Jun 08:56 geico.hcl
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 geoipupdate/
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 gke-audit-log-sink/
-rw-r--r-- 1 mwarkentin staff 877B 12 Jun 08:56 goldmansachs.hcl
drwxr-xr-x 9 mwarkentin staff 288B 12 Jun 14:38 iam/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 indexer-memcache/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 ingest-v4/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 ingest-v6/
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 kafka/
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 kafka-zookeeper-shared/
drwxr-xr-x 4 mwarkentin staff 128B 12 Jun 08:56 kafka-zookeeper-test/
drwxr-xr-x 9 mwarkentin staff 288B 12 Jun 08:56 keda/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 legacy/
-rw-r--r-- 1 mwarkentin staff 888B 12 Jun 08:56 ly.hcl
drwxr-xr-x 4 mwarkentin staff 128B 27 Nov 2023 memcached/
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 mesh/
drwxr-xr-x 3 mwarkentin staff 96B 6 Oct 2023 private-service-connect-consume/
drwxr-xr-x 4 mwarkentin staff 128B 25 Sep 2023 private-service-connect-publish/
drwxr-xr-x 12 mwarkentin staff 384B 12 Jun 08:56 project-services/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 rabbitmq/
drwxr-xr-x 7 mwarkentin staff 224B 24 May 12:50 redis-memorystore/
drwxr-xr-x 5 mwarkentin staff 160B 12 Jun 08:56 relocation-kms/
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 routes/
-rw-r--r-- 1 mwarkentin staff 934B 12 Jun 08:56 s4s.hcl
drwxr-xr-x 12 mwarkentin staff 384B 12 Jun 08:56 scratch/
drwxr-xr-x 12 mwarkentin staff 384B 12 Jun 08:56 secrets/
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 snapshot-schedule/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 snuba-generic-metrics/
drwxr-xr-x 5 mwarkentin staff 160B 18 Jun 07:28 snuba-generic-metrics-hyper/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 snuba-metrics/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 snuba-query/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 snuba-query-legacy/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 snuba-shared/
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 snuba-st/
drwxr-xr-x 5 mwarkentin staff 160B 12 Jun 08:56 snuba-test/
drwxr-xr-x 6 mwarkentin staff 192B 31 Aug 2023 symbolicator_ilb/
-rw-r--r-- 1 mwarkentin staff 100B 31 Aug 2023 tenancy.hcl
-rw-r--r-- 1 mwarkentin staff 439B 16 Oct 2023 test-control.hcl
-rw-r--r-- 1 mwarkentin staff 532B 16 Oct 2023 test-region.hcl
drwxr-xr-x 4 mwarkentin staff 128B 12 Jun 08:56 topicctl-configs/
-rw-r--r-- 1 mwarkentin staff 887B 12 Jun 08:56 zendesk-eu.hcl
drwxr-xr-x 10 mwarkentin staff 320B 12 Jun 08:56 zookeeper-clickhouse/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 13:14 zookeeper-test/
terragrunt/regions/single-tenant on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd iam/salt
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 8
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 ./
drwxr-xr-x 9 mwarkentin staff 288B 12 Jun 14:38 ../
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 disney/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 geico/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 goldmansachs/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 ly/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:38 s4s/
-rw-r--r-- 1 mwarkentin staff 258B 26 Oct 2023 service.hcl
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 test-control/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 test-region/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 zendesk-eu/
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd test-region
iam/salt/test-region on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/google versions matching "4.72.0"...
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-test-region-terraform/regions/single-tenant/iam/salt/test-region/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514903446575
│ Path: gs://sentry-test-region-terraform/regions/single-tenant/iam/salt/test-region/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:23.120490128 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0014] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-region/.terragrunt-cache/bVsiFbSKEqOrG6DdDbP1qhKn8Zs/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-region]
ERRO[0014] 1 error occurred:
* exit status 1
iam/salt/test-region on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 14s
❯ tg force-unlock 1714514903446575
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-region]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0006] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-region]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624419563443192941
iam/salt/test-region on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 8s
❯ ..
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd test-control
iam/salt/test-control on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Finding hashicorp/google versions matching "4.72.0"...
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-test-control-terraform/regions/single-tenant/iam/salt/test-control/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514908342418
│ Path: gs://sentry-test-control-terraform/regions/single-tenant/iam/salt/test-control/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:28.261160596 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0012] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-control/.terragrunt-cache/aXLTWLD_f9sLg5OSIgH_69h-JMw/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-control]
ERRO[0012] 1 error occurred:
* exit status 1
iam/salt/test-control on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 12s
❯ tg force-unlock 1714514908342418
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-control]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0003] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/test-control]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624420119941601229
iam/salt/test-control on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 4s
❯ ..
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 8
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 ./
drwxr-xr-x 9 mwarkentin staff 288B 12 Jun 14:38 ../
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 disney/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 geico/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 goldmansachs/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 ly/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:38 s4s/
-rw-r--r-- 1 mwarkentin staff 258B 26 Oct 2023 service.hcl
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:28 test-control/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:27 test-region/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 zendesk-eu/
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd zendesk-eu
iam/salt/zendesk-eu on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/google versions matching "4.72.0"...
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-st-zendesk-eu-terraform/regions/single-tenant/iam/salt/zendesk-eu/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514904758268
│ Path: gs://sentry-st-zendesk-eu-terraform/regions/single-tenant/iam/salt/zendesk-eu/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:24.30327188 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0014] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/zendesk-eu/.terragrunt-cache/qGA-u1feFg04iJ7foupf9SG0fgk/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/zendesk-eu]
ERRO[0014] 1 error occurred:
* exit status 1
iam/salt/zendesk-eu on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 14s
❯ tg force-unlock 1714514904758268
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/zendesk-eu]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0003] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/zendesk-eu]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624420641427242607
iam/salt/zendesk-eu on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 5s
❯ ..
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 8
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 ./
drwxr-xr-x 9 mwarkentin staff 288B 12 Jun 14:38 ../
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 disney/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 geico/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 goldmansachs/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 ly/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:38 s4s/
-rw-r--r-- 1 mwarkentin staff 258B 26 Oct 2023 service.hcl
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:28 test-control/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:27 test-region/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:28 zendesk-eu/
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd s4s/
iam/salt/s4s on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://mattrobenolt-kube-terraform-ho6js/regions/single-tenant/iam/salt/s4s/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514906933764
│ Path: gs://mattrobenolt-kube-terraform-ho6js/regions/single-tenant/iam/salt/s4s/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:26.763998941 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0004] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/s4s/.terragrunt-cache/KvkDu7HHhmX5fIF5WPCd8jtLZqs/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/s4s]
ERRO[0004] 1 error occurred:
* exit status 1
iam/salt/s4s on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 4s
❯ tg force-unlock 1714514906933764
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/s4s]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0004] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/s4s]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624421049762941606
iam/salt/s4s on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 5s
❯ cd ../
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 8
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 ./
drwxr-xr-x 9 mwarkentin staff 288B 12 Jun 14:38 ../
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 disney/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 geico/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 goldmansachs/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 ly/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:38 s4s/
-rw-r--r-- 1 mwarkentin staff 258B 26 Oct 2023 service.hcl
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:28 test-control/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:27 test-region/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:28 zendesk-eu/
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd ly
iam/salt/ly on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/google versions matching "4.72.0"...
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-st-ly-terraform/regions/single-tenant/iam/salt/ly/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1718712106720245
│ Path: gs://sentry-st-ly-terraform/regions/single-tenant/iam/salt/ly/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: mwarkentin@11021.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-06-18 12:01:46.21047154 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0017] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/ly/.terragrunt-cache/7mPG1nAod14qDVIE6hgKyBXdBDE/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/ly]
ERRO[0017] 1 error occurred:
* exit status 1
iam/salt/ly on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 17s
❯ tg force-unlock 1718712106720245
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/ly]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0005] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/ly]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624428421267658559
iam/salt/ly on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 7s
❯ ..
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ ll
total 8
drwxr-xr-x 11 mwarkentin staff 352B 12 Jun 08:56 ./
drwxr-xr-x 9 mwarkentin staff 288B 12 Jun 14:38 ../
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 disney/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 geico/
drwxr-xr-x 4 mwarkentin staff 128B 14 Jun 08:50 goldmansachs/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:29 ly/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 07:38 s4s/
-rw-r--r-- 1 mwarkentin staff 258B 26 Oct 2023 service.hcl
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:28 test-control/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:27 test-region/
drwxr-xr-x 6 mwarkentin staff 192B 18 Jun 08:28 zendesk-eu/
single-tenant/iam/salt on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ cd goldmansachs/
iam/salt/goldmansachs on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Finding hashicorp/google versions matching "4.72.0"...
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-st-goldmansachs-terraform/regions/single-tenant/iam/salt/goldmansachs/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514904884335
│ Path: gs://sentry-st-goldmansachs-terraform/regions/single-tenant/iam/salt/goldmansachs/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:24.82943047 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0013] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/goldmansachs/.terragrunt-cache/eebfYODCohOrN9EMSP7QrYcBKQo/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/goldmansachs]
ERRO[0013] 1 error occurred:
* exit status 1
iam/salt/goldmansachs on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 13s
❯ tg force-unlock 1714514904884335
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/goldmansachs]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0003] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/goldmansachs]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624428904975431473
iam/salt/goldmansachs on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 5s
❯ cd ../geico
iam/salt/geico on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plan
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Finding hashicorp/google versions matching "4.72.0"...
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-st-geico-terraform/regions/single-tenant/iam/salt/geico/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514897837437
│ Path: gs://sentry-st-geico-terraform/regions/single-tenant/iam/salt/geico/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:17.750196017 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0012] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/geico/.terragrunt-cache/-K6z8mulXzsYMT43_mSDBVUXuw8/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/geico]
ERRO[0012] 1 error occurred:
* exit status 1
iam/salt/geico on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 12s
❯ tg force-unlock 1714514897837437
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/geico]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0003] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/geico]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624429350923184329
iam/salt/geico on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 4s
❯ cd ../disney
iam/salt/disney on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1)
❯ tg plna
Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_clickhouse...
- salt_clickhouse in .terraform/modules/salt_clickhouse/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_consul...
- salt_consul in .terraform/modules/salt_consul/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_database...
- salt_database in .terraform/modules/salt_database/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_kafka...
- salt_kafka in .terraform/modules/salt_kafka/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_nat...
- salt_nat in .terraform/modules/salt_nat/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_rabbitmq...
- salt_rabbitmq in .terraform/modules/salt_rabbitmq/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_redis_cluster...
- salt_redis_cluster in .terraform/modules/salt_redis_cluster/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_service_accounts...
- salt_service_accounts in .terraform/modules/salt_service_accounts/service_account_basic
Downloading git::ssh://git@github.com/getsentry/terraform-modules.git?ref=main for salt_zookeeper...
- salt_zookeeper in .terraform/modules/salt_zookeeper/service_account_basic
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/google-beta versions matching "4.72.0"...
- Finding hashicorp/random versions matching "~> 3.1.0"...
- Finding hashicorp/google versions matching "4.72.0"...
- Installing hashicorp/random v3.1.3...
- Installed hashicorp/random v3.1.3 (signed by HashiCorp)
- Installing hashicorp/google v4.72.0...
- Installed hashicorp/google v4.72.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v4.72.0...
- Installed hashicorp/google-beta v4.72.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Terraform has no command named "plna". Did you mean "plan"?
To see all of Terraform's top-level commands, run:
terraform -help
ERRO[0010] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/disney/.terragrunt-cache/CtnI_W5qDPgK7N2KO3ZhWJImWSw/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/disney]
ERRO[0010] 1 error occurred:
* exit status 1
iam/salt/disney on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 11s
❯ tg plan
╷
│ Error: Error acquiring the state lock
│
│ Error message: writing
│ "gs://sentry-st-disney-terraform/regions/single-tenant/iam/salt/disney/terraform.tfstate/default.tflock"
│ failed: googleapi: Error 412: At least one of the pre-conditions you
│ specified did not hold., conditionNotMet
│ Lock Info:
│ ID: 1714514901844718
│ Path: gs://sentry-st-disney-terraform/regions/single-tenant/iam/salt/disney/terraform.tfstate/default.tflock
│ Operation: OperationTypeInvalid
│ Who: maxwelllwang@10279.ops.getsentry.github.invalid
│ Version: 1.5.3
│ Created: 2024-04-30 22:08:21.684643411 +0000 UTC
│ Info:
│
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵
ERRO[0002] Terraform invocation failed in /Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/disney/.terragrunt-cache/CtnI_W5qDPgK7N2KO3ZhWJImWSw/0rghWROp0gJdHK4BUmBouoAzp4g/iam/salt prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/disney]
ERRO[0002] 1 error occurred:
* exit status 1
iam/salt/disney on OPS-6598-autogrow-permission-all-projects [$?] via 🐍 v3.11.7 (.ops-venv) on ☁️ michael@sentry.io(us-central1) took 2s
❯ tg force-unlock 1714514901844718
INFO[0000] Executing hook: verify_sentry_kube prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/disney]
/Users/mwarkentin/repos/sentry/ops/.ops-venv/bin/sentry-kube
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may still be in use. Only 'yes' will be accepted to confirm.
Enter a value: yes
Terraform state has been successfully unlocked!
The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.
INFO[0003] Executing hook: log_to_datadog prefix=[/Users/mwarkentin/repos/sentry/ops/terragrunt/regions/single-tenant/iam/salt/disney]
Reported the action to DataDog events:
https://app.datadoghq.com/event/event?id=7624429938474110352
I'm just going to assume that all unlock related issues are somehow this issue now
https://github.com/getsentry/ops/pull/11709#issuecomment-2263173846
Environment
https://github.com/getsentry/ops/pull/10930#issuecomment-2163030298 https://github.com/getsentry/ops/pull/10279#issuecomment-2163030298
Steps to Reproduce
iam/salt/s4s
sliceExpected Result
Thought TACOS would plan my change
Actual Result
Error regarding existing lock, however the PR it references is closed: https://github.com/getsentry/ops/pull/10279#issuecomment-2163030298
https://github.com/getsentry/ops/actions/runs/9470624684