1-org: Apply fails for scc resources with API version error.

zexoor commented 7 months ago

TL;DR

When running an initial deployment for cloud-build. During the deployment of SCC resources for CAI monitoring terraform fails.

Error: Error creating NotificationConfig: googleapi: Error 400: This API is no longer available. Please use API V2.

  with google_scc_notification_config.scc_notification_config,
  on scc_notification.tf line 32, in resource "google_scc_notification_config" "scc_notification_config":
  32: resource "google_scc_notification_config" "scc_notification_config" {

Error: Error creating Source: googleapi: Error 400: This API is no longer available. Please use API V2.

  with module.cai_monitoring.google_scc_source.cai_monitoring,
  on ../../modules/cai-monitoring/main.tf line 136, in resource "google_scc_source" "cai_monitoring":
 136: resource "google_scc_source" "cai_monitoring" {

Expected behavior

Expected behaviour is for the resources to be created as intended

Observed behavior

Resource creation failed due to the API not being supported.

Terraform Configuration

/**
 * Copyright 2021 Google LLC
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

# Must include the domain of the organization you are deploying the foundation.
domains_to_allow = ["example.co.uk"]

essential_contacts_domains_to_allow = ["@example.co.uk"]

scc_notification_name = "scc-notify"

remote_state_bucket = "remote-state-bucket"

//scc_notification_filter = "state=\\\"ACTIVE\\\""

//enable_hub_and_spoke = true

//create_access_context_manager_access_policy = false

// Optional - If you are deploying Foundation Example in a parent folder
// consider using below create_unique_tag_key var because as Tag Keys are
// unique organization-wide it will add a random suffix at each tag key

//create_unique_tag_key = true

Terraform Version

Terraform v1.3.0
on darwin_arm64

Additional information

I have deployed standard tier SCC in the Organization as per https://github.com/umbrl-limited/umbrl-infra/blob/main/1-org/README.md#prerequisites

fmichaelobrien commented 7 months ago

I have seen a similar SCC API error in a 1st run through the TEF - a fix and a 2nd run were ok

try adding directly to the tf service account as a temporary workaround until a check on the roles added during sa creation are tested

account sa-terraform-bootstrap@seed... is used for steps 1+ add roles/securitycenter.admin

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID}
ERROR: (gcloud.scc.notifications.describe) INVALID_ARGUMENT: Security Command Center Legacy has been permanently disabled as of June 7, 2021. Migrate to Security Command Center's Standard tier or Premium tier to maintain access to Security Command Center. See https://cloud.google.com/security-command-center/docs/quickstart-security-command-center for more info.

https://github.com/terraform-google-modules/terraform-example-foundation/issues/1145

on a 2nd run after enabling roles/securitycenterAdmin on the super admin running the deployment see PR chang https://github.com/terraform-google-modules/terraform-example-foundation/pull/1175/files#diff-510bd7bb34f359f116d7029a41db09abb222834e57c39feb56e2acf6f74be8eeR86

although this sa is only used for 0-bootstrap, as sa-terraform-bootstrap@seed... is used for steps 1+

check SCC

michael@cloudshell:~/tef-olxyz/github/gcp-org (tef-olxyz)$ mv ./envs/shared/terraform.example.tfvars ./envs/shared/terraform.tfvars
michael@cloudshell:~/tef-olxyz/github/gcp-org (tef-olxyz)$ export ORGANIZATION_ID=$(terraform -chdir="../pbmm-on-gcp-onboarding/0-bootstrap/" output -json common_config | jq '.org_id' --raw-output)
michael@cloudshell:~/tef-olxyz/github/gcp-org (tef-olxyz)$ gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID}
ERROR: (gcloud.scc.notifications.describe) NOT_FOUND: Requested entity was not found.

Deployment of 1-org was ok

Step #2 - "tf plan": Plan: 288 to add, 0 to change, 0 to destroy.

Step #4 - "tf apply": Apply complete! Resources: 288 added, 0 changed, 0 destroyed.
Step #4 - "tf apply": 
Step #4 - "tf apply": Outputs:
Step #4 - "tf apply": 
Step #4 - "tf apply": base_net_hub_project_id = "prj-c-base-net-hub-5y8h"
Step #4 - "tf apply": billing_sink_names = {
Step #4 - "tf apply":   "prj" = "sk-c-logging-prj-billing-wh58"
Step #4 - "tf apply":   "pub" = "sk-c-logging-pub-billing-wh58"
Step #4 - "tf apply":   "sto" = "sk-c-logging-bkt-billing-wh58"
Step #4 - "tf apply": }
Step #4 - "tf apply": cai_monitoring_artifact_registry = "ar-cai-monitoring-4241"
Step #4 - "tf apply": cai_monitoring_asset_feed = "organizations/1064386348915/feeds/fd-cai-monitoring-4241"
Step #4 - "tf apply": cai_monitoring_bucket = "bkt-cai-monitoring-4241-sources-726972909649-us-central1"
Step #4 - "tf apply": cai_monitoring_topic = "top-cai-monitoring-4241-event"
Step #4 - "tf apply": common_folder_name = "folders/96486704059"
Step #4 - "tf apply": dns_hub_project_id = "prj-c-dns-hub-6f4b"
Step #4 - "tf apply": domains_to_allow = tolist([
Step #4 - "tf apply":   "obrienlabs.xyz",
Step #4 - "tf apply": ])
Step #4 - "tf apply": interconnect_project_id = "prj-c-interconnect-s2zg"
Step #4 - "tf apply": interconnect_project_number = "6710...54"
Step #4 - "tf apply": logs_export_project_linked_dataset_name = "projects/prj-c-logging-fn0h/locations/us-central1/buckets/AggregatedLogs/links/ds_c_prj_aggregated_logs_analytics"
Step #4 - "tf apply": logs_export_project_logbucket_name = "AggregatedLogs"
Step #4 - "tf apply": logs_export_pubsub_topic = "tp-org-logs-o9q2"
Step #4 - "tf apply": logs_export_storage_bucket_name = "bkt-prj-c-logging-fn0h-org-logs-o9q2"
Step #4 - "tf apply": network_folder_name = "folders/65...643"
Step #4 - "tf apply": org_audit_logs_project_id = "prj-c-logging-fn0h"
Step #4 - "tf apply": org_billing_logs_project_id = "prj-c-billing-logs-ve1w"
Step #4 - "tf apply": org_id = "10...5"
Step #4 - "tf apply": org_kms_project_id = "prj-c-kms-eeg3"
Step #4 - "tf apply": org_secrets_project_id = "prj-c-secrets-2lxo"
Step #4 - "tf apply": parent_resource_id = "73...67"
Step #4 - "tf apply": parent_resource_type = "folder"
Step #4 - "tf apply": restricted_net_hub_project_id = "prj-c-restricted-net-hub-a8d5"
Step #4 - "tf apply": restricted_net_hub_project_number = "12...62"
Step #4 - "tf apply": scc_notification_name = "scc-notify"
Step #4 - "tf apply": scc_notifications_project_id = "prj-c-scc-8zsj"
Step #4 - "tf apply": shared_vpc_projects = {
Step #4 - "tf apply":   "development" = {
Step #4 - "tf apply":     "base_shared_vpc_project_id" = "prj-d-shared-base-nlqs"
Step #4 - "tf apply":     "base_shared_vpc_project_number" = "10...505"
Step #4 - "tf apply":     "restricted_shared_vpc_project_id" = "prj-d-shared-restricted-j004"
Step #4 - "tf apply":     "restricted_shared_vpc_project_number" = "10..921"
Step #4 - "tf apply":   }
Step #4 - "tf apply":   "non-production" = {
Step #4 - "tf apply":     "base_shared_vpc_project_id" = "prj-n-shared-base-b12y"
Step #4 - "tf apply":     "base_shared_vpc_project_number" = "60..5"
Step #4 - "tf apply":     "restricted_shared_vpc_project_id" = "prj-n-shared-restricted-qnv6"
Step #4 - "tf apply":     "restricted_shared_vpc_project_number" = "85...26"
Step #4 - "tf apply":   }
Step #4 - "tf apply":   "production" = {
Step #4 - "tf apply":     "base_shared_vpc_project_id" = "prj-p-shared-base-oae0"
Step #4 - "tf apply":     "base_shared_vpc_project_number" = "17..1"
Step #4 - "tf apply":     "restricted_shared_vpc_project_id" = "prj-p-shared-restricted-2pqc"
Step #4 - "tf apply":     "restricted_shared_vpc_project_number" = "98..8"
Step #4 - "tf apply":   }
Step #4 - "tf apply": }
Step #4 - "tf apply": tags = {
Step #4 - "tf apply":   "environment_bootstrap" = "tagValues/281484537587812"
Step #4 - "tf apply":   "environment_development" = "tagValues/281483791828482"
Step #4 - "tf apply":   "environment_non-production" = "tagValues/281484388371311"
Step #4 - "tf apply":   "environment_production" = "tagValues/281483304603502"
Step #4 - "tf apply": }
Step #4 - "tf apply": policy-library/policies  doesn't match production; skipping
Step #4 - "tf apply": policy-library/lib  doesn't match production; skipping
Step #4 - "tf apply": policy-library/.git  doesn't match production; skipping
Finished Step #4 - "tf apply"

zexoor commented 6 months ago

Hi, Thanks for the reply. I have tested the suggested fix above, updating the assigned roles for sa-teraform-bootstrap@seedproject to:

Access Context Manager Admin
Browser
Folder Admin
Organization Administrator
Project Creator
Security Center Admin
Service Usage Consumer

I then tested running an scc command whilst impersonating the user:

gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID} --impersonate-service-account=$(terraform -chdir="../terraform-example-foundation/0-bootstrap/" output -raw organization_step_terraform_service_account_email)

But I am still seeing the API no longer available error in response.

WARNING: This command is using service account impersonation. All API calls will be executed as [sa-terraform-org@seedproject.iam.gserviceaccount.com].
ERROR: (gcloud.scc.notifications.describe) INVALID_ARGUMENT: This API is no longer available. Please use API V2.

To be certain I also attempted to deploy the resources in cloud build but similarly had no luck.

daniel-cit commented 6 months ago

Hi @zexoor, the scc_notification_config entry in the registry has a note regarding enrolling in SCC.

Note: In order to use Cloud SCC resources, your organization must be enrolled in SCC Standard/Premium. Without doing so, you may run into errors during resource creation

maybe you need to follow that enrollment procedure for your organization.

nlamot commented 6 months ago

Hi @daniel-cit @zexoor, I am also running through the fundamentals setup (using terraform cloud) and I am experiencing the same issues. I am enrolled in SCC Premium, but still get the error mentioned above:

I also tried what @fmichaelobrien suggested, but did not solve the issue for me.

daniel-cit commented 6 months ago

@zexoor and @nlamot

I was able to run the code for the creation of the notification config in my organization, but it already had Security Command Center enable for a long time.

Could you do a test to check if the problem is specific to Terraform or is it a general one?

Could you please try to use gcloud to create the SCC notification on the same Organization using the resources created in the deploy of the foundation?

export PUB_SUB_PROJECT=<SCC-PROJECT>
export ORGANIZATION_ID=<ORGANIZATION-ID>
export ORG_STEP_SA="sa-terraform-org@<SEED-PROJECT-ID>.iam.gserviceaccount.com"
export PUBSUB_TOPIC="top-scc-notification-test"

gcloud pubsub topics create "${PUBSUB_TOPIC}" \
--project="${PUB_SUB_PROJECT}"

gcloud pubsub subscriptions  create "sub-scc-notification-test" \
--topic="${PUBSUB_TOPIC}" \
--topic-project="${PUB_SUB_PROJECT}"

gcloud scc notifications create \
"scc-notify-test" \
--pubsub-topic="projects/${PUB_SUB_PROJECT}/topics/${PUBSUB_TOPIC}" \
--description="SCC Notification for all active findings" \
--filter="state = \"ACTIVE\"" \
--organization="${ORGANIZATION_ID}" \
--impersonate-service-account="${ORG_STEP_SA}"

if it fails, cloud you add --log-http --verbosity="debug" in the last command to check for detailed information on the error?

nlamot commented 6 months ago

Hi @daniel-cit

Thank you for your help! I tried the manual steps, but run into the same error: ERROR: (gcloud.scc.notifications.create) INVALID_ARGUMENT: This API is no longer available. Please use API V2. I don't think this is terraform related.

In the logs, I see the following:

DEBUG: Starting new HTTPS connection (1): securitycenter.googleapis.com:443
DEBUG: https://securitycenter.googleapis.com:443 "POST /v1/organizations/(MY ORG ID)/notificationConfigs?alt=json&configId=scc-notify-test HTTP/1.1" 400 None
---- response start ----
status: 400
-- headers start --
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Encoding: gzip
Content-Type: application/json; charset=UTF-8
Date: Tue, 16 Apr 2024 06:23:36 GMT
Server: ESF
Transfer-Encoding: chunked
Vary: Origin, X-Origin, Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 0
-- headers end --
-- body start --
{
  "error": {
    "code": 400,
    "message": "This API is no longer available. Please use API V2.",
    "status": "INVALID_ARGUMENT"
  }
}

-- body end --
total round trip time (request+response): 0.763 secs
---- response end ----
----------------------
DEBUG: (gcloud.scc.notifications.create) INVALID_ARGUMENT: This API is no longer available. Please use API V2.
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/google-cloud-sdk/471.0.0/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 998, in Execute
    resources = calliope_command.Run(cli=self, args=args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/471.0.0/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 815, in Run
    resources = command_instance.Run(args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/471.0.0/google-cloud-sdk/lib/surface/scc/notifications/create.py", line 159, in Run
    result = client.organizations_notificationConfigs.Create(req)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/471.0.0/google-cloud-sdk/lib/googlecloudsdk/generated_clients/apis/securitycenter/v1/securitycenter_v1_client.py", line 2435, in Create
    return self._RunMethod(
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/471.0.0/google-cloud-sdk/lib/third_party/apitools/base/py/base_api.py", line 737, in _RunMethod
    return self.ProcessHttpResponse(method_config, http_response, request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/471.0.0/google-cloud-sdk/lib/third_party/apitools/base/py/base_api.py", line 743, in ProcessHttpResponse
    self.__ProcessHttpResponse(method_config, http_response, request))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/471.0.0/google-cloud-sdk/lib/third_party/apitools/base/py/base_api.py", line 609, in __ProcessHttpResponse
    raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://securitycenter.googleapis.com/v1/organizations/(MY ORG ID)/notificationConfigs?alt=json&configId=scc-notify-test>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'content-encoding': 'gzip', 'date': 'Tue, 16 Apr 2024 06:23:36 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000', 'transfer-encoding': 'chunked', 'status': 400}>, content <{
  "error": {
    "code": 400,
    "message": "This API is no longer available. Please use API V2.",
    "status": "INVALID_ARGUMENT"
  }
}
>
ERROR: (gcloud.scc.notifications.create) INVALID_ARGUMENT: This API is no longer available. Please use API V2.

The output of gcloud --version:

gcloud --version
Google Cloud SDK 471.0.0
beta 2024.03.29
bq 2.1.3
core 2024.03.29
gcloud-crc32c 1.0.0
gsutil 5.27
terraform-tools 0.11.1

Based on gcloud components update, it seems like everything should be up to date.

amandakarina commented 6 months ago

hey folks, did you enable SCC api in which level? Organization, folder or project?

nlamot commented 6 months ago

Hi @amandakarina, I see the SCC api enabled on the project level (I can confirm it's at least enabled in prj-c-scc & prj-b-seed). I didn't do this manually, but I did just go the the SCC via the UI and I went through the wizard before applying my gcp-org with terraform. I enabled SCC Premium (not the API specifically) on organization level.

However, I'm not able to enable API's at the organization or folder level. Should I?

eeaton commented 6 months ago

I'm encountering the same issue in a demo org that has not enabled SCC before.

When deploying with the helpers/foundation_deployer script, step gcp-org will consistently fail at this point. Running once and re-applying, or manually enabling the SCC subscription (security center.googleapis.com enabled at project level, SCC standard activated at the organization level, the SCC service account given the service agent role at org) does not fix the issue.

Step #4 - "tf apply": Error: Error creating NotificationConfig: googleapi: Error 400: This API is no longer available. Please use API V2.
Step #4 - "tf apply": 
Step #4 - "tf apply":   with google_scc_notification_config.scc_notification_config,
Step #4 - "tf apply":   on scc_notification.tf line 32, in resource "google_scc_notification_config" "scc_notification_config":
Step #4 - "tf apply":   32: resource "google_scc_notification_config" "scc_notification_config" {
Step #4 - "tf apply": 
Step #4 - "tf apply": 
Step #4 - "tf apply": Error: Error creating Source: googleapi: Error 400: This API is no longer available. Please use API V2.
Step #4 - "tf apply": 
Step #4 - "tf apply":   with module.cai_monitoring.google_scc_source.cai_monitoring,
Step #4 - "tf apply":   on ../../modules/cai-monitoring/main.tf line 136, in resource "google_scc_source" "cai_monitoring":
Step #4 - "tf apply":  136: resource "google_scc_source" "cai_monitoring" {

eeaton commented 6 months ago

I found I can also reliably trigger this issue from the -validate step of the foundation_deploy script, before any resources are created.

$HOME/go/bin/foundation-deployer -tfvars_file <PATH_TO_FILE> -validate

# Validating tfvar file.
2024/04/17 13:21:51 error while running command: exit status 1; ERROR: (gcloud.scc.notifications.list) INVALID_ARGUMENT: This API is no longer available. Please use API V2.
- '@type': type.googleapis.com/google.rpc.DebugInfo
  detail: '[ORIGINAL ERROR] generic::invalid_argument: com.google.apps.framework.request.BadRequestException:
    This API is no longer available. Please use API V2. [google.rpc.error_details_ext]
    { message: "This API is no longer available. Please use API V2." }'
panic: testing.T failed, see logs for output (if any)

goroutine 1 [running]:
github.com/mitchellh/go-testing-interface.(*RuntimeT).FailNow(...)
    /usr/local/google/home/ellioteaton/go/pkg/mod/github.com/mitchellh/go-testing-interface@v1.14.2-0.20210821155943-2d9075ca8770/testing.go:112
github.com/mitchellh/go-testing-interface.(*RuntimeT).Fatal(0x1517ac0?, {0xc0005d4060?, 0xc000482090?, 0x81?})
    /usr/local/google/home/ellioteaton/go/pkg/mod/github.com/mitchellh/go-testing-interface@v1.14.2-0.20210821155943-2d9075ca8770/testing.go:121 +0x67
github.com/GoogleCloudPlatform/cloud-foundation-toolkit/infra/blueprint-test/pkg/gcloud.RunCmd({0x1517ac0, 0xc0001543f0}, {0xc000482090?, 0x7f999b300278?}, {0x0?, 0xc00057e098?, 0x681adf?})
    /usr/local/google/home/ellioteaton/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/gcloud/gcloud.go:84 +0xa2
github.com/GoogleCloudPlatform/cloud-foundation-toolkit/infra/blueprint-test/pkg/gcloud.Run({0x1517ac0, 0xc0001543f0}, {0xc000482090?, 0xc00057e198?}, {0x0?, 0x7f999bb1b908?, 0x7f99e35a89e8?})
    /usr/local/google/home/ellioteaton/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/gcloud/gcloud.go:108 +0x65
github.com/GoogleCloudPlatform/cloud-foundation-toolkit/infra/blueprint-test/pkg/gcloud.Runf({0x1517ac0, 0xc0001543f0}, {0x1193db8?, 0x2?}, {0xc00061a060, 0x2, 0x19?})
    /usr/local/google/home/ellioteaton/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/gcloud/gcloud.go:142 +0x169
github.com/terraform-google-modules/terraform-example-foundation/helpers/foundation-deployer/gcp.GCP.HasSccNotification({0x11c0340?, 0xc000090068?}, {0x1517ac0, 0xc0001543f0}, {0xc000015c60, 0xc}, {0xc000015c80, 0xa})
    /PATH/terraform-example-foundation/helpers/foundation-deployer/gcp/gcp.go:133 +0x1a6
github.com/terraform-google-modules/terraform-example-foundation/helpers/foundation-deployer/stages.ValidateBasicFields({_, _}, {{0xc000015c60, 0xc}, {0xc00011ac48, 0x14}, {0xc000015910, 0xb}, 0xc000744e10, {0xc000119160, ...}, ...})
    /PATH/terraform-example-foundation/helpers/foundation-deployer/stages/validate.go:51 +0x1c5
main.main()
    /PATH/terraform-example-foundation/helpers/foundation-deployer/main.go:129 +0x1bb2

eeaton commented 6 months ago

I think I've identified the root cause.

The helper script has a line that calls gcloud scc notifications list.

It seems that API v2 must specify a location, gcloud has some logic to differentiate versions based on the format of the notification name and/or flags, and v1 of the API has recently been deprecated. You can demonstrate this with some simple commands on gcloud:

gcloud scc notifications list $ORG_ID --log-http 
# uri beginning with "https://securitycenter.googleapis.com/v1/" 
# and the error "ERROR: (gcloud.scc.notifications.list) INVALID_ARGUMENT: This API is no longer available. Please use API V2."

gcloud scc notifications list $ORG_ID --location=global -log-http 
# uri beginning with "https://securitycenter.googleapis.com/v2/"
# returns the expected result, no error

And, I can reliably recreate this when trying to use the Terraform SCC resources. Running TF apply with the following will reliably produce the same error about api v2. (Make sure you've authenticated with a service account, not user identity, otherwise there's a different WAI error about user credentials).

provider "google" {
  project = <PROJECT_ID>
  region = "us-central1"
}

resource "google_scc_source" "cai_monitoring" {
  display_name = "cai monitoring"
  organization = <ORG_ID>
  description  = "SCC Finding Source for cai Monitoring Cloud Functions."
}

resource "google_pubsub_topic" "scc_notification" {
  name = "my-topic"
}

resource "google_scc_notification_config" "custom_notification_config" {
  config_id    = "my-config"
  organization = "879853064255"
  description  = "My custom Cloud Security Command Center Finding Notification Configuration"
  pubsub_topic =  google_pubsub_topic.scc_notification.id

  streaming_config {
    filter = "category = \"OPEN_FIREWALL\" AND state = \"ACTIVE\""
  }
}

To fix this across the repo, there are a few changes required:

we need an update on all instances in the script and code that run something with gcloud scc notifications to specify a location flag
Update the Terraform resources google_scc_notification_config and google_scc_source to use v2 as well. However, I can't readily find a way to do that, from the terraform definition I don't think we can set a location flag. This might be a provider issue?

eeaton commented 5 months ago

Updates:

the v2 API (currently in preview) is incompatible with terraform resources, and requires a product-side fix from the Security Command Center team.
once you've enabled v2 API by choosing a regionalization setting in SCC first-time setup, there is no way to go back to v1
this issue impacts organizations who are enabling SCC for the first time, but not orgs who were using SCC before to the release of the v2 API.

Unfortunately there isn't a good workaround until the product team fixes the v2 API and provider. (targeting end of q2). In the meantime, the best I can recommend is to remove the offending resources from your terraform config:

#gcp-org/modules/cai-monitoring/main.tf L136
resource "google_scc_source" "cai_monitoring" {

#gcp-org/envs/shared/scc_notification.tf L32
resource "google_scc_notification_config" "scc_notification_config" {

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

kkrastev-cloudoffice commented 3 months ago

Hi @eeaton

It seems you also need to remove this bit -> on modules/cai-monitoring/main.tf line 166, in module "cloud_function": │ 166: SOURCE_ID = google_scc_source.cai_monitoring.id │ │ A managed resource "google_scc_source" "cai_monitoring" has not been declared in module.cai_monitoring.

I am not sure what is left to be honest. Maybe we can just scrap the whole module, until it is fixed?

Cheers, Krasi

lpezet commented 3 months ago

@eeaton @kkrastev-cloudoffice I'm working on a PR (#1304 ) to disable this through a variable for now (easy to enable then when v2 ready and tested). I have my changes up and during the tf-pull-request (using Github) I get the following error:

*************** TERRAFORM VALIDATE ******************
      At environment: envs/shared 
      Using policy from: /home/runner/work/abseed-org/abseed-org/policy-library 
*****************************************************
ERROR: [module.base_restricted_environment_network["nonproduction"].module.base_shared_vpc_host_project.module.project-factory.google_service_account.default_service_account[0]: converting TF resource to CAI: getting resource ancestry or parent failed: user does not have the correct permissions for projects/prj-n-shared-base-b9cv. For more info: https://cloud.google.com/docs/terraform/policy-validation/troubleshooting#ProjectCallerForbidden]. Additional details: [terraform-validator-internal.git.corp.google.com/terraform-tools.git/cmd.Execute
    /tmpfs/src/git/terraform-tools/cmd/root.go:93
main.main
    /tmpfs/src/git/terraform-tools/main.go:16
runtime.main
    /usr/local/go/src/runtime/proc.go:250]
Error: Process completed with exit code 33.

Did I miss something or mess something up?

eeaton commented 3 months ago

The product team initially said they were prioritizing the fix by June but their timeline has slipped, so I'm ok with the short-term fix to disable these resources with a feature flag.

@lpezet Exit code 33 in Github Actions is a time-out, there might be flaky issues that are addressed if you run the code again, but I see some other issues in the Lint tests. I'll respond to the details on #1304.

eeaton commented 3 months ago

Thanks @lpezet for the contribution! #1304 is now merged, so none of the SCC resources are created in Terraform unless the user specifies a boolean flag to enable them. This will mitigate the case where new customers have enabled SCC v2 features that break Terraform.

I'll leave this issue open as a reminder to re-assess the defaults once the SCC team has fixed their API issues.

eeaton commented 2 months ago

I've gotten confirmation from the product team that they've released Terraform support for the v2 API. This requires changing the name of Terraform resources like google_scc_source to google_scc_v2_organization_source.

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/scc_v2_organization_source https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/scc_v2_organization_notification_config

@daniel-cit can you please update these tf resources to use the new v2 definitions. Also, for the short-term fix in #1304, I'm considering whether to revert that or keep it. I think the easiest way forward would be to leave the variable to toggle enable_scc_resources_in_terraform, but set the default to true. But let's discuss if you think there's a better way to resolve it. Thanks!

github-actions[bot] commented 3 days ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

terraform-google-modules / terraform-example-foundation