google_service_networking_connection has no options for importing/exporting custom routes

[red8888]

Affected Resource(s)

• google_service_networking_connection

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Following docs for configuring cloudsql with private IP: https://cloud.google.com/sql/docs/postgres/configure-private-ip#vpn

I need to update the private IP peering connection with custom routes: https://cloud.google.com/vpc/docs/using-vpc-peering#update-peer-connection

gcloud compute networks peerings update PEERING_NAME \
    --network=NETWORK \
    [--import-custom-routes] \
    [--export-custom-routes] \
    [--export-subnet-routes-with-public-ip] \
    [--import-subnet-routes-with-public-ip]

I see no options for doing anything with custom routes in the resource documentation: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_networking_connection

This is the specialized google_service_networking_connection resource used for setting up private IP NOT the google_compute_network_peering resource.

[red8888]

Actually need to add more to this...

servicenetworking-googleapis-com peering connection is created by the google_service_networking_connection resource, BUT the cloudsql peering connections are created dynamically when a VPCed cloudsql instance is created: cloudsql-mysql-googleapis-com cloudsql-postgres-googleapis-com

It's kinda crappy how it works like that. The google_sql_database_instance resource should probably have an option to set export/import routes on these dynamically created peers. Even while these peerings are shared with all instances in the same network I cant see another way to manage this.

[fstr]

@red8888 You can use the resource google_compute_network_peering_routes_config which allows you to modify the import and export settings on the peering connection. The resource is a no-op resource, so if you delete this resource, the peering connection (which was auto generated) will not be affected.

You can use the same approach with your GKE control-plane peering connection if you plan to use GKE.

resource "google_compute_network_peering_routes_config" "private_service_access_generic" {
  depends_on = [google_service_networking_connection.private_service_access]

  project = google_project.networking_project.project_id
  peering = "servicenetworking-googleapis-com" # Hardcoded name
  network = module.vpc.network.network_name

  import_custom_routes = false
  export_custom_routes = true # We change this from the default to export our custom routes
}

resource "google_compute_network_peering_routes_config" "private_service_access_mysql" {
  depends_on = [google_service_networking_connection.private_service_access]

  project = google_project.networking_project.project_id
  peering = "cloudsql-mysql-googleapis-com" # Hardcoded name
  network = module.vpc.network.network_name

  import_custom_routes = false
  export_custom_routes = true # We change this from the default to export our custom routes
}

I still have an issue with this approach. 1) I need to hardcode the names of the peering connections cloudsql-mysql-googleapis-com and servicenetworking-googleapis-com 2) I don't know why the cloudsql-mysql-googleapis-com peering connection is created. This peering connection is already created before I create any Cloud SQL instance. I actually create a MS SQL Server instance, so this (now hardcoded) name is even more confusing. 3) My reserved IP range is only imported in the cloudsql-mysql-googleapis-com. After going through the Google Docs, I am not sure what the exact purpose of the servicenetworking-googleapis-com peering connection is.

Most of these are more related to GCP than to the TF provider.

What the Terraform provider could improve (if the API provides it, I didn't check):

• Return the names of the generated peering connections as an attribute, so I don't need to hardcode their names. EDIT: This is actually described in #5248

[fstr]

After creating a Postgres instance in the user-interface, I also have the cloudsql-postgres-googleapis-com peering connection.

This confirms what @red8888 said before, that cloudsql-postgres-googleapis-com is created dynamically after the Postgres instance was provisioned.

My Microsoft SQL Server is re-using the cloudsql-mysql-googleapis-com though and did not receive it's own peering connection.

If you delete your peering connections cloudsql-postgres-googleapis-com, cloudsql-mysql-googleapis-com and servicenetworking-googleapis-com and disable your private services access and enable it again, all of these peering connections will appear again, even without provisioning any Cloud SQL instance.

The only way to solve this with Terraform right now, is to let the google_compute_network_peering_routes_config depend on your SQL instance and then to modify the settings. Since you never know if it already exists and if it was already modified, you better add the google_compute_network_peering_routes_config to every SQL instance creation.

The Google API doesn't seem to support a flag to set custom route export when you create a private SQL instance.

# Mysql
resource "google_sql_database_instance" "my_mysql_instance" {
  ...
}

resource "google_compute_network_peering_routes_config" "private_service_access_mysql" {
  depends_on = [google_sql_database_instance.my_mysql_instance]

  project = google_project.networking_project.project_id
  peering = "cloudsql-mysql-googleapis-com" # Hardcoded name
  network = module.vpc.network.network_name

  import_custom_routes = false
  export_custom_routes = true # We change this from the default to export our custom routes
}

# Postgres
resource "google_sql_database_instance" "my_postgres_instance" {
  ...
}

resource "google_compute_network_peering_routes_config" "private_service_access_postgres" {
  depends_on = [google_sql_database_instance.my_postgres_instance]

  project = google_project.networking_project.project_id
  peering = "cloudsql-postgres-googleapis-com" # Hardcoded name
  network = module.vpc.network.network_name

  import_custom_routes = false
  export_custom_routes = true # We change this from the default to export our custom routes
}

# SQL Server
resource "google_sql_database_instance" "my_sqlserver_instance" {
  ...
}

resource "google_compute_network_peering_routes_config" "private_service_access_sqlserver" {
  depends_on = [google_sql_database_instance.my_sqlserver_instance]

  project = google_project.networking_project.project_id
  peering = "cloudsql-mysql-googleapis-com" # Hardcoded name. For some reason uses the mysql peering connection.
  network = module.vpc.network.network_name

  import_custom_routes = false
  export_custom_routes = true # We change this from the default to export our custom routes
}

[ggtisc]

Currently there is a way to achieve this. You could follow this example