hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.62k stars 9.01k forks source link

SESv2 Plan #26796

Open kamilturek opened 1 year ago

kamilturek commented 1 year ago

Community Note

SESv2

This issue contains a plan for SESv2 service implementation. It includes a list of planned resources along with proposed sample schemas. Any feedback is very appreciated.

Not all SESv2 API actions are covered. Attention was primarily paid to main SESv2 resources and the most up-voted SES(v2)-related issues. Although, the remaining SESv2 resources might be also implemented in the future.

This issue also aims to track the progress of SESv2 service implementation.

Table of Contents

Resource List

This section lists all service resources with proposed schemas and relevant AWS API actions. Corresponding data sources (singular and plural) are skipped in the list but also should be implemented.

Click here to expand ### aws_sesv2_configuration_set #### Schema ```terraform resource "aws_sesv2_configuration_set" "example" { configuration_set_name = "example" delivery_options { sending_pool_name = aws_sesv2_dedicated_ip_pool.example.pool_name tls_policy = "REQUIRE" } reputation_options { last_fresh_start = "timestamp" reputation_metrics_enabled = true } sending_options { sending_enabled = true } suppression_options { suppressed_reasons = ["BOUNCE", "COMPLIANT"] } tracking_options { custom_redirect_domain = "example.com" } } ``` #### API Actions - [CreateConfigurationSet](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateConfigurationSet.html) - [DeleteConfigurationSet](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_DeleteConfigurationSet.html) - [GetConfigurationSet](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetConfigurationSet.html) - [ListConfigurationSets](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_ListConfigurationSets.html) - [PutConfigurationSetDeliveryOptions](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutConfigurationSetDeliveryOptions.html) - [PutConfigurationSetReputationOptions](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutConfigurationSetReputationOptions.html) - [PutConfigurationSetSendingOptions](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutConfigurationSetSendingOptions.html) - [PutConfigurationSetSuppressionOptions](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutConfigurationSetSuppressionOptions.html) - [PutConfigurationSetTrackingOptions](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutConfigurationSetTrackingOptions.html) ### aws_sesv2_configuration_set_event_destination #### Schema ```terraform resource "aws_sesv2_configuration_set_event_destination" "example" { configuration_set_name = aws_sesv2_configuration_set.example.name event_destination_name = "example" event_destination { enabled = true matching_event_types = ["BOUNCE", "COMPLIANT"] cloud_watch_service_destination { dimension_configuration { default_dimension_value = "example" dimension_name = "example" dimension_value_source = "MESSAGE_TAG" } } kinesis_firehose_destination { delivery_stream_arn = "arn" iam_role_arn = "arn" } pinpoint_destination { application_arn = "arn" } sns_destination { topic_arn = "arn" } } } ``` #### API Actions - [CreateConfigurationSetEventDestination](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateConfigurationSetEventDestination.html) - [DeleteConfigurationSetEventDestination](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_DeleteConfigurationSetEventDestination.html) - [GetConfigurationSetEventDestinations](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetConfigurationSetEventDestinations.html) - [UpdateConfigurationSetEventDestination](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_UpdateConfigurationSetEventDestination.html) ### aws_sesv2_dedicated_ip_pool #### Schema ```terraform resource "aws_sesv2_dedicated_ip_pool" "example" { pool_name = "example" } ``` #### API Actions - [CreateDedicatedIpPool](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateDedicatedIpPool.html) - [DeleteDedicatedIpPool](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_DeleteDedicatedIpPool.html) - [ListDedicatedIpPools](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_ListDedicatedIpPools.html) ### aws_sesv2_dedicated_ip_assignment #### Schema ```terraform resource "aws_sesv2_dedicated_ip_assignment" "example" { ip = "ip" destination_pool_name = aws_sesv2_dedicated_ip_pool.example.pool_name } ``` #### API Actions - [PutDedicatedIpInPool](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutDedicatedIpInPool.html) - [GetDedicatedIp](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetDedicatedIp.html) - [GetDedicatedIps](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetDedicatedIps.html) ### aws_sesv2_contact_list #### Schema ```terraform resource "aws_sesv2_contact_list" "example" { contact_list_name = "example" description = "description" topic { default_subscription_status = "OPT_OUT" description = "description" display_name = "example" topic_name = "exmaple" } } ``` #### API Actions - [CreateContactList](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateContactList.html) - [DeleteContactList](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_DeleteContactList.html) - [GetContactList](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetContactList.html) - [ListContactLists](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_ListContactLists.html) - [UpdateContactList](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_UpdateContactList.html) ### aws_sesv2_contact #### Schema ```terraform resource "aws_sesv2_contact" "example" { contact_list_name = aws_sesv2_contact_list.example.contact_list_name attributes_data = "example" email_address = "example@example.com" unsubscribe_all = false topic_preference { subscription_status = "OPT_IN" topic_name = "exmaple" } } ``` #### API Actions - [CreateContact](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateContact.html) - [DeleteContact](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_DeleteContact.html) - [GetContact](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetContact.html) - [ListContacts](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_ListContacts.html) - [UpdateContact](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_UpdateContact.html) ### aws_sesv2_custom_verification_email_template To use this feature, your Amazon SES account has to be out of the sandbox. #### Schema ```terraform resource "aws_sesv2_custom_verification_email_template" "example" { from_email_address = "example@example.com" failure_redirection_url = "url" success_redirection_url = "url" template_content = "content" template_name = "name" template_subject = "subject" } ``` #### API Actions - [CreateCustomVerificationEmailTemplate](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateCustomVerificationEmailTemplate.html) - [DeleteCustomVerificationEmailTemplate](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_DeleteCustomVerificationEmailTemplate.html) - [GetCustomVerificationEmailTemplate](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetCustomVerificationEmailTemplate.html) - [ListCustomVerificationEmailTemplates](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_ListCustomVerificationEmailTemplates.html) - [UpdateCustomVerificationEmailTemplate](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_UpdateCustomVerificationEmailTemplate.html) ### aws_sesv2_deliverability_test_report The test is available after 24 hours. It is unclear how the provider plugin should behave in this case. #### Schema TBC #### API Actions - [CreateDeliverabilityTestReport](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateDeliverabilityTestReport.html) - [GetDeliverabilityTestReport](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetDeliverabilityTestReport.html) - [ListDeliverabilityTestReports](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_ListDeliverabilityTestReports.html) ### aws_sesv2_email_identity #### Schema ```terraform resource "aws_sesv2_email_identity" "example" { configuration_set_name = aws_sesv2_configuration_set.example.configuration_set_name dkim_signing_attributes { domain_signing_private_key = "a" domain_signing_selector = "b" # next_signing_key_length = "RSA_1024_BIT" } email_identity = "example@example.com" } ``` #### API Actions - [CreateEmailIdentity](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_CreateEmailIdentity.html) - [DeleteEmailIdentity](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_DeleteEmailIdentity.html) - [GetEmailIdentity](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_GetEmailIdentity.html) - [ListEmailIdentities](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_ListEmailIdentities.html) - [PutEmailIdentityConfigurationSetAttributes](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutEmailIdentityConfigurationSetAttributes.html) - [PutEmailIdentityDkimAttributes](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutEmailIdentityDkimAttributes.html) - [PutEmailIdentityDkimSigningAttributes](https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutEmailIdentityDkimSigningAttributes.html) ### aws_sesv2_email_identity_policy #### Schema ```terraform resource "aws_sesv2_email_identity_policy" "example" { email_identity = aws_sesv2_email_identity.example.email_identity policy_name = "name" policy = <

Related

https://github.com/hashicorp/terraform-provider-aws/issues/14320 https://github.com/hashicorp/terraform-provider-aws/issues/10703 https://github.com/hashicorp/terraform-provider-aws/issues/17570 https://github.com/hashicorp/terraform-provider-aws/issues/21129

JoshiiSinfield commented 1 year ago

Can I ask why you're planning new schemas / resource names?

There's not a SESv2 service, or any other difference within the console or resources within AWS. It's purely an SDK / API thing.

No one using Terraform will choose between a v1 or v2 resouces. It's just "ses resources".

Also, shameless plug but I've done the ground work & email identity work in here:

https://github.com/hashicorp/terraform-provider-aws/pull/26604

kamilturek commented 1 year ago

Hey @JoshiiSinfield

Thank you for your comment. Since the SESv1 and SESv2 are substantially different, the goal is to follow the approach of other services like waf & waf2, apigateway & apigatewayv2, kinesisanalytics &. kinesisanalytics2 and so on.

Also, nice to see your work on the identities 🙌 I think it can become part of sesv2.

JoshiiSinfield commented 1 year ago

Thanks @kamilturek .

It's not actually that much different?

WAFv2 was a whole different set of actual resources. As was apigwv2- a http apigw.

Slightly different for ses.

As can see from my PR, it's very similar api functions.

Not looking forward to having to swap out all ses resources tho tbh. Gonna be a lot of moved resource blocks.

Doesn't need to be separate IMO. Apologies, realise I'm not a maintainer or anything so my argument probably doesn't stand. But still wanted to air it!

Cheers - would appreciate my work not going redundant. Also shows new resources aren't needed 😂

Cheers

jar-b commented 1 year ago

How would you feel about converting aws_sesv2_dedicated_ip to a data source and renaming the proposed resource to indicate assignment to a pool (aws_sesv2_dedicated_ip_assignment)? The justification being the IP itself must be created via an AWS support case (both when AWS provides it, or bringing your own), and the _assignment suffix describes the action of assigning the IP to a pool versus "creating" it.

# assigns an existing IP to an existing dedicated IP pool
resource "aws_sesv2_dedicated_ip_assignment" "example" {
  ip                    = "0.0.0.0"
  destination_pool_name = aws_sesv2_dedicated_ip_pool.example.pool_name
}

# a single dedicated IP, returns information on warmup percentage, status, etc.
data "aws_sesv2_dedicated_ip" "example" {
  ip = "0.0.0.0"
}
kamilturek commented 1 year ago

@jar-b That makes sense. Sounds good to me! 👍

JoshiiSinfield commented 1 year ago

@kamilturek @jar-b FYI https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-ses-model-provisioning-managing-dedicated-ips/ Probably changes things.

JoshiiSinfield commented 1 year ago

@kamilturek judging by your 27260 shall I close mine then ?

jar-b commented 1 year ago

Thanks for the link @JoshiiSinfield!

Sounds like this will require a new scaling_mode attribute on the aws_sesv2_dedicated_ip_pool resource to allow creation of managed pools. I believe the proposed aws_sesv2_dedicated_ip_assignment resource can remain because IP assignment will still be necessary for standard (non-managed) pools.

Edit: ScalingMode is already available in v1.14.0 of the sesv2 Go SDK, which will be bumped via #27321 after todays release.

F21 commented 1 year ago

Would definitely love to see aws_sesv2_email_identity_mail_from_attributes implemented. It's a separate endpoint in Amazon's SESv2 API, but would it make sense to put it as an attribute under aws_sesv2_email_identity?

kamilturek commented 1 year ago

Hey @F21, thanks for your view on this! 🙌

As you mentioned, CreateEmailIdentity API action does not support specifying Mail From attributes but it can be done through a separate endpoint. I'd say we should avoid mixing the PutEmailIdentityMailFromAttributes into aws_sesv2_email_identity CRUD cycle as it increases its complexity and drifts it away from the AWS API design. If we keep adding more and more attributes to aws_sesv2_email_identity that are managed by different API actions, it can become really bloated, error-prone, and hard to maintain.

There is a general tendency of splitting out PUT APIs into their own resources in cases like that. aws_s3_bucket_* resources are a great example of this.

F21 commented 1 year ago

I have been testing out the aws_sesv2_configuration_set resource, and I think suppressed_reasons in suppression_options should have its MinItems set to 0 rather than 1: https://github.com/kamilturek/terraform-provider-aws/blob/753ff2aacae8a69f897b64564211b728ad2c3d8c/internal/service/sesv2/configuration_set.go#L109

By setting it to 0, we can use the following:

resource "aws_sesv2_configuration_set" "my_config_set" {
  configuration_set_name = "my-config-set"
  suppression_options {
    suppressed_reasons = []
  }
}

This will allow for the case where we want to override the account-level suppression option and disable the suppression list for this configuration set.

EderNorivalSouza commented 1 year ago

Hi, trying use resource -> "aws_sesv2_contact_list" but received the error. Error: Invalid resource type │ │ on ../../modules/contact-list/main.tf line 6, in resource "aws_sesv2_contact_list" "contact-list": │ 6: resource "aws_sesv2_contact_list" "contact-list" { │ │ The provider hashicorp/aws does not support resource type "aws_sesv2_contact_list".

version Terraform v1.3.6 on linux_amd64

Maybe this is a ISSUE? Or already being implemented? Thanks

jar-b commented 1 year ago

Hi @EderNorivalSouza - aws_sesv2_contact_list has not been implemented yet. This issue is tracking the plan for proposed sesv2 resources, but only those with a strikethrough in the Table of Contents have been completed to this point.

gmx-git commented 1 year ago

Hi, I'm trying to configure a sesv2 based verified domain identity.

How can we configure notifications (Bounce, Complaint, Delivery) using SESv2 resources? I haven't been able to figure it out using aws_sesv2_email_identity.

Or, is there a different sesv2 resource that should be used instead?

Thank you!

kamilturek commented 1 year ago

Hey @gmx-git, maybe aws_sesv2_configuration_set_event_destination is what you're looking for.

Scifire commented 1 year ago

@kamilturek I guess I´m missing the same setting as @gmx-git. The configuration sets are something different than the feedback notifications.

The feedback notifications are configured in each verified identity under notifications image For this I couldn´t find any TF setting.

The configuration sets are an own topic and could be configured as you linked.

kamilturek commented 1 year ago

Hey @Scifire. Thank you, I haven't seen that.

Looking at this guide, I think it is available only for API v1. I can't find its equivalent in the API v2 specification. I believe it can be achieved by using the configuration set event destination and configuration set assigned to the email identity.

There is an SES v1 resource for managing these feedback notifications. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_identity_notification_topic

gmx-git commented 1 year ago

Hey guys,

I've done some research on my own on this topic as well. The APIv1 indeed contained the SetIdentityNotificationTopichttps://docs.aws.amazon.com/ses/latest/APIReference/API_SetIdentityNotificationTopic.html, while there's no correspondence in the APIv2.

I've noted a hint while creating a SES identity (see screenshot below). To me it looks like AWS wants to fade out the notifications at the identity level and recommends shifting to notifications on the configuration set level. We can create/set a default configuration set for an SES identity. The default config set will be used by default when sending emails using the SES identity. We don't need to specify it, but the sender should have the rights to use both the SES identity and the default configuration set assigned to it.

image

On my side I'll review my current design and shift the notifications to the default configuration sets.

Hope this helps!

Manouchehri commented 1 year ago

This is a bit off-topic, but not sure where else to write this note/warning...

Be very careful when migrating from a SESv1 domain identity to SESv2. Destroying the identity doesn't actually instantly cause it to be removed; it seems to stick around, and blocks you from creating a new identity with the exact same domain name. I managed to work around it by manually messing with the state, but it wasn't something I'd recommend repeating.

terraform import aws_sesv2_email_identity.ses_domain YOUR_DOMAIN
terraform state rm aws_ses_domain_identity.ses_domain
terraform state rm aws_ses_domain_dkim.ses_domain_dkim
terraform state rm aws_ses_domain_mail_from.main

terraform apply

I don't think Terraform can address this edge case, since it's straight up the AWS API response that's returning confusing information. (i.e. it can sometimes return a 200 when creating a new identity, and then fail right after when you try go get that identity..)

mattbnz commented 12 months ago

Is there any reason why the ability to request sandbox removal / production migration from Terraform doesn't seem to be mentioned in any of the plans above?

Currently I find this to be a very tedious and manual addition to our otherwise nicely declarative process for turning up in a new region.

Specifically I'm looking for a resource that matches the https://docs.aws.amazon.com/ses/latest/APIReference-V2/API_PutAccountDetails.html API call.

jar-b commented 12 months ago

Hey @mattbnz 👋 - Seems reasonable to me. If you could open a New Resource issue and link back to this one I can add it to the Meta-issue list.

Also, to set expectations, implementation of these resources has largely been community driven thus far as this issue is not currently on the maintainer roadmap.

kamilturek commented 6 months ago

As core SESv2 resources have been already implemented and it's becoming quite difficult to follow and manage the work through a single GitHub issue, I suggest closing this one and opening new fine-grained issues for remaining resources as needed.

I'll keep it open for a few more days in case there are any objections.

jeremyatenovis commented 6 months ago

Is aws_sesv2_domain_identity implemented? I can't seem to find it. Additionally, I'd very much like a way to associate a default configuration set to an SES identity.

kamilturek commented 6 months ago

Hey @jeremyatenovis, the aws_sesv2_email_identity resource allows you to manage both email address identities and domain identities. Also, there is a way to associate a default configuration set to it.

Please have a look at the documentation page. Let me know if you have any questions. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sesv2_email_identity

shamx9ir commented 5 months ago

We are currently using aws_ses_domain_identity and would like to use aws_sesv2_email_identity to be able to assign default configuration set. How do we approach this without accidentally dropping the existing domain identity? Do we have a guide somewhere?

raffraffraff commented 5 months ago

Since there isn't a resource aws_sesv2_email_identity_policy yet, I assume there's no way to create an SESv2 policy that can allow a Cognito pool to send email via SESv2. The AWS cli has aws sesv2 create-email-identity-policy --email-identity <id> --policy <json_string>, so I suppose I could write some horrendous null-resource stuff (like I had to do with OpenSearch endpoint authorization). I really don't want to have to do that again.

kamilturek commented 5 months ago

@raffraffraff I opened a PR with aws_sesv2_email_identity_policy resource. I should be able to finalize it soon. https://github.com/hashicorp/terraform-provider-aws/pull/35486

pspot2 commented 4 months ago

https://github.com/hashicorp/terraform-provider-aws/issues/22655

nightpool commented 3 months ago

What is the difference between SES and SESv2? Which one should I use for a new terraform setup? If I choose V2, am I going to run into issues where I cannot create certain resources since they're not supported yet? Why is this a completely separate resource type instead of Terraform just choosing the best API to use on the backend?

Scifire commented 3 months ago

@nightpool I´d recommend use SESv2. For normal setups everything should work there. I´ve started with SES but reworked it on v2 shortly later cause there are more recent features available.

OJFord commented 2 months ago

I agree with @nightpool and @JoshiiSinfield (way back at the start of this issue) - it's a real shame to have these as new resources IMO, it's confusing.

People are going to search resources on the registry for 'ses', then use v1 because it's listed first and v2 not even on the screen. If they do see v2, they'll look in the console for SESv2 and find nothing; search for information for it and find the CLI reference or a blog post, and think why am I having to think about this I'm just using terraform this is an implementation detail, and choose arbitrarily between them (v1 because it looks cleaner and is more established/battle-tested? v2 because it's newer and presumably further from being ever retired?).