Elastic Beanstalk settings are not sticking

[hashibot]

This issue was originally opened by @ccampo133 as hashicorp/terraform#8950. It was migrated here as part of the provider split. The original body of the issue is below.

---

When updating an elastic beanstalk environment resource settings, Terraform does not seem to update the state. Each successive run of terraform plan or terraform apply indicates the resource needs to be updated, despite actually being correct on AWS and unchanged in the terraform code. The output seems to indicate it has to do with the LoadBalancerHttpPort and SSLCertificateId settings.

Terraform Version

0.7.4

Affected Resource(s)

aws_elastic_beanstalk_environment

Terraform Configuration Files

resource "aws_elastic_beanstalk_environment" "mercury_api_develop" {
  name = "mercury-api"
  application = "${aws_elastic_beanstalk_application.mercury_api.name}"
  solution_stack_name = "64bit Amazon Linux 2016.03 v2.1.6 running Docker 1.11.2"
  tier = "WebServer"

  # VPC settings
  setting {
    namespace = "aws:ec2:vpc"
    name = "VPCId"
    value = "${var.vpc_id}"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "Subnets"
    value = "${join(",", var.worker_subnets)}"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "ELBSubnets"
    value = "${join(",", var.elb_subnets)}"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "AssociatePublicIpAddress"
    value = "false"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "ELBScheme"
    value = "public"
  }

  # ELB settings
  setting {
    namespace = "aws:elb:loadbalancer"
    name = "CrossZone"
    value = "true"
  }
  setting {
    namespace = "aws:elb:loadbalancer"
    name = "SecurityGroups"
    value = "${aws_security_group.mercury_api_elb_sg.id}"
  }
  setting {
    namespace = "aws:elb:loadbalancer"
    name = "ManagedSecurityGroup"
    value = "${aws_security_group.mercury_api_elb_sg.id}"
  }
  setting {
    namespace = "aws:elb:loadbalancer"
    name = "LoadBalancerHTTPSPort"
    value = "443"
  }
  setting {
    namespace = "aws:elb:loadbalancer"
    name = "LoadBalancerHTTPPort"
    value = "80"
  }
  setting {
    namespace = "aws:elb:listener"
    name = "ListenerProtocol"
    value = "HTTPS"
  }
  setting {
    namespace = "aws:elb:listener"
    name = "InstanceProtocol"
    value = "HTTP"
  }
  setting {
    namespace = "aws:elb:listener"
    name = "SSLCertificateId"
    value = "${var.elb_cert_arn}"
  }
  setting {
    namespace = "aws:elbv2:loadbalancer"
    name = "AccessLogsS3Enabled"
    value = "true"
  }
  setting {
    namespace = "aws:elbv2:loadbalancer"
    name = "AccessLogsS3Bucket"
    value = "${var.access_logs_s3_bucket_name}"
  }

  # General Beanstalk settings
  setting {
    namespace = "aws:elasticbeanstalk:command"
    name = "BatchSize"
    value = "30"
  }
  setting {
    namespace = "aws:elasticbeanstalk:command"
    name = "BatchSizeType"
    value = "Percentage"
  }
  setting {
    namespace = "aws:elasticbeanstalk:hostmanager"
    name = "LogPublicationControl"
    value = "true"
  }
  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name = "ServiceRole"
    value = "${var.service_role_name}"
  }

  # Auto-scaling (EC2 instance) settings
  setting {
    namespace = "aws:autoscaling:asg"
    name = "Availability Zones"
    value = "Any 2"
  }
  setting {
    namespace = "aws:autoscaling:asg"
    name = "MinSize"
    value = "1"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "IamInstanceProfile"
    value = "${var.instance_profile_name}"
  }
  # TODO: set ec2 instance security groups -ccampo 2016-05-25
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "SecurityGroups"
    value = "${aws_security_group.mercury_api_instance_sg.id}"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "EC2KeyName"
    value = "${var.key}"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "InstanceType"
    value = "t2.micro"
  }
  setting {
    namespace = "aws:autoscaling:updatepolicy:rollingupdate"
    name = "RollingUpdateType"
    value = "Health"
  }

  # Health reporting
  setting {
    namespace = "aws:elasticbeanstalk:healthreporting:system"
    name = "SystemType"
    value = "enhanced"
  }

  tags {
    Name = "mercury-api"
    Environment = "${var.environment}"
    Product = "Mercury"
  }
}

Debug Output

~ module.mercury.mercury_api.aws_elastic_beanstalk_environment.mercury_api_develop
    setting.#:                    "22" => "27"
    setting.1145997792.name:      "SecurityGroups" => "SecurityGroups"
    setting.1145997792.namespace: "aws:autoscaling:launchconfiguration" => "aws:autoscaling:launchconfiguration"
    setting.1145997792.resource:  "" => ""
    setting.1145997792.value:     "sg-260a9a5c" => "sg-260a9a5c"
    setting.1314173431.name:      "LoadBalancerHTTPSPort" => "LoadBalancerHTTPSPort"
    setting.1314173431.namespace: "aws:elb:loadbalancer" => "aws:elb:loadbalancer"
    setting.1314173431.resource:  "" => ""
    setting.1314173431.value:     "443" => "443"
    setting.1384328253.name:      "ELBScheme" => "ELBScheme"
    setting.1384328253.namespace: "aws:ec2:vpc" => "aws:ec2:vpc"
    setting.1384328253.resource:  "" => ""
    setting.1384328253.value:     "public" => "public"
    setting.1429395430.name:      "AssociatePublicIpAddress" => "AssociatePublicIpAddress"
    setting.1429395430.namespace: "aws:ec2:vpc" => "aws:ec2:vpc"
    setting.1429395430.resource:  "" => ""
    setting.1429395430.value:     "false" => "false"
    setting.1489838287.name:      "" => "AccessLogsS3Enabled"
    setting.1489838287.namespace: "" => "aws:elbv2:loadbalancer"
    setting.1489838287.resource:  "" => ""
    setting.1489838287.value:     "" => "true"
    setting.1497664210.name:      "CrossZone" => "CrossZone"
    setting.1497664210.namespace: "aws:elb:loadbalancer" => "aws:elb:loadbalancer"
    setting.1497664210.resource:  "" => ""
    setting.1497664210.value:     "true" => "true"
    setting.168697961.name:       "" => "InstanceProtocol"
    setting.168697961.namespace:  "" => "aws:elb:listener"
    setting.168697961.resource:   "" => ""
    setting.168697961.value:      "" => "HTTP"
    setting.1865048766.name:      "LogPublicationControl" => "LogPublicationControl"
    setting.1865048766.namespace: "aws:elasticbeanstalk:hostmanager" => "aws:elasticbeanstalk:hostmanager"
    setting.1865048766.resource:  "" => ""
    setting.1865048766.value:     "true" => "true"
    setting.2220815128.name:      "VPCId" => "VPCId"
    setting.2220815128.namespace: "aws:ec2:vpc" => "aws:ec2:vpc"
    setting.2220815128.resource:  "" => ""
    setting.2220815128.value:     "vpc-5752cb30" => "vpc-5752cb30"
    setting.2262995125.name:      "Availability Zones" => "Availability Zones"
    setting.2262995125.namespace: "aws:autoscaling:asg" => "aws:autoscaling:asg"
    setting.2262995125.resource:  "" => ""
    setting.2262995125.value:     "Any 2" => "Any 2"
    setting.2276893638.name:      "RollingUpdateType" => "RollingUpdateType"
    setting.2276893638.namespace: "aws:autoscaling:updatepolicy:rollingupdate" => "aws:autoscaling:updatepolicy:rollingupdate"
    setting.2276893638.resource:  "" => ""
    setting.2276893638.value:     "Health" => "Health"
    setting.2396587397.name:      "MinSize" => "MinSize"
    setting.2396587397.namespace: "aws:autoscaling:asg" => "aws:autoscaling:asg"
    setting.2396587397.resource:  "" => ""
    setting.2396587397.value:     "1" => "1"
    setting.2420299722.name:      "SystemType" => "SystemType"
    setting.2420299722.namespace: "aws:elasticbeanstalk:healthreporting:system" => "aws:elasticbeanstalk:healthreporting:system"
    setting.2420299722.resource:  "" => ""
    setting.2420299722.value:     "enhanced" => "enhanced"
    setting.2912896423.name:      "LoadBalancerHTTPPort" => ""
    setting.2912896423.namespace: "aws:elb:loadbalancer" => ""
    setting.2912896423.resource:  "" => ""
    setting.2912896423.value:     "OFF" => ""
    setting.2972739376.name:      "" => "ListenerProtocol"
    setting.2972739376.namespace: "" => "aws:elb:listener"
    setting.2972739376.resource:  "" => ""
    setting.2972739376.value:     "" => "HTTPS"
    setting.3023161937.name:      "Subnets" => "Subnets"
    setting.3023161937.namespace: "aws:ec2:vpc" => "aws:ec2:vpc"
    setting.3023161937.resource:  "" => ""
    setting.3023161937.value:     "subnet-38ffc312,subnet-c4c4288d" => "subnet-38ffc312,subnet-c4c4288d"
    setting.3111731957.name:      "InstanceType" => "InstanceType"
    setting.3111731957.namespace: "aws:autoscaling:launchconfiguration" => "aws:autoscaling:launchconfiguration"
    setting.3111731957.resource:  "" => ""
    setting.3111731957.value:     "t2.micro" => "t2.micro"
    setting.3180054284.name:      "IamInstanceProfile" => "IamInstanceProfile"
    setting.3180054284.namespace: "aws:autoscaling:launchconfiguration" => "aws:autoscaling:launchconfiguration"
    setting.3180054284.resource:  "" => ""
    setting.3180054284.value:     "mercury-elasticbeanstalk-ec2-role" => "mercury-elasticbeanstalk-ec2-role"
    setting.3517974641.name:      "" => "AccessLogsS3Bucket"
    setting.3517974641.namespace: "" => "aws:elbv2:loadbalancer"
    setting.3517974641.resource:  "" => ""
    setting.3517974641.value:     "" => "elasticbeanstalk-us-east-1-927185244192"
    setting.3602277136.name:      "BatchSize" => "BatchSize"
    setting.3602277136.namespace: "aws:elasticbeanstalk:command" => "aws:elasticbeanstalk:command"
    setting.3602277136.resource:  "" => ""
    setting.3602277136.value:     "30" => "30"
    setting.3665244782.name:      "ManagedSecurityGroup" => "ManagedSecurityGroup"
    setting.3665244782.namespace: "aws:elb:loadbalancer" => "aws:elb:loadbalancer"
    setting.3665244782.resource:  "" => ""
    setting.3665244782.value:     "sg-290a9a53" => "sg-290a9a53"
    setting.3895537189.name:      "ServiceRole" => "ServiceRole"
    setting.3895537189.namespace: "aws:elasticbeanstalk:environment" => "aws:elasticbeanstalk:environment"
    setting.3895537189.resource:  "" => ""
    setting.3895537189.value:     "mercury-elasticbeanstalk-service-role" => "mercury-elasticbeanstalk-service-role"
    setting.3925075571.name:      "ELBSubnets" => "ELBSubnets"
    setting.3925075571.namespace: "aws:ec2:vpc" => "aws:ec2:vpc"
    setting.3925075571.resource:  "" => ""
    setting.3925075571.value:     "subnet-32ffc318,subnet-d9c42890" => "subnet-32ffc318,subnet-d9c42890"
    setting.44009617.name:        "EC2KeyName" => "EC2KeyName"
    setting.44009617.namespace:   "aws:autoscaling:launchconfiguration" => "aws:autoscaling:launchconfiguration"
    setting.44009617.resource:    "" => ""
    setting.44009617.value:       "mercury-terraform-key" => "mercury-terraform-key"
    setting.478148791.name:       "SecurityGroups" => "SecurityGroups"
    setting.478148791.namespace:  "aws:elb:loadbalancer" => "aws:elb:loadbalancer"
    setting.478148791.resource:   "" => ""
    setting.478148791.value:      "sg-290a9a53" => "sg-290a9a53"
    setting.747815089.name:       "BatchSizeType" => "BatchSizeType"
    setting.747815089.namespace:  "aws:elasticbeanstalk:command" => "aws:elasticbeanstalk:command"
    setting.747815089.resource:   "" => ""
    setting.747815089.value:      "Percentage" => "Percentage"
    setting.784312882.name:       "" => "LoadBalancerHTTPPort"
    setting.784312882.namespace:  "" => "aws:elb:loadbalancer"
    setting.784312882.resource:   "" => ""
    setting.784312882.value:      "" => "80"
    setting.889637335.name:       "" => "SSLCertificateId"
    setting.889637335.namespace:  "" => "aws:elb:listener"
    setting.889637335.resource:   "" => ""
    setting.889637335.value:      "" => "arn:aws:iam::REDACTED:server-certificate/REDACTED"

Expected Behavior

After running terraform apply, the next run of terraform plan or terraform apply should have nothing to update, and exit quietly.

Actual Behavior

After running terraform apply, the next run of terraform plan or terraform apply says the elastic beanstalk resource has changed.

Steps to Reproduce

1. terraform apply
2. terraform plan or terraform apply

   References

   • GH-6890
   • GH-8742

[stuffandthings]

Any update on this issue? I'm still being plagued by this. Looking at the diff in terraform plan I only see setting.#: "22" => "27" having been changed, and consecutive terraform applys show both these numbers unchanged (i.e. setting.# does not correctly set to 27 after the apply)

[michalc]

I have solved this issue for my particular case: I was setting an instance profile using an ARN...

setting {
  namespace = "aws:autoscaling:launchconfiguration"
  name      = "IamInstanceProfile"
  value     = "${aws_iam_instance_profile.geolog_instance.arn}"
}

... and it appeared to work fine, but it suffered from the not-sticking problem, and every terraform plan or terraform apply seemed to want to recreate the setting. Changing it to the instance profile name seemed to fix it

setting {
  namespace = "aws:autoscaling:launchconfiguration"
  name      = "IamInstanceProfile"
  value     = "${aws_iam_instance_profile.geolog_instance.name}"
}

[vchan2002]

I still have the same issue..... where, if you look at the plan, it seems to 'forget' the following settings:

namespace = "aws:elb:listener:443"
name      = "SSLCertificateID"

namespace = "aws:autoscaling:launchconfiguration"
name      = "Ec2KeyName"

namespace = "aws:elb:listener"
name      = "InstanceProtocol"

Once I commented out the settings in my config and ran a terraform plan,

[shawnmclean]

I think I may have narrowed this down alittle more to just having SSLCertificateId settings there.

[felipe-vvoosh]

I'm having the same issues, even worse for me as it's understanding only 24 of my 39 settings...

[m1schka]

is there an update on this? it's kinda annoying when updating parts of my infrastructure not related at all to beanstalk. when you have rolling update enabled, this procedure takes like 5 minutes or more every time you apply...

[felipe-vvoosh]

Hi @m1schka, you need to add this: lifecycle { ignore_changes = ["setting"] } to your aws_elastic_beanstalk_environment

[m1schka]

thx @felipe-vvoosh that's a good workaround! Is it possible to only narrow it down to the application version so it picks up all the rest?

[felipe-vvoosh]

I don't know... I googled it but saw people doing it for tags only...

[saxonww]

I'm seeing this on any setting in the aws:elasticbeanstalk:environment:process(:*) namespaces, but only when I've set aws:elasticbeanstalk:environment/EnvironmentType to SingleInstance. It's the same behavior everyone else is reporting:

   setting.2782498060.name:        "" => "Port"
   setting.2782498060.namespace:   "" => "aws:elasticbeanstalk:environment:process:registry"
   setting.2782498060.resource:    "" => ""
   setting.2782498060.value:       "" => "9090"

The resource is being updated with every plan/apply even when there is not supposed to be a change.

I don't think ignoring settings is a workaround at all. The majority of the TF configuration for beanstalk is wrapped up in those settings blocks; ignoring these means any changes I do want to make are not picked up.

EDIT: If it's not obvious to you - it wasn't to me, and the documentation is not explicit about this - the above namespace/settings are only valid in a load balanced environment. So what's going on is that TF is supplying the settings, and while AWS takes them, it does not persist them. My guess is that some of the above similar issues are due to setting values that are not relevant for the associated beanstalk environment.

[nszilard]

Hi all,

this is affecting us as well. We are setting SQS, ElastiCache and Environment auto terminating if unhealthy options in the environments, and after the configuration is applied, each consecutive plan/apply will try to re-set those options (even though they have been already applied to the env - checked at amazon console).

Terraform version: Terraform v0.10.8

[jamesrwhite]

I'm seeing the same issue as @saxonww where the resource is being updated on every apply, regardless of if I have made a change.

[Igorshp]

@jamesrwhite also having same problem :)

[serenitus]

I am also seeing a similar issue with the listener_port namespace ("aws:elb:listener:443" in my case). There is some indication that the settings have been used according to the config of the ELB in the EC2 console. However, settings are not reflected in the Elastic Beanstalk configuration display.

[jordanconway]

I'm seeing this with SSLCertificateID Terraform v0.11.3 provider.aws v1.6.0

      setting.179171839.name:       "" => "SSLCertificateID"
      setting.179171839.namespace:  "" => "aws:elb:listener:443"
      setting.179171839.value:      "" => "arn:aws:acm:us-west-2:xxx:certificate/xxx"
      setting.3168095902.name:      "" => "SSLCertificateID"
      setting.3168095902.namespace: "" => "aws:elb:listener:444"
      setting.3168095902.value:     "" => "arn:aws:acm:us-west-2:xxx:certificate/xxx"
      setting.3529276132.name:      "" => "SSLCertificateID"
      setting.3529276132.namespace: "" => "aws:elb:loadbalancer"
      setting.3529276132.value:     "" => "arn:aws:acm:us-west-2:xxx:certificate/xxx"

[serenitus]

I got frustrated and just switched to use an application load balancer instead of classic. So far, so good, and my options are all sticking too.

[whyvez]

@serenitus Could you share which settings you're setting to get this to work?

[ebarault]

it seems none of the settings from the aws:elasticbeanstalk:environment:process:defaultnamespace are applicable when using a classic loadbalancer.

also i'm not able to change the Stickiness Policy setting from the aws:elb:policies namespace.

I need to use the specify the default policy name as in

  setting {
    namespace = "aws:elb:policies:AWSEB-ELB-StickinessPolicy"
    name       = "Stickiness Policy"
    value     = "false
  }

to have the Stickyness Policy setting applied, but then it's not sticking and gets reapplied every time... great!

[github-actions[bot]]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!