AWS::ElasticBeanstalk::Environment - OptionSettings: Updates Can Remove Manually Added Environment Variables

[colinbjohnson]

1. Title

AWS::ElasticBeanstalk::Environment-OptionSettings: Updates Can Remove Environment Variables

2. Scope of request

AWS::ElasticBeanstalk::Environment-OptionSettings - aws:elasticbeanstalk:application:environment

3. Expected behavior

In Create or Update, a AWS::ElasticBeanstalk::Environment-OptionSettings - change should not

remove aws:elasticbeanstalk:application:environment environment variables added outside of CloudFormation (the present behavior) - the behavior when updating other resources (i.e. a S3 bucket) with property outside of CloudFormation is to leave these properties alone

but rather could:

• warn the user that environment variables (aws:elasticbeanstalk:application:environment values) will be removed
• allow the user to control the behavior using reset (where all OptionSettings are returned to values in CloudFormation stack), updateoradd (where all OptionSettings are updated or added and existing configuration is left untouched)
• have a documentation update here (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-beanstalk-environment.html#cfn-beanstalk-environment-optionsettings) where OptionSettings documentation is updated to reflect manually created environment variables will be removed.
• do something else

Feel free to describe other desired and relevant implementation details.

4. Suggest specific test cases

To test this:

create a CloudFormation stack using the template below

---

AWSTemplateFormatVersion: 2010-09-09

Description: AWS CloudFormation Stack for Running ElasticBeanstalk.

Parameters: {}

Mappings: {}

Resources:

  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/24

  InternetGateway:
    Type: AWS::EC2::InternetGateway

  PublicInternetRoute:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway
      RouteTableId: !Ref PublicInternetRouteTable

  VPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

  PublicInternetRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC

  PublicSubnet01:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select [ 0, !GetAZs "" ]
      CidrBlock: 10.0.0.0/25
      VpcId: !Ref VPC

  PublicSubnet02:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select [ 1, !GetAZs "" ]
      CidrBlock: 10.0.0.128/25
      VpcId: !Ref VPC

  PublicSubnet01RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicInternetRouteTable
      SubnetId: !Ref PublicSubnet01

  PublicSubnet02RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicInternetRouteTable
      SubnetId: !Ref PublicSubnet02

  ElasticBeanstalkApplication:
    Type: AWS::ElasticBeanstalk::Application
    Properties:
      # note that if ApplicationName is set and is not unique you
      # will not able able to run a given stack multiple times
      ApplicationName: !Ref AWS::StackName

  ElasticBeanstalkIAMRole:
    Type: AWS::IAM::Role
    Properties: 
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns: 
        - arn:aws:iam::aws:policy/AWSElasticBeanstalkWebTier

  ElasticBeanstalkInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties: 
      Roles: 
        - !Ref ElasticBeanstalkIAMRole

  ElasticBeanstalkConfigurationTemplate:
    Type: AWS::ElasticBeanstalk::ConfigurationTemplate
    Properties:
      ApplicationName: !Ref ElasticBeanstalkApplication
      OptionSettings:
        - Namespace: aws:autoscaling:asg
          OptionName: MinSize
          Value: 1
        - Namespace: aws:autoscaling:asg
          OptionName: MaxSize
          Value: 1
        - Namespace: aws:autoscaling:launchconfiguration
          OptionName: IamInstanceProfile
          Value: !Ref ElasticBeanstalkInstanceProfile
        - Namespace: aws:autoscaling:launchconfiguration
          OptionName: InstanceType
          Value: t3.micro
        - Namespace: aws:elasticbeanstalk:application:environment
          OptionName: CITY
          Value: SAN_FRANCISCO
        - Namespace: aws:elasticbeanstalk:application:environment
          OptionName: STATE
          Value: CALIFORNIA
        - Namespace: aws:elasticbeanstalk:environment
          OptionName: EnvironmentType
          Value: LoadBalanced
        - Namespace: aws:elasticbeanstalk:environment
          OptionName: LoadBalancerType
          Value: application
        - Namespace: aws:ec2:vpc
          OptionName: VPCId
          Value: !Ref VPC
        - Namespace: aws:ec2:vpc
          OptionName: Subnets
          # Value: !Join turns the individual subnets into a string
          Value: !Join [ ",", [ !Ref PublicSubnet01, !Ref PublicSubnet02 ] ]
        - Namespace: aws:ec2:vpc
          OptionName: AssociatePublicIpAddress
          Value: true
      SolutionStackName: 64bit Amazon Linux 2 v3.1.5 running PHP 7.4

  ElasticBeanstalkEnvironment:
    Type: AWS::ElasticBeanstalk::Environment
    Properties:
      ApplicationName: !Ref ElasticBeanstalkApplication
      Description: AWS ElasticBeanstalk Environment
      # EnvironmentName must be greater than or equal to 4 (you can't use "QA")
      EnvironmentName: !Sub ${AWS::StackName}
      # TemplateName: with environments where every element is exactly the same
      # you may be able to use a single AWS::ElasticBeanstalk::ConfigurationTemplate
      TemplateName: !Ref ElasticBeanstalkConfigurationTemplate

The create the stack using a command similar to the following: aws cloudformation create-stack --stack-name EBwithEnvVars --capabilities CAPABILITY_IAM --template-body file://elasticbeanstalk_with_envvars.yaml

update an OptionSetting outside of CloudFormation

Example: outside of CloudFormation add an environment variable or c:

Namespace: aws:elasticbeanstalk:application:environment:
OptionName: COUNTRY
Value: UNITED_STATES

modify an OptionSettings property and run an "Update Stack" using a command similar to the below:

Example: change the following:

Namespace: aws:autoscaling:launchconfiguration
OptionName: InstanceType
Value: t3.micro

then run the following: aws cloudformation update-stack --stack-name EBwithEnvVars --capabilities CAPABILITY_IAM --template-body file://elasticbeanstalk_with_envvars.yaml

confirm the manually add Environment Variable has been removed

Configuration of environment variables outside of CloudFormation have been removed - in the case above this would have removed the environment variable COUNTRY.

5. Helpful Links to speed up research and evaluation

NA

6. Category

Use the categories as displayed in the AWS Management Console (simplified):

1. Compute

7. Any additional context

I'm not sure what the correct behavior should be... I know that existing changes made outside of CloudFormation are not removed (i.e. a property on an S3 bucket) and I know that any change that results in configuration removal is surprising behavior.

[zhihuayazhy]

@colinbjohnson The option setting from update-stack would override updates from elasticbeanstalk cli for the same environment, this is the expected behavior. Option settings should be added to the cfn template if the template would be reused. Are there any updates/features you are looking for this behavior?

[ced455]

Hi there, we ran into the same issue, we would like to manage the variable from our CI/CD, however every time we do a change in the CF file ALL variables are erased. We specifically did not set any aws:elasticbeanstalk:application:environment but we still have some OptionSettings in AWS::ElasticBeanstalk::ConfigurationTemplate is it why they get erased ?

I expect CF to not interact with my variable since I do not tell him to do so.

Should I open another ticket ?