Parameter Store config client IAM permissions, environments and application startup issue

[Holophrasis-uk]

Issue description

When deploying an app to Elastic Beanstalk the Micronaut environments are set to ec2, cloud along with any others specified (in this case uat).

This causes the config client to look for parameters under application, application, appname, appname as documented.

The problem arises when setting the permissions for the ec2 role and access to the parameter store. If we do not specify allow access to parameter store paths such as /config/appname_ec2 then the application fails to start. We don't require any parameters to be stored under this path so this extra setup on the IAM role seems a burden? It's also, perhaps, fragile if a new environment was introduced to Micronaut we would have to add permission to allow access to this even if we don't use it.

Would it be better (more polite) behavior to show a warning about being unable to access some automatically generated paths rather than the exception which stops the application from starting?

(error shown is Error starting Micronaut server: Error reading distributed configuration from AWS Parameter Store: software.amazon.awssdk.services.ssm.model.SsmException: etc)

[sdelamo]

If I understand correctly, the issue is that you are forced to write a more verbose policy. Could you past here the JSON policy you are forced to create and what you would ideally want.

[Holophrasis-uk]

"Action": [
                "ssm:GetParametersByPath",
                "ssm:GetParameters",
                "ssm:GetParameter"
            ], 
"Resource": [
                "arn:aws:ssm:*:<id>:parameter/product/adv2_uat*",
                "arn:aws:ssm:*:<id>:parameter/product/adv2_uat",
                "arn:aws:ssm:*:<id>:parameter/product/adv2_ec2*",
                "arn:aws:ssm:*:<id>:parameter/product/adv2_ec2",
                "arn:aws:ssm:*:<id>:parameter/product/adv2_cloud*",
                "arn:aws:ssm:*:<id>:parameter/product/adv2_cloud",
                "arn:aws:ssm:*:<id>:parameter/product/adv2",
                "arn:aws:ssm:*:<id>:parameter/product/adv2/*",
                "arn:aws:ssm:*:<id>:parameter/product/application*"
            ]

I'm not very good at specifying the correct policies and tend to add too many '/' and '*'s.

The problem I found was that I needed to specify '_cloud', '_ec2', 'application' etc even if they are not being used (there are only parameters under adv2_uat/). I didn't want to use 'adv2*' as that would pull back other environment that this role/policy should not have access to (prod etc).

It's entirely possible I'm not using this correctly or some other change to the policy would make all the difference.

So ideally the policy would just have "arn:aws:ssm:*:<id>:parameter/product/adv2_uat*" .

Thanks.

[sdelamo]

I will check this the week August the 30th.

[sdelamo]

@Holophrasis-uk something to try is to disable environment deduction and remove any environment you don't use:

You could specify these two environment variables directly in Elastic Beanstalk Configuration:

MICRONAUT_ENVIRONMENTS to uat MICRONAUT_ENV_DEDUCTION to false

However, it may not work because I think some of our features require ec2 environment to be present.

I think what we have to do is allow the user to configure the environments they want to be checked for distributed configuration.

If you don't specify anything we search in every resolved environment but you can specify via configuration. For AWS Parameter store only use environmentsuat. Will that work for you? What do you think?

[Holophrasis-uk]

Thanks for the update. The AWS parameter store client requires the ec2 env (which I'm not sure is the 100% correct behaviour? Maybe should only require that it has a config and is enabled? If you only want it for ec2 environments then you have that config in the bootstrap-ec2.yml. The ec2 environment probably controls other behaviour in Micronaut).

I think your solution of the user specifying the environments/paths to search is the correct one.

Thanks.

[miguelaferreira]

I'm facing a similar issue reading configuration from SSM parameter store.

I'm running micronaut version 3.5.0.

My bootstrap.yml looks like this:

micronaut:
  application:
    name: notifications
  config-client:
    enabled: true

aws:
  client:
    system-manager:
      parameterstore:
        enabled: true
        search-active-environments: false

The parameters for my app are stored under /config/notifications/*, and therefore the policy I want to set on the IAM role assumed by the app is like this:

{
    "Action": [
        "ssm:GetParameter",
        "ssm:GetParameters",
        "ssm:GetParametersByPath"
    ],
    "Effect": "Allow",
    "Resource": [
        "arn:aws:ssm:eu-central-1:123456789100:parameter/config/notifications/*"
    ],
    "Sid": ""
}

The error I'm getting is the following:

Caused by: software.amazon.awssdk.services.ssm.model.SsmException: 
  User: arn:aws:sts::123456789100:assumed-role/app-notification/app-notification is not authorized to perform: 
    ssm:GetParameters on resource: arn:aws:ssm:eu-central-1:123456789100:parameter/config/application 
    because no identity-based policy allows the ssm:GetParameters action 
    (Service: Ssm, Status Code: 400, Request ID: 0540a7b9-ffca-42d2-a9a9-cf1b258d84ef)

It seems I need to allow the application IAM role to read all the parameters that are tried, otherwise if it tries to read parameters under a specific path that it is not allowed, the app won't start.

The app will boot and be configured if I set the policy for the role like this:

{
    "Action": [
        "ssm:GetParameter",
        "ssm:GetParameters",
        "ssm:GetParametersByPath"
    ],
    "Effect": "Allow",
    "Resource": [
        "arn:aws:ssm:eu-central-1:123456789100:parameter/config/application*",
        "arn:aws:ssm:eu-central-1:123456789100:parameter/config/notifications*"
    ],
    "Sid": ""
}

It would be best to not have to allow the app to read parameters under /config/application* when there won't be any parameters there anyway. In addition to that, being able to place a forward slash / between the application name and the wildcard * would also be good because it would prevent matching parameters for different applications that happen to have a name that starts with notifications. Basically do this

"arn:aws:ssm:eu-central-1:123456789100:parameter/config/notifications/*"

instead of

"arn:aws:ssm:eu-central-1:123456789100:parameter/config/notifications*"