ssm put_parameter add to default

[thoroc]

Hi,

I have been trying to understand how to add the following ssm parameter to the default sets provided in https://github.com/spulec/moto/blob/master/moto/ssm/resources/ami-amazon-linux-latest/eu-west-1.json:

• /aws/service/ecs/optimized-ami/amazon-linux-2/recommended

But obviously this isn't possible by just calling put_parameter as showing in https://github.com/spulec/moto/blob/master/tests/test_ssm/test_ssm_boto3.py#L390

Is there an obvious way to do this with moto, or do I need to mock the ssm client without moto?

The result value from aws cli should be the following (at the time or writing):

{
    "Parameter": {
        "Name": "/aws/service/ecs/optimized-ami/amazon-linux-2/recommended",
        "Type": "String",
        "Value": "{\"ecs_agent_version\":\"1.63.1\",\"ecs_runtime_version\":\"Docker version 20.10.13\",\"image_id\":\"ami-002e2fef4b94f8fd0\",\"image_name\":\"amzn2-ami-ecs-hvm-2.0.20220921-x86_64-ebs\",\"image_version\":\"2.0.20220921\",\"os\":\"Amazon Linux 2\",\"schema_version\":1,\"source_image_name\":\"amzn2-ami-minimal-hvm-2.0.20220912.1-x86_64-ebs\"}",
        "Version": 94,
        "LastModifiedDate": 1664230158.399,
        "ARN": "arn:aws:ssm:eu-west-1::parameter/aws/service/ecs/optimized-ami/amazon-linux-2/recommended",
        "DataType": "text"
    }
}

[bblommers]

Hi @thoroc, there is no way to do this using Moto at the moment, unfortunately.

I'll mark it as an enhancement for Moto to store these parameters (similarly to what we do for /aws/service/ami-amazon-linux-latest and /aws/service/global-infrastructure-params).

[thoroc]

Hi @bblommers,

Thank you for confirming that. I'll explore other avenue to run my tests.

[thoroc]

Btw are you thinking about doing the same as for this PR: https://github.com/spulec/moto/pull/5427?

[bblommers]

Yes, that's the idea @thoroc. If you're looking to tackle this yourself, PR's are always welcome!

[thoroc]

Where would it make more sense to save this data from AWS? I was thinking of retrieving everything under /aws/service/ecs/optimized-ami/.

I have tried to retrieve everything under /aws/service/ecs/ and everything under /aws/service/ecs/optimized-ami/ and comparing the output look the same.

[bblommers]

The exact location where to store the data doesn't matter much, as long as the naming is sensible, for example moto/ssm/resources/ecs/.

If there is no difference, than we might as well retrieve everything under /aws/service/ecs - if AWS decides to add more data, we're already set-up to retrieve it.

[thoroc]

Ok I was wrong, somehow I have managed to retrieve a subset only with a python script (I think I needed to use the paginator). Using the aws cli there are a lot more items calling /aws/service/ecs than calling /aws/service/ecs/optimized-ami, which makes sense.

[bblommers]

FYI @thoroc , our scripts directory has some examples on calling SSM with a paginator. You can always copy/pasting one of those.

[thoroc]

Ok I was doing something stupid, but the result is that I can confirm there is currently as many parameters in /aws/service/ecs/ as in /aws/service/ecs/optimized-ami.

To confirm this I have run the following commands:

λ aws ssm get-parameters-by-path --path=/aws/service/ecs/optimized-ami --recursive | grep ARN | wc -l
3990

and

λ aws ssm get-parameters-by-path --path=/aws/service/ecs --recursive | grep ARN | wc -l
3990

That means the Python code was returning the correct values. Sorry for the confusion.

[thoroc]

You can always copy/pasting one of those.

Ah I see. One thing I am wondering though. Is it the correct thing to convert the field LastModifiedDate from a string to a timestamp? the response from the AWS cli is a string after all for this field.

λ aws ssm get-parameter --name=/aws/service/ecs/optimized-ami/amazon-linux-2/ami-04030f151d88f43bc
{
    "Parameter": {
        "Name": "/aws/service/ecs/optimized-ami/amazon-linux-2/ami-04030f151d88f43bc",
        "Type": "String",
        "Value": "{\"ecs_agent_version\":\"1.62.2\",\"ecs_runtime_version\":\"Docker version 20.10.13\",\"image_id\":\"ami-04030f151d88f43bc\",\"image_name\":\"amzn2-ami-ecs-hvm-2.0.20220822-x86_64-ebs\",\"image_version\":\"2.0.20220822\",\"os\":\"Amazon Linux 2\",\"schema_version\":1,\"source_image_name\":\"amzn2-ami-minimal-hvm-2.0.20220805.0-x86_64-ebs\"}",
        "Version": 1,
        "LastModifiedDate": "2022-08-24T20:46:23.449000+01:00",
        "ARN": "arn:aws:ssm:eu-west-1::parameter/aws/service/ecs/optimized-ami/amazon-linux-2/ami-04030f151d88f43bc",
        "DataType": "text"
    }
}

[bblommers]

Moto intercepts the underlying HTTP request that is send by the SDK, and the AWS CLI/boto3 will then convert the HTTP response to whatever format they want. I haven't checked, but I'm assuming that the HTTP response from AWS contains a timestamp, so that's what Moto should return as well

[bblommers]

Hi @thoroc, just checking on this - is this still on your radar?

If you're stuck, or don't have any time to finish up, feel free to open up a draft PR with what you have so far, I'd be happy to finish it.

[thoroc]

Hey @bblommers I'll admit that has slipped down the list of priorities over here. I haven't deleted any of the experiments I have done to try and fit that in though: https://github.com/thoroc/moto/tree/feature/ssm-ecs

[bblommers]

Thanks for sharing the branch @thoroc. I've taken a shot at this, based on your script/commits - the linked PR will add these parameters to Moto.

The biggest change I made is probably the storage format: this is all now stored in a tree-format, which saves a lot of space by removing duplicate information. Other SSM parameters are stored in the same format, and they are all decoded back to the AWS format on request.

[thoroc]

Oh wow. Thank you for sharing this @bblommers.