Cfn stack create keeps failing with error 'Value '' at 'value' failed to satisfy constraint'

[Shukla-Ankur]

I am trying to find the latest snapshot identifier for a RDS instances. I use this output to restore a DB instance from the snapshot identifier. I have given RDSFullReadAccess to the commandrunner role

cfn template :

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
  Generator: "cfn"
Description: "test"

Resources:
  LatestSnapshotIdentifier:
    Type: AWSUtility::CloudFormation::CommandRunner
    Properties:
      Command: 'aws rds describe-db-snapshots --region ap-south-1 --db-instance-identifier ap-s1-test-rds-pg-sg-0 --query "DBSnapshots[0].DBInstanceIdentifier" --output text > /command-output.txt'

Expected output: commandrunner should successfully execute this command

actual output: getting an error

1 validation error detected: Value '' at 'value' failed to satisfy constraint: Member must have length greater than or equal to 1. (Service: AWSSimpleSystemsManagement; Status Code: 400; Error Code: ValidationException; Request ID: 341ac0af-5b67-46fd-9cdd-d99504a9e7eb)
--

I tried a bunch of things- in order to see if this is permissions issue,

1. modified template to run this command -

aws s3 mb s3://test-storage-as-0 --region ap-south-1 > /command-output.txt

failed

2. aws s3 ls |tail -n 1 > command-output.txt

   Failed

4. aws ec2 describe-instances --instance-ids i-042f829ed0c341724 --region ap-south-1 --profile dev --query="Reservations[0]|Instances[0]|InstanceId

Failed

The overall experience was that cfn failed to work with any aws cli command in general.

[izzaheer]

This issue Happens when you don't specify LogGroup and/or Role. Both these Properties are required to prevent this error from occurring. It looks like the Readme needs to be updated to set Required: Yes for Role and LogGroup

[craigataws]

@Shukla-Ankur. Thanks for the detailed desription. I think the issue you're running into is that the commandrunner expects that anything put into command-output.txt is a single word. In the readme:

The value written to the file must be a single word value without quotation marks like vpc-0a12ab123abc9876 as they are intended to be used inside the CloudFormation template using Fn::GetAtt.

Can you expand on your use cases? Where would you be putting/referencing the output from the commands you called out above? (aws ec2 describe-instances, aws s3 mb, aws rds describe-db-snapshots)

[Shukla-Ankur]

1. @izzaheer I ran cnf.yaml from this doc https://aws.amazon.com/blogs/mt/running-bash-commands-in-aws-cloudformation-templates/ and it worked which means LogGroup and role are not mandatory. 'Role' field actually accepts profile, so the name is kinda misleading. Earlier I was creating role (and profile )via console, so they had same name. This was causing issues here. Once I manually created profile and role with different names using CLI locally, it worked. Another thing, the version for which this failed does not auto-create loggroup.

2. cleanup.sh fails to deregister the default version of CR in cfn registry. I had to manually deregister it. But it seems to have been fixed in latest version of CR

3. I ran into another issue wherein when i registered commandrunner in our different AWS account. I verified that the executionrole and the self-created role had same trust policies and permissions. However, it kept failing in new account on stack creation giving just the error- 'Index: 0, Size: 0'

I've run into 3-4 issues in last 2 days. It is not a good solution to use in production. Even AWS support teams were not able to get any solutions. I found out the problem myself while going thought documentation and then experimenting with profile-role. Biggest issues has been around the error info either being negligible or completely cryptic.

[craigataws]

@Shukla-Ankur Agreed on the error text returned, we have opened up in #14 to improve error handling cases. If you can open up an issue with detailed steps to reproduce on the Index: 0, Size: 0, we can address this as well.

[Shukla-Ankur]

@craigataws I did not follow any steps other than standard registration in one of our 2 AWS accounts. It worked fine in one while kept failing in another.

1. Registered commandrunner in ap-southeast-1 region in AWS account
2. Created an IAM role with ec2 trust relationship
3. Added RDS read permissions, cloudwatch permissions(3 as mentioned in CR documentation)
4. used my previous templates (working in another AWS account) and also sample cfn template from https://aws.amazon.com/blogs/mt/running-bash-commands-in-aws-cloudformation-templates/

cfn was unable to create the stack. It could not even launch cfn stack to create the commandrunner resource. Only error seen is

Index: 0, Size: 0

Could not find anything in CW (cfn could not publish anything to CW loggroup obviously) and nothing in cloudtrail.

[zs787]

Hope this helps, I have recently used CommandRunner and had the same issue @Shukla-Ankur had in his last comment Index: 0, Size: 0 so tried to see if CloudTrail can provide any information related to the error and the first thing I noticed is this.

"errorCode": "AccessDenied", "errorMessage": "User: arn:aws:sts::...:assumed-role/awsutility-cloudformation-commandrun-ExecutionRole-17YW0CROMIEWI/5e864644-7aaf-9f67-3f59-001d87c7bb50 is not authorized to perform: logs:CreateLogGroup on resource: arn:...:log-group:awsutility-cloudformation-commandrunner-logs2:log-stream:"

Example used: https://aws.amazon.com/blogs/mt/running-bash-commands-in-aws-cloudformation-templates/

After fixing the permission and adding a LogGroup and a Role I was able to go past the above issue. However, now im getting this one and no information about what could be the issue.

However, now im getting this error Resource handler timed out.

[zs787]

I would also like to add another comment related to what @Shukla-Ankur mentioned above, where it worked in one account but not the other. This happened to me as well, it works in one account in us-east-1 but in another account (in us-west-2) I get Resource handler timed out.

Note: I followed the exact same installation steps in the read me and used the same exact test template in both regions/accounts, as well as a user with full admin privileges.

[Shukla-Ankur]

@zs787 I noticed this in one case i.e. the permissions issue. However, I verified that the role I was using has all the required permissions for CW but it still continued to fail with same error.

[zs787]

@zs787 I noticed this in one case i.e. the permissions issue. However, I verified that the role I was using has all the required permissions for CW but it still continued to fail with same error.

Two things I have noticed during an initial deployment in the account that is having the issues:

1. The role created by CommandRunner seems to need more permissions like the ability to create log group (Note: first time I used an IAM user that does not have full admin permissions, next time I used an admin user, which seems to have solved the first issue but not the time timeout)

The question here is would the IAM user running the initial deployment makes a difference in how the finial awsutility-cloudformation-commandrun-ExecutionRole... will look like ?

2. Also, I think some of the properties are not optional but rather mandatory.

I may be mistaken but looks we both had similar errors and log group is empty with no information on what could be wrong.

[shantgup]

Hi all, I just read through this issue.

The Index: 0, Size: 0 error occurs when you don't have a Default VPC in that region. The solution is simply to specify SubnetId and it will use that Subnet instead and automatically infer the VPC from the SubnetId. I agree that the error message needs to be improved, and I will be fixing it in https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-awsutilities-commandrunner/issues/14.

[kkress2000]

I too was receiving this error and added the role and LogGroup properties. This allowed me to get the actual command execution to work, but cloudformation still recorded the Value error. I solved this by writing a value into /command-output.txt. Apparently, you must have an output value. My command does not, so I have to spoof it. The cloudformation template now succeeds.

Based on the discussion, the following are required:

• default VPC
• Role
• LogGroup
• Single (string) value in /command-output.txt

Here's what is working for me: CommandRunner: Type: AWSUtility::CloudFormation::CommandRunner Properties: Role: command-runner-role LogGroup: /aws/cloudformation/commandrunner Command: >- set -xe && ls -l && pip3 install --user boto3 argparse && aws s3 cp s3://command-bucket-8ujn3efh/scripts/write-secure-ssm.py . && ls -l && python3 write-secure-ssm.py --hint="Z3zn0Ne" && echo 'Success' > /command-output.txt