File "/var/task/union.py", line 61, in lambda_handler

[davidrobertson]

I've tried doing this both via the instructions, as well as via CloudFormation. Both ways result in the following error:

'Item': KeyError Traceback (most recent call last): File "/var/task/union.py", line 61, in lambda_handler instance = instance['Item']['InstanceAttributes'] KeyError: 'Item'

[jicowan]

David, when do you get this error? When you terminate an instance? Can you send me the item for this instance in the DDB table?

[davidrobertson]

I get the error when I try to start, stop, or terminate an instance. DynamoDB table appears to be empty.

[davidrobertson]

Looks like it's timing out, which I didn't notice before. Here's everything in the log:

START RequestId: c8bc718c-0f51-11e7-89e7-91fb6821101a Version: $LATEST DynamoDB table already exists END RequestId: c8bc718c-0f51-11e7-89e7-91fb6821101a REPORT RequestId: c8bc718c-0f51-11e7-89e7-91fb6821101a Duration: 30002.30 ms Billed Duration: 30000 ms Memory Size: 128 MB Max Memory Used: 38 MB
2017-03-22T22:53:41.791Z c8bc718c-0f51-11e7-89e7-91fb6821101a Task timed out after 30.00 seconds

[jicowan]

Thanks David. Can you add the following to line 25: print event this will output the full event to the log group for the function. I'd like to see what is being passed to the function.

[davidrobertson]

After adding 'print event' to line 25, nothing new was being logged. After realizing the DDB table is being created (just left empty), I've moved 'print event' to line 23. This is what's logged now:

{u'account': u'xxx', u'region': u'us-east-1', u'detail': {u'state': u'running', u'instance-id': u'i-0698e7d5720e30809'}, u'detail-type': u'EC2 Instance State-change Notification', u'source': u'aws.ec2', u'version': u'0', u'time': u'2017-03-23T13:26:47Z', u'id': u'717aa547-878c-4591-b737-92a84c4bb064', u'resources': [u'arn:aws:ec2:us-east-1:367839381035:instance/i-0698e7d5720e30809']}

[jicowan]

I'm having trouble seeing where the problem is. The function extracts the instance ID from the event, instance_id=event[detail][instance-id] and then passes it to the describe_instances method. Can you add another print instance statement after line 37 and 42? Does the IAM policy you attached to the function have permission to query EC2?

[davidrobertson]

The role (ddns-lambda-role) has access to EC2, per the policy: {"Version":"2012-10-17","Statement":[{"Action":"ec2:Describe","Resource":"","Effect":"Allow"},{"Action":["dynamodb:"],"Resource":"","Effect":"Allow"},{"Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],"Resource":"","Effect":"Allow"},{"Action":["route53:"],"Resource":["*"],"Effect":"Allow"}]}

That said, the Access Advisor says that the role has not tried to access either EC2, nor Route 53 at all

Amazon CloudWatch Logs ddns-lambda-role Today Amazon DynamoDB ddns-lambda-role Today Amazon EC2 ddns-lambda-role Not accessed in the tracking period Amazon Route 53 ddns-lambda-role Not accessed in the tracking period

Remember - this is all done via the CloudFormation template.

[davidrobertson]

Also, there are no new log showing up after adding 'print instance' after lines 37 and 42.

[jicowan]

That policy doesn't look right. Can you try replacing it with the policy in the ddns-pol.json file or re-post it here surrounded by triple `. Thanks for your patience.

[davidrobertson]

No, thanks for your patience.

I've replaced the policy, and it's still not working.

Still nothing useful in the logs either:

'Item': KeyError Traceback (most recent call last): File "/var/task/union.py", line 63, in lambda_handler instance = instance['Item']['InstanceAttributes'] KeyError: 'Item'

[davidrobertson]

I figured it out.

There's a 'sleep 60' in there, but the lambda app has a 30 second timeout. I raised the the lambda app timeout to 90 seconds, and everything is working fine now.

Thanks again...

[jicowan]

Awesome! I may need to increase the timeout in the CFN template.

[mustanggb]

Just ran into this issue, any chance you can update the blog as well? https://aws.amazon.com/blogs/compute/building-a-dynamic-dns-for-route-53-using-cloudwatch-events-and-lambda/

[jstrahn]

I also encountered this same error. Thank you @davidrobertson for figuring this out. Saved me a lot of time.

[painty1]

Hi. I'm getting this error also. I have tried increasing the timeout in the function but still get the same error. Does it make a difference that I'm using a windows instance for this? Thanks