Honeypot feature add CloudFront IP to Bad Bot IP List

[peteroruba]

Hello, I have deployed the Honeypot feature and defined both an Origin and Behavior in my CloudFront distribution. When I issue a request to the Honeypot path I see a CloudFront IP (64.252.82.168/32) being added to the Bad Bot IP list instead of my actual IP.

As per https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html the aforementioned IP belongs to AWS:

{
  "ip_prefix": "64.252.82.0/24",
  "region": "eu-west-2",
  "service": "AMAZON",
  "network_border_group": "eu-west-2"
},

[aijunpeng]

What is the version of the solution you are using?

[peteroruba]

@aijunpeng I am using v2.3.3

[aijunpeng]

Bad bot lambda (access-handler.py) extracts the source ip(event['requestContext']['identity']['sourceIp']) from the response returned by the request. Can you see what source ip is in cloudwatch logs for the lambda? Also please provide detailed steps on how you configured your honeypot link in your web app and how you issued your request to help our investigation. Thank you!

[peteroruba]

Hi @aijunpeng, sorry for the late response. My setup ist as follows:

• CloudFront distribution deployed via a CloudFormation stack in eu-central-1
• AWS WAF Security Automations deployed in us-east-1
• Modified the reference to API Gateway Hostname in CloudFront stack (eu-central-1):
• I did not set up a link in our webapp, the only thing I have configured is a CloudFront Behavior (set to /.git/* ) and CloudFront Origin (set to Bad Bot API Gateway)
• Manually entered the afore mentioned CloudFront distribution in AWS WAF as the associated resource

CloudWatch Logs output for the bad bot Lambda say: sourceIp': '64.252.86.180'

The request was issued from the terminal via curl to the honeypot url ("/.git/")

[aijunpeng]

Thanks for the detailed information. I followed your steps and ran a test. My test worked as expected - my actual sourceip instead of cloudfront ip was added to the bad bot ip set. What does your CloudFront distribution look like after you "Modified the reference to API Gateway Hostname in CloudFront stack (eu-central-1)"? Please send a screenshot that shows its configurations. You can go to AWS CloudFront console -> select distribution -> take a screenshot of the page. Also please send a screenshot of your CloudFront origin. You can select Origins and Origin Groups tab -> select the origin -> Edit -> take a screenshot of the Edit Origin page. Lastly, I would suggest you to manually create a CloudFront distribution via AWS console instead of using CloudFormation stack for testing purpose. After creating the distribution, create origin by following steps here: https://docs.aws.amazon.com/solutions/latest/aws-waf-security-automations/deployment.html#step3. Ignore step 8 and 9 if you don't want to set up robots.txt file to disallow your web application to access to the honeypot link.

[peteroruba]

@aijunpeng I'm sorry, I forgot to mention that I deployed version v.3.0.0 this time. Either way, the observed effect is the same.

I am attaching a set of screenshots.

[peteroruba]

One interesting observation - WAF sees my actual IP, when check the details of a sampled request:

[aijunpeng]

Thanks for all the information! The cloudfront configurations on your screenshots look fine. I mimicked your configurations and ran another test. It worked for me again. I would suggest you to delete all ips from the badbot ip set to do a fresh test. Also what was the ip in the message returned by your curl command? The message is like "{"message": "[your ip] Thanks for the visit."}. That ip is the one should be added to the badbot ip set. You can also get the message by going to AWS CloudFormation Console -> select your waf stack -> select Outputs tab -> click on the honeypot link.

I would also like a copy of the cloudwatch logs for your badbot lambda function if the issue remains.

[peteroruba]

Output from curl accessing the honeypot URL in our application (via CloudFront): message: [64.252.86.241] Thanks for the visit.* Closing connection 0 (CloudFront IP)

Output from curl accessing the honeypot URL in API Gateway: message: [94.231.XXX.XX] Thanks for the visit.* Closing connection 0 (My actual IP)

[peteroruba]

Attaching the badbot lamda logs (replaced my actual IP with "1.1.1.1")

badbotlambdalogs.zip

[peteroruba]

@aijunpeng I noticed that the Lambda logs from the test via CloudFront show my actual IP in "X-Forwarded-For".

[aijunpeng]

Thanks for the information. It is very helpful. Finally I am able to reproduce the issue and found out the reason for it. It is a bug in the code. We have added it to our backlog and will fix it in our future release. For now, if you can, as a workaround you can manually modify your badbot lambda (access-handler.py) to fetch "X-Forwarded-For" ip for requests from cloudfront. Below is the basic logic for the fix:

1. When event['requestContext']['identity’][‘userAgent’] == ‘Amazon CloudFront’, fetch X-Forwarded-For[0] ip: source_ip = event['headers']['X-Forwarded-For'].split(',')[0].strip()
2. For other cases, use sourceip: source_ip = str(event['requestContext']['identity']['sourceIp’])

I apologize for the inconvenience. Thanks again for reporting the issue!

[peteroruba]

@aijunpeng thank you so much for your assistance. I confirm the issue has been resolved.

[Patrick-56Bit]

I have the same issue. @aijunpeng can you confirm whether this will be resolved soon?

[aijunpeng]

@Patrick-56Bit I don't have an ETA. It is in our backlog and should be addressed when we release our next solution update. I apologize for the inconvenience. There is a workaround above that you can try.

[burnzy]

Looks like this issue was caused by this PR that was done for security reasons: https://github.com/awslabs/aws-waf-security-automations/pull/123

[estahn]

@aijunpeng Any updates on this?

[dcopestake]

I've raised pull request #195 which attempts to address this by changing to use an injected header of True-Client-Ip instead (via a CloudFront function). Seems to be doing the trick for me.

[aijunpeng]

Thanks for your contribution. We have added your request to our solution backlog items and it will be evaluated in future solution releases.

[dscpinheiro]

Hi!

We just released v3.2.0 of the solution, and this issue has been fixed.