hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.82k stars 9.16k forks source link

[Bug]: aws_guardduty_filter - The request is rejected because the JSON could not be processed #28367

Closed dnx-aiven closed 1 year ago

dnx-aiven commented 1 year ago

Terraform Core Version

1.3.6

AWS Provider Version

4.46.0

Affected Resource(s)

aws_guardduty_filter

Expected Behavior

Successful apply

Actual Behavior

Error & exit

Relevant Error/Panic Output Snippet

Error: error creating GuardDuty Filter: BadRequestException: The request is rejected because the JSON could not be processed.
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "9e34e998-6ced-422f-bcfa-fefea91f96d9"
│   },
│   Message_: "The request is rejected because the JSON could not be processed.",
│   Type: "InvalidInputException"
│ }
│
│   with module.us-east-2.aws_guardduty_filter.ssh_bruteforce,
│   on modules/guardduty/guardduty.tf line 33, in resource "aws_guardduty_filter" "ssh_bruteforce":
│   33: resource "aws_guardduty_filter" "ssh_bruteforce" {
│
╵
Releasing state lock. This may take a few moments...

Terraform Configuration Files

resource "aws_guardduty_detector" "primary" {
  enable = true

  datasources {
    s3_logs {
      enable = true
    }
    kubernetes {
      audit_logs {
        enable = false
      }
    }
    malware_protection {
      scan_ec2_instance_with_findings {
        ebs_volumes {
          enable = true
        }
      }
    }
  }
}

resource "aws_guardduty_filter" "ssh_bruteforce" {
  name        = "ssh-bruteforce-inbound"
  description = "Suppress inbound SSH bruteforce attempts"
  action      = "ARCHIVE"
  detector_id = aws_guardduty_detector.primary.id
  rank        = 1

  finding_criteria {
    criterion {
      field  = "type"
      equals = ["UnauthorizedAccess:EC2/SSHBruteForce"]
    }
    criterion {
      field  = "service.resourceRole"
      equals = ["TARGET"]
    }
  }
}

Steps to Reproduce

terraform apply of the above config.

Debug Output

2022-12-15T15:50:30.453+1100 [INFO]  Starting apply for module.eu-west-3.aws_guardduty_filter.ssh_bruteforce
2022-12-15T15:50:30.454+1100 [DEBUG] module.eu-west-3.aws_guardduty_filter.ssh_bruteforce: applying the planned Create change
2022-12-15T15:50:30.455+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: [DEBUG] setting computed for "tags_all" from ComputedKeys
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: [DEBUG] Creating GuardDuty Filter: {
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   Action: "ARCHIVE",
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   Description: "Suppress inbound SSH bruteforce attempts",
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   DetectorId: "0cc2727403030fa773d6ba56b22c5122",
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   FindingCriteria: {
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:     Criterion: {
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:       type: {
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:         Equals: ["UnauthorizedAccess:EC2/SSHBruteForce"]
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:       },
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:       service.resourceRole: {
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:         Equals: ["TARGET"]
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:       }
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:     }
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   },
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   Name: "ssh-bruteforce-inbound",
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   Rank: 1
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: }
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: [DEBUG] [aws-sdk-go] DEBUG: Request guardduty/CreateFilter Details:
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: ---[ REQUEST POST-SIGN ]-----------------------------
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: POST /detector/0cc2727403030fa773d6ba56b22c5122/filter HTTP/1.1
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Host: guardduty.eu-west-3.amazonaws.com
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.3.6 (+https://www.terraform.io) terraform
-provider-aws/4.46.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.44.153 (go1.19.3; linux; arm64)
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Content-Length: 363                                           
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Authorization: AWS4-HMAC-SHA256 Credential=REDACTED
dduty/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=REDACTED
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Content-Type: application/json
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: X-Amz-Date: 20221215T045030Z
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: X-Amz-Security-Token: REDACTED
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Accept-Encoding: gzip
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: {"action":"ARCHIVE","clientToken":"REDACTED","description":
"Suppress inbound SSH bruteforce attempts","findingCriteria":{"criterion":{"service.resourceRole":{"equals":["TARG
ET"]},"type":{"equals":["UnauthorizedAccess:EC2/SSHBruteForce"]}}},"name":"ssh-bruteforce-inbound","rank":1}
2022-12-15T15:50:30.456+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: -----------------------------------------------------
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: [DEBUG] [aws-sdk-go] DEBUG: Response guardduty/CreateFilter Details:
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: ---[ RESPONSE ]--------------------------------------
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: HTTP/2.0 400 Bad Request
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Access-Control-Allow-Headers: Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Sec
urity-Token,X-Amz-Content-Sha256,X-Amz-User-Agent,*,Date,X-Amz-Target,x-amzn-platform-id,x-amzn-trace-id
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Access-Control-Allow-Origin: *
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Access-Control-Expose-Headers: x-amzn-ErrorType,x-amzn-requestid,x-amzn-errormessage,x-
amzn-trace-id,x-amz-apigw-id,Date
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Access-Control-Max-Age: 86400
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Content-Type: application/json
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: Date: Thu, 15 Dec 2022 04:50:30 GMT
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: X-Amz-Apigw-Id: dK8fjEg8SQ0FkXg=
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: X-Amzn-Errortype: BadRequestException
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: X-Amzn-Requestid: 9b271113-a5fc-4fdc-aeaf-b43a6693e7c1
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: X-Amzn-Trace-Id: Root=1-639aa796-22bb39e3501014f40fd746c7;Sampled=0
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: -----------------------------------------------------
2022-12-15T15:50:30.772+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: [DEBUG] [aws-sdk-go] {"message":"The request is rejected because the JSON could not be
processed.","__type":"InvalidInputException"}
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: [DEBUG] [aws-sdk-go] DEBUG: Validate Response guardduty/CreateFilter failed, attempt 0/
25, error BadRequestException: The request is rejected because the JSON could not be processed.
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: {
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   RespMetadata: {
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:     StatusCode: 400,
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:     RequestID: "9b271113-a5fc-4fdc-aeaf-b43a6693e7c1"
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   },
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   Message_: "The request is rejected because the JSON could not be processed.",
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5:   Type: "InvalidInputException"
2022-12-15T15:50:30.775+1100 [DEBUG] provider.terraform-provider-aws_v4.46.0_x5: }
2022-12-15T15:50:30.775+1100 [ERROR] provider.terraform-provider-aws_v4.46.0_x5: Response contains error diagnostic: @caller=github.com/hashicorp/terraform-plugin-go@v0
.14.2/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= tf_rpc=ApplyResourceChange tf_resource_type=aws_guardduty_filter diagnostic_severit
y=ERROR diagnostic_summary="error creating GuardDuty Filter: BadRequestException: The request is rejected because the JSON could not be processed.
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "9b271113-a5fc-4fdc-aeaf-b43a6693e7c1"
  },
  Message_: "The request is rejected because the JSON could not be processed.",
  Type: "InvalidInputException"
}" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=b8d224b6-92a9-d36f-3545-e61523948ef6 timestamp=2022-12-15T15:50:30.775+1100
2022-12-15T15:50:30.780+1100 [ERROR] vertex "module.ap-southeast-1.aws_guardduty_filter.ssh_bruteforce" error: error creating GuardDuty Filter: BadRequestException: The
 request is rejected because the JSON could not be processed.
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "9b271113-a5fc-4fdc-aeaf-b43a6693e7c1"
  },
  Message_: "The request is rejected because the JSON could not be processed.",
  Type: "InvalidInputException"
}

Panic Output

No response

Important Factoids

This is within a module to allow me to loop through regions. All regions are failing. The detectors have created successfully prior to me adding the guardduty_filter section.

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 1 year ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

dnx-aiven commented 1 year ago

I've also tried running this with simplified criteria as I wondered if type of UnauthorizedAccess:EC2/SSHBruteForce caused some issues due to the characters in that string, but the same issue occurs.

dnx-aiven commented 1 year ago

bump still no movement on this?

dnx-aiven commented 1 year ago

I have just confirmed that this is still an issue with the latest provider version, 4.64 . I also tried using the type ID instead of the type name. Same issue.

│ Error: creating GuardDuty Filter: BadRequestException: The request is rejected because the JSON could not be processed.
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "7672ec78-525b-4765-9920-57455a700c28"
│   },
│   Message_: "The request is rejected because the JSON could not be processed.",
│   Type: "InvalidInputException"
│ }
│
│   with module.ap-northeast-3.aws_guardduty_filter.ssh_bruteforce,
│   on modules/guardduty/guardduty.tf line 42, in resource "aws_guardduty_filter" "ssh_bruteforce":
│   42: resource "aws_guardduty_filter" "ssh_bruteforce" {
finnigja commented 1 year ago

I've tried to reproduce this but failed, using the latest Terraform & AWS provider versions (1.4.6 and 4.64.0) as well as those you originally reported this with (1.3.6 and 4.46.0).

The setup & output from the basic standalone case I tested is below, but I was also able to exercise it further - modified the new rule it had created, added a second rule, removed the second rule, terraform destroy'd for cleanup. Also tried against a couple different regions.

The only errors I got where when - as expected - I tried setting bad configuration types / values and also tried creating a rule with the same name as an existing rule. I didn't see the "The request is rejected because the JSON could not be processed." error at any point in testing.

Not really sure where to go from here - only thought is perhaps it's related to your module setup somehow? Can I suggest trying this standalone config & seeing if that works for you as a baseline?

Terraform version:

$ terraform --version
Terraform v1.3.6
on darwin_arm64

Your version of Terraform is out of date! The latest version
is 1.4.6. You can update by downloading from https://www.terraform.io/downloads.html

Standalone reproduction configuration:

$ cat repro.tf
terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "4.46.0"
    }
  }
}

provider "aws" {
  region = "eu-west-3"
}

resource "aws_guardduty_detector" "primary" {
  enable = true

  datasources {
    s3_logs {
      enable = true
    }
    kubernetes {
      audit_logs {
        enable = false
      }
    }
    malware_protection {
      scan_ec2_instance_with_findings {
        ebs_volumes {
          enable = true
        }
      }
    }
  }
}

resource "aws_guardduty_filter" "ssh_bruteforce" {
  name        = "ssh-bruteforce-inbound"
  description = "Suppress inbound SSH bruteforce attempts"
  action      = "ARCHIVE"
  detector_id = aws_guardduty_detector.primary.id
  rank        = 1

  finding_criteria {
    criterion {
      field  = "type"
      equals = ["UnauthorizedAccess:EC2/SSHBruteForce"]
    }
    criterion {
      field  = "service.resourceRole"
      equals = ["TARGET"]
    }
 }
}

terraform init & AWS provider version:

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/aws versions matching "4.46.0"...
- Installing hashicorp/aws v4.46.0...
- Installed hashicorp/aws v4.46.0 (signed by HashiCorp)

[snip]
Terraform has been successfully initialized!
[snip]

terraform plan:

$ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_guardduty_detector.primary will be created
  + resource "aws_guardduty_detector" "primary" {
      + account_id                   = (known after apply)
      + arn                          = (known after apply)
      + enable                       = true
      + finding_publishing_frequency = (known after apply)
      + id                           = (known after apply)
      + tags_all                     = (known after apply)

      + datasources {
          + kubernetes {
              + audit_logs {
                  + enable = false
                }
            }

          + malware_protection {
              + scan_ec2_instance_with_findings {
                  + ebs_volumes {
                      + enable = true
                    }
                }
            }

          + s3_logs {
              + enable = true
            }
        }
    }

  # aws_guardduty_filter.ssh_bruteforce will be created
  + resource "aws_guardduty_filter" "ssh_bruteforce" {
      + action      = "ARCHIVE"
      + arn         = (known after apply)
      + description = "Suppress inbound SSH bruteforce attempts"
      + detector_id = (known after apply)
      + id          = (known after apply)
      + name        = "ssh-bruteforce-inbound"
      + rank        = 1
      + tags_all    = (known after apply)

      + finding_criteria {
          + criterion {
              + equals     = [
                  + "TARGET",
                ]
              + field      = "service.resourceRole"
              + not_equals = []
            }
          + criterion {
              + equals     = [
                  + "UnauthorizedAccess:EC2/SSHBruteForce",
                ]
              + field      = "type"
              + not_equals = []
            }
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

terraform apply:

$ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_guardduty_detector.primary will be created
  + resource "aws_guardduty_detector" "primary" {
      + account_id                   = (known after apply)
      + arn                          = (known after apply)
      + enable                       = true
      + finding_publishing_frequency = (known after apply)
      + id                           = (known after apply)
      + tags_all                     = (known after apply)

      + datasources {
          + kubernetes {
              + audit_logs {
                  + enable = false
                }
            }

          + malware_protection {
              + scan_ec2_instance_with_findings {
                  + ebs_volumes {
                      + enable = true
                    }
                }
            }

          + s3_logs {
              + enable = true
            }
        }
    }

  # aws_guardduty_filter.ssh_bruteforce will be created
  + resource "aws_guardduty_filter" "ssh_bruteforce" {
      + action      = "ARCHIVE"
      + arn         = (known after apply)
      + description = "Suppress inbound SSH bruteforce attempts"
      + detector_id = (known after apply)
      + id          = (known after apply)
      + name        = "ssh-bruteforce-inbound"
      + rank        = 1
      + tags_all    = (known after apply)

      + finding_criteria {
          + criterion {
              + equals     = [
                  + "TARGET",
                ]
              + field      = "service.resourceRole"
              + not_equals = []
            }
          + criterion {
              + equals     = [
                  + "UnauthorizedAccess:EC2/SSHBruteForce",
                ]
              + field      = "type"
              + not_equals = []
            }
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_guardduty_detector.primary: Creating...
aws_guardduty_detector.primary: Creation complete after 1s [id=c6c3e2300f8f03795c5d4eba72f8c5bc]
aws_guardduty_filter.ssh_bruteforce: Creating...
aws_guardduty_filter.ssh_bruteforce: Creation complete after 1s [id=c6c3e2300f8f03795c5d4eba72f8c5bc:ssh-bruteforce-inbound]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
dnx-aiven commented 1 year ago

Wow OK, I've just found where the problem was triggered, and in my Issue I had actually not copied the correct contents for some reason, d'oh!

In my description line, I actually had a comma. description = "Suppress inbound SSH bruteforce attempts, because we monitor via SSHd audit logs"

Removing this comma makes it work.

So, the issue triggers when there is a comma in description, which I believe it should successfully handle.

finnigja commented 1 year ago

Hate to say it, but this look like an AWS server-side issue - not a Terraform / terraform-provider-aws issue.

I can reproduce it by using the AWS CLI tool to attempt to update an existing filter to include a , in the description field:

$ aws guardduty update-filter --detector-id e4c3e4737dac77f4249a5eec3f67974a --filter-name ssh-bruteforce-inbound --description "test, test"

An error occurred (BadRequestException) when calling the UpdateFilter operation: The request is rejected because the JSON could not be processed.

Also reproducible in the web console:

Screenshot 2023-04-28 at 10 40 30 AM
justinretzolk commented 1 year ago

Hey y'all 👋 Thank you for taking the time to raise this, and for the helpful comments! Since this seems to be related to the underlying API, I've opened a ticket with AWS to have this looked into.

dnx-aiven commented 1 year ago

Thank you @justinretzolk , @chair6 . Apologies for missing the full detail in the original report.

justinretzolk commented 1 year ago

Hey @chair6 and @dnx-aiven 👋 I heard back from AWS on this, and it looks like this is actually expected behavior. They pointed me in the direction of this document which mentions:

  1. Enter a Name and Description for the suppression rule. Valid characters include alphanumeric characters, period (.), dash (-), underscore (_), and whitespaces.

They called out that the comma is not a supported special character, and that the API documentation will be updated to make this more clear.

dnx-aiven commented 1 year ago

Thanks @justinretzolk ! I'd completely missed this. I'll close this issue then. If they update the API docs that'll help.

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.