Implement more secrets checks

[Fennerr]

New feature motivation

Similar to the secrets checks for the other services (lambda/ec2/ecs/etc), more checks can be implemented

Solution Proposed

Elastic Beanstalk:

• Configuration files (.ebextensions) used for environment customization might contain secrets.
• aws elasticbeanstalk describe-environment-resources --environment-name [Environment_Name]

API Gateway:

• Integration request parameters or mapping templates can potentially include secrets.
• aws apigateway get-integration --rest-api-id [RestApi_Id] --resource-id [Resource_Id] --http-method [HTTP_Method]

CodeBuild and CodePipeline:

• Buildspec files or pipeline definitions could contain embedded secrets, especially in environment variable definitions.
• aws codebuild batch-get-projects --names [Project_Name]
• aws codepipeline get-pipeline --name [Pipeline_Name]

Glue Jobs:

• ETL scripts used in AWS Glue can include hard-coded credentials for data sources or sinks.
• Check the script located at ScriptLocation and arguements passed using DefaultArguments
• aws glue get-job --job-name [Job_Name]
• aws glue get-connection --name [Connection_Name]

Step Functions:

• State machine definitions might include sensitive information in task parameters.
• aws stepfunctions describe-state-machine --state-machine-arn [State_Machine_Arn]

AppSync:

• Resolver mapping templates or data source configurations can include hard-coded API keys or database credentials.
• aws appsync get-graphql-api --api-id [API_Id]

This might not be all the API calls that need to be made to get the secrets, but should be a good starting point

Describe alternatives you've considered

None

Additional context

No response

[Fennerr]

More secrets to consider:

AWS CodeCommit:

• Check for secrets in AWS CodeCommit repositories (including branches and commit history - a tool designed for git would be useful like trufflehog)

AWS Elastic Container Registry (ECR):

• Images stored in ECR might contain secrets. Scanning Dockerfiles and image layers for secrets can be crucial. (Trivy could be good here)

AWS Lambda Layer Contents:

• Lambda Layers are used to manage code and dependencies. Scanning the contents of these layers for secrets is as important as scanning the Lambda function code.

AWS Systems Manager State Manager:

• State Manager documents might include scripts or commands that contain secrets.

AWS Batch Job Definitions:

• Similar to ECS tasks, AWS Batch job definitions might include environment variables or command parameters that contain secrets.

Amazon SageMaker:

• Notebooks and model training scripts in SageMaker can sometimes include embedded secrets.

AWS Amplify Console:

• Check for secrets in Amplify app build settings and environment variables. (should be in environment secrets)

AWS Data Pipeline:

• Data Pipeline definitions, especially the ones that contain custom scripts or SQL commands, could have embedded secrets.

AWS Glue Data Catalog:

• Scanning AWS Glue Data Catalog for database connection details that might include hardcoded credentials.

[n4ch04]

Hi @Fennerr, all of them interesting ideas, but maybe we need to rethink a little bit our "secrets detection engine" ... The detect-secrets package generates false positives and the current way of scanning generates a lot of resource exhaustion ...

[Fennerr]

This is true. Maybe move it to an optional flag and/or a config option to select what secrets you want to scan for

It would also be nice to generalize the way that secrets are scanned for so that stuff like writing to the temp files to disk, search for secrets, and using multiprocessing for this (as it's cpu intensive) can just be handled in one place.

[Gfuen]

@Fennerr Was there any decision on this ticket going forward? Because if the detect-secrets package is going to be changed I would love to work on that. Thanks!

[jfagoagas]

Hello @Gfuen there are no plans on changing the detect-secrets package. Do you have any suggestion for a different Python package to look for secrets?

Related to the checks requested above they are still pending to be done.