search

bridgecrewio / checkov

Prevent cloud misconfigurations and find vulnerabilities during build-time in infrastructure as code, container images and open source packages with Checkov by Bridgecrew.
https://www.checkov.io/
Apache License 2.0
6.83k stars 1.09k forks source link

Custom Policies in Python #5274

Closed rusherr02 closed 1 year ago

rusherr02 commented 1 year ago

I have created a custom policy in Python and I need to run it against a CFT.yaml which I am unable, please help how can I run it

Example Value Policy I have created:

from checkov.common.models.enums import CheckResult, CheckCategories
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck

class DemoCheck(BaseResourceCheck):

    def __init__(self, file):
        name = "Ensure any resource should not have abc"
        id = "CLUMIO_CFT_01"
        supported_resources = ['*']
        categories = [CheckCategories.GENERAL_SECURITY]
        self.file = file
        super().__init__(name=name, id=id, categories=categories, supported_resources=supported_resources)

    def scan_resource_conf(self, conf, file):
        if 'abc' not in conf.get[file]:
            return CheckResult.PASSED

        return CheckResult.FAILED

        # if conf.get("tags",[]):
        #     env = conf["tags"][0].get("Environment",{})
        #     if env in ["Developemnt","Staging","Production"]:
        #         return CheckResult.PASSED
        # return CheckResult.FAILED

scanner = DemoCheck()

yaml file

    somePolicy:
        Type: AWS::IAM::ManagedPolicy
        Properties:
            PolicyDocument:
                Version: 2012-10-17
                Statement:
                    - Sid: DescribeDynamoResources
                      Effect: Allow
                      Action:
                        - dynamodb:DescribeBackup
                        - dynamodb:DescribeContinuousBackups
                        - dynamodb:DescribeGlobalTable
                        - dynamodb:DescribeGlobalTableSettings
                        - dynamodb:DescribeTable
                        - dynamodb:DescribeTableReplicaAutoScaling
                        - dynamodb:ListBackups
                        - dynamodb:ListGlobalTables
                        - dynamodb:ListTables
                        - dynamodb:ListTagsOfResource
                      Resource: "abc"

basically, it's a very big yaml file I just want to create a custom policy if the resource contains 'abc' it should throw an error. I need help with how to create and run against the same. thanks

gruebel commented 1 year ago

hey @rusherr02 thanks for reaching out.

As I understand you are trying to create a custom policy for CloudFormation, then you accidentally imported the wrong base class. Should be like this

from checkov.common.models.enums import CheckResult, CheckCategories
from checkov.cloudformation.checks.resource.base_resource_check import BaseResourceCheck

class DemoCheck(BaseResourceCheck):

...
rusherr02 commented 1 year ago

hey @rusherr02 thanks for reaching out.

As I understand you are trying to create a custom policy for CloudFormation, then you accidentally imported the wrong base class. Should be like this

from checkov.common.models.enums import CheckResult, CheckCategories
from checkov.cloudformation.checks.resource.base_resource_check import BaseResourceCheck

class DemoCheck(BaseResourceCheck):

...

hey @gruebel after changing this to import what command should I run to test my custom policies?

gruebel commented 1 year ago

If you also added the __init__.py file as mentioned in the docs into the same folder as the custom policy, then you can reference it with your next checkov run.

ex.

checkov -d . --external-checks-dir [path to custom checks folder]
rusherr02 commented 1 year ago

I ran and got below error

3-07-01 00:42:58,899 [MainThread  ] [ERROR]  Cannot load external check 'temp' from checkov/DemoPolicy/temp.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/temp.py", line 1, in <module>
    from checkov.DemoPolicy import Demo
ModuleNotFoundError: No module named 'checkov.DemoPolicy'
2023-07-01 00:42:58,902 [MainThread  ] [ERROR]  Cannot load external check 'temp' from checkov/DemoPolicy/temp.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/temp.py", line 1, in <module>
    from checkov.DemoPolicy import Demo
ModuleNotFoundError: No module named 'checkov.DemoPolicy'
2023-07-01 00:42:58,906 [MainThread  ] [ERROR]  Cannot load external check 'temp' from checkov/DemoPolicy/temp.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/temp.py", line 1, in <module>
    from checkov.DemoPolicy import Demo
ModuleNotFoundError: No module named 'checkov.DemoPolicy'
2023-07-01 00:42:58,917 [MainThread  ] [ERROR]  Cannot load external check 'Demo' from checkov/DemoPolicy/Demo.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/Demo.py", line 5, in <module>
    class DemoCheck(BaseResourceCheck):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/multi_signature.py", line 64, in __new__
    raise NotImplementedError(f"The signature {multi_signature_key} for {name} is not supported.")
NotImplementedError: The signature (('self', 'conf', 'file'), None, None) for scan_resource_conf is not supported.
2023-07-01 00:42:58,923 [MainThread  ] [ERROR]  Cannot load external check 'Demo' from checkov/DemoPolicy/Demo.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/Demo.py", line 5, in <module>
    class DemoCheck(BaseResourceCheck):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/multi_signature.py", line 64, in __new__
    raise NotImplementedError(f"The signature {multi_signature_key} for {name} is not supported.")
NotImplementedError: The signature (('self', 'conf', 'file'), None, None) for scan_resource_conf is not supported.
2023-07-01 00:42:58,928 [MainThread  ] [ERROR]  Cannot load external check 'Demo' from checkov/DemoPolicy/Demo.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/Demo.py", line 5, in <module>
    class DemoCheck(BaseResourceCheck):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/multi_signature.py", line 64, in __new__
    raise NotImplementedError(f"The signature {multi_signature_key} for {name} is not supported.")
NotImplementedError: The signature (('self', 'conf', 'file'), None, None) for scan_resource_conf is not supported.
2023-07-01 00:42:58,928 [MainThread  ] [ERROR]  Cannot load external check 'temp' from checkov/DemoPolicy/temp.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/temp.py", line 1, in <module>
    from checkov.DemoPolicy import Demo
ModuleNotFoundError: No module named 'checkov.DemoPolicy'
2023-07-01 00:42:58,953 [MainThread  ] [ERROR]  Cannot load external check 'Demo' from checkov/DemoPolicy/Demo.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/Demo.py", line 5, in <module>
    class DemoCheck(BaseResourceCheck):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/multi_signature.py", line 64, in __new__
    raise NotImplementedError(f"The signature {multi_signature_key} for {name} is not supported.")
NotImplementedError: The signature (('self', 'conf', 'file'), None, None) for scan_resource_conf is not supported.
2023-07-01 00:42:58,957 [MainThread  ] [ERROR]  Cannot load external check 'temp' from checkov/DemoPolicy/temp.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/temp.py", line 1, in <module>
    from checkov.DemoPolicy import Demo
ModuleNotFoundError: No module named 'checkov.DemoPolicy'
2023-07-01 00:42:58,957 [MainThread  ] [ERROR]  Cannot load external check 'temp' from checkov/DemoPolicy/temp.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/manavmalhotra/PycharmProjects/checkov/checkov/DemoPolicy/temp.py", line 1, in <module>
    from checkov.DemoPolicy import Demo
ModuleNotFoundError: No module named 'checkov.DemoPolicy'
2023-07-01 00:42:58,986 [MainThread  ] [ERROR]  Cannot load external check 'temp' from checkov/DemoPolicy/temp.py
Traceback (most recent call last):
  File "/Users/manavmalhotra/.pyenv/versions/3.11.3/lib/python3.11/site-packages/checkov/common/checks/base_check_registry.py", line 207, in load_external_checks
    spec.loader.exec_module(module)  # type: ignore[union-attr] # loader can't be None here

can you please help what's wrong

gruebel commented 1 year ago

it seems like you put other files in the same folder as the custom policy. Please share the exact checkov CLI call you use and remove everything from the folder, where the custom policy is, except the __init__.py file and the custom policy file.

rusherr02 commented 1 year ago

image I want to run my custom against cft.yaml file which is present in the folder.

gruebel commented 1 year ago

as said, put only the custom check files into the folder. you can also create a subfolder in your DemoPolicy folder.

rusherr02 commented 1 year ago

hey @gruebel can we set up a call sometimes? let me know