--external-checks-git generates error, custom policies not checked

[benglerHIG]

Describe the bug The --external-checks-git argument generates the following error and custom policies are not checked: "Error in atexit._run_exitfuncs: Traceback (most recent call last): File "C:\Users\BE53599\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 616, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 5] Access is denied: 'c:\Users\BE53599\Documents\GitHub\cto_cloud_refarch\templates\sns\v1.0\3df0b144_checks\clone\.git\objects\pack\pack-b4638997d5a4a734af44106d70eb92321d408879.idx'"

To Reproduce Steps to reproduce the behavior:

1. Go to directory with a file to scan
2. Run cli command 'checkov -f template.yml --external-checks-git https://github.test.com/test/cto_cloud_iac_scanning.git'
3. See error

Expected behavior Using the --external-checks-git argument should scan the code without error and scan the code using the custom policies defined in the git repo

Screenshots

Desktop (please complete the following information):

• OS: Windows 10
• Checkov Version 2.0.495

Additional context Add any other context about the problem here (e.g. code snippets).

[kartikp10]

Hey @benglerHIG, the option does seem to work when I test it out on a Macbook or using docker. Based on the error in the screenshot, I can see that the issue is related to permissions for deleting some temp files using python. I suspect that if you run this in an "admin" Powershell window, it should be good.

Just FYI, for scanning using checks in an external git repo, Checkov will attempt to clone the repo with external checks and create a temporary folder in the current directory. Once the scanning is complete, the cloned repo will be deleted.

I'm not very sure why the custom checks are not applied to your terraform. Since I cannot see the contents of https://github.test.com/test/cto_cloud_iac_scanning.git I cannot say if the policies are valid. I suggest that you try cloning that repo locally and use the --external-checks-dir flag and testing it out again.

I tested using this repo, which has a couple policies: checkov -f main.tf --external-checks-git https://github.com/kartikp10/bridgecrew-custom-policies

[benglerHIG]

Thank you for the info and the pointer to the custom policy test repo. They were very helpful.

I ran several more tests using the most recent version of checkov. I believe you're absolutely correct about the error message I'm seeing in Windows being caused by the attempt to delete the directory created when the repo is cloned into the directory. After running I see directories such as "7fbe86b4_checks" left behind.

Also, I can run the same --external-checks-git command on my linux server without receiving the same error message. I can confirm running Powershell or Command Prompt in Admin mode did not resolve the issue, still getting the same error and the directory is left behind.

I ran several tests using --external-checks-dir in both Windows and Linux and did not receive the same error.

It would be great to resolve the issue with deleting the directory to get rid of the error message and cleanup after cloning the repo.

However, what I believe to be a much larger issue, Checkov isn't using the custom policies. Both git and dir options noted above. When I run checkov with either option, I get the exact same results as when I run without using any custom policies. To eliminate the possibility the problem is being caused by the custom policy I'm using, I pulled this policy into a file locally to test with: https://github.com/kartikp10/bridgecrew-custom-policies/blob/main/policies/aws/BC_AWS_C_001.yaml .

How can I see the results of custom policy scans?

[benglerHIG]

@kartikp10 should I submit a separate Issue for the main issue here - not using the custom policies? I just retested with latest version v2.0.528 and confirmed both --external-checks-dir and --external-checks-git are not using the custom policies

[nimrodkor]

Related to #1778

[stale[bot]]

Thanks for contributing to Checkov! We've automatically marked this issue as stale to keep our issues list tidy, because it has not had any activity for 6 months. It will be closed in 14 days if no further activity occurs. Commenting on this issue will remove the stale tag. If you want to talk through the issue or help us understand the priority and context, feel free to add a comment or join us in the Checkov slack channel at https://slack.bridgecrew.io Thanks!

[stale[bot]]

Closing issue due to inactivity. If you feel this is in error, please re-open, or reach out to the community via slack: https://slack.bridgecrew.io Thanks!