Open nicholas-marchini opened 1 year ago
Updated to latest version 2.3.150 today and it's better, as in most consecutive runs are the same, 1 out of 5 are different.
hey @nicholas-marchini thanks for reaching out.
It looks like the inconsistency comes from the massive usage of the same module rds
not so surprising for me. Additionally using multiple tfvars files doesn't make it easier.
@gruebel Thanks for the reply. We operate a multi-tenent AWS account and do have the need to use multiple tfvars files right now.
Thanks for contributing to Checkov! We've automatically marked this issue as stale to keep our issues list tidy, because it has not had any activity for 6 months. It will be closed in 14 days if no further activity occurs. Commenting on this issue will remove the stale tag. If you want to talk through the issue or help us understand the priority and context, feel free to add a comment or join us in the Checkov slack channel at https://slack.bridgecrew.io Thanks!
This is still an issue. What can be done to resolve it?
hey @nmarchini
Do you have the same setup, multiple times using the same module? If yes, how many times is it?
@gruebel We call the module 9 times. I have used the latest docker image and am still getting inconsistent results
Any way we can progress this please?
Same issue here - different tests are running on my local machine (Mac) than in a container AWS CodeBuild is managing (more tests are checked in the container). However in my case the issue is happening even within just a single module (not referencing any other modules).
Is there way to define/enforce a list of tests that you want checkov to run? Something like checkov -d . --run-tests: 'CVE_AWS'
? How does checkov determine what tests to run when presented a file or directory?
I've given up on checkov as having this issue open since Apr 2, 2023 is poor, since Checkov got bought the level of response and interaction to issues posted here has dropped drastically. We moved to TFsec now and are very happy with it.
Thanks for contributing to Checkov! We've automatically marked this issue as stale to keep our issues list tidy, because it has not had any activity for 6 months. It will be closed in 14 days if no further activity occurs. Commenting on this issue will remove the stale tag. If you want to talk through the issue or help us understand the priority and context, feel free to add a comment or join us in the Checkov slack channel at codifiedsecurity.slack.com Thanks!
Describe the issue I have been running checkov locally (installed on MAC) and in the docker container but getting different results each time I can the exact same code.
The output below is for 4 executions of Checkov on the exact same code but with 3 different results. This run was just using custom checks only.
If I exclude the custom checks and don't use the config.yaml, so just a normal terraform run with CLI switches then I still get inconsistent results between runs.
Additional context This happens if I run the commands directly on the MAC and not within the docker container.