Policy Check Error: unable to unmarshal conftest output

pkudaiyar commented 6 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

Policy checks are failing after the successful 'RUN conftest test' when performing test against the terraform plan using opa policies. confest test outputs are generated in table format but failing the policy check post the test. error message : unable to unmarshal conftest output

Reproduction Steps

Logs

Environment details

If not already included, please provide the following:

Atlantis version: v0.27.1
Deployment method: ecs module
Atlantis flags:

Atlantis server-side config file:

repos:
  - id: /.*/  
    branch: /^(production|develop|release)$/

    # pre_workflow_hooks defines arbitrary list of scripts to execute before workflow execution.
    pre_workflow_hooks:        
        - run: echo "executing atlantis_config_gen" && /home/atlantis/atlantis_config_gen.sh        

    # allowed_overrides specifies which keys can be overridden by this repo in
    # its atlantis.yaml file.
    allowed_overrides: [workflow]

    # allowed_workflows specifies which workflows the repos that match 
    # are allowed to select.
    allowed_workflows: [dev, prod, showcase, test, uat, state_bucket, poc, qa]
    # apply_requirements sets the Apply Requirements for all repos that match.
    apply_requirements:
      - approved
      - mergeable

workflows:
  dev:
    plan:
      steps:    
      - run: if [ "${BASE_BRANCH_NAME}" != "develop" ]; then echo "The BASE branch ($BASE_BRANCH_NAME) is NOT allowed to deploy in DEV environment" && exit 1; fi
      - init
      - plan
      - run: terraform show -json $PLANFILE > $SHOWFILE
      - run: cp $SHOWFILE "${PROJECT_NAME}_${PULL_NUM}_$(git rev-parse HEAD).json"
      - run: aws s3 cp "${DIR}/${PROJECT_NAME}_${PULL_NUM}_$(git rev-parse HEAD).json" s3://${S3_STORING_PLANS}/plans/dev/planned/$(date +"%m_%d_%y")/ >/dev/null
    apply:
      steps:
      - apply
      - run: cp $SHOWFILE "${PROJECT_NAME}_${PULL_NUM}_$(git rev-parse HEAD).json"
      - run: aws s3 cp "${DIR}/${PROJECT_NAME}_${PULL_NUM}_$(git rev-parse HEAD).json" s3://${S3_STORING_PLANS}/plans/dev/applied/$(date +"%m_%d_%y")/ >/dev/null    
    policy_check:
      steps:
        - show
        - run: conftest pull git::https://bitbucket.org/<project>/<opa-policies>.git
        - run: conftest test --update git::https://bitbucket.org/<project>/<opa-policies>.git $(basename $SHOWFILE) -o table -p policy/common_policies/  --all-namespaces --no-fail

Repo atlantis.yaml file:

# config file

Any other information you can provide about the environment/deployment (efs/nfs, aws/gcp, k8s/fargate, etc) ECS Forgate, NFS mounted at /data --->

Additional Context

oliver-vini commented 6 months ago

I have the same issue which was somehow kinda mentioned here as well -> https://github.com/runatlantis/atlantis/issues/3682. Got it working by rolling back to v0.23.5 which is not an acceptable solution for me yet. @pkudaiyar, I noticed that you are using mergeable with bitbucket cloud VCS. Is that working for you? Have you tested it? It just doesn't work for me at all. I am also deploying with AWS ECS the same way as you. I have raised this question in the Slack community but haven't got any help on that subject so far. Are you adding any specific configuration to the repository for it to work? Do you mind sharing the flags you are parsing in your deployment?

kumaresh0 commented 6 months ago

@pkudaiyar @oliver-vini it's been raised here too https://github.com/runatlantis/atlantis/issues/4243

kumaresh0 commented 6 months ago

@pkudaiyar can you try with custom_policy_check: true on your server side config ?

runatlantis / atlantis