search

GoogleCloudPlatform / config-validator

Golang library which provides functionality to evaluate GCP resources against Rego-based policies
Apache License 2.0
45 stars 34 forks source link

Latest Docker image for config-validator with latest version of forseti==> StatusCode.INTERNAL: "Not supported" #159

Open krab-skunk opened 3 years ago

krab-skunk commented 3 years ago

Continuing on my bugs reports ;)

Runing latest forseti version 2.25.2

took the latest docker images for config validator from here (tag b3da694) https://console.cloud.google.com/gcr/images/forseti-containers/GLOBAL/config-validator?gcrImageListsize=30

Run docker as follow (no doc anywhere, so i assume we should run it this way)

    docker run --rm \
        -v /home/ubuntu/policy-library/policy-library/policies:/tmp/policies \
        -v /home/ubuntu/policy-library/policy-library/lib:/tmp/lib \
        -p50052:50052 \
        gcr.io/forseti-containers/config-validator \
        -policyPath=/tmp/policies \
        -policyLibraryPath=/tmp/lib \
        -port=50052

telnet works great on 50052, so i run the scanner

forseti scanner run

Error in foresti.log:

2020-12-17 10:35:55,795 INFO [forseti-security][2.25.2] google.cloud.forseti.services.scanner.service(Run): Run scanner service with model: 30e6001f56ebf2097e0fa70c01258d6e
2020-12-17 10:35:55,805 INFO [forseti-security][2.25.2] google.cloud.forseti.scanner.scanner_builder(_instantiate_scanner): {'module_name': 'config_validator_scanner', 'class_name': 'ConfigValidatorScanner', 'rules_filename': ''}
2020-12-17 10:35:55,805 INFO [forseti-security][2.25.2] google.cloud.forseti.scanner.scanner_builder(_instantiate_scanner): Initializing the rules engine:
Using rules:
2020-12-17 10:35:55,807 ERROR [forseti-security][2.25.2] google.cloud.forseti.scanner.scanners.config_validator_util.validator_client(reset): ConfigValidatorResetError: <_Rendezvous of RPC that terminated with:
    status = StatusCode.INTERNAL
    details = "Not supported"
    debug_error_string = "{"created":"@1608201355.807504094","description":"Error received from peer ipv6:[::1]:50052","file":"src/core/lib/surface/call.cc","file_line":1052,"grpc_message":"Not supported","grpc_status":13}"
>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.25.2-py3.6.egg/google/cloud/forseti/scanner/scanners/config_validator_util/validator_client.py", line 196, in reset
    self.stub.Reset(validator_pb2.ResetRequest())
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 565, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
    status = StatusCode.INTERNAL
    details = "Not supported"
    debug_error_string = "{"created":"@1608201355.807504094","description":"Error received from peer ipv6:[::1]:50052","file":"src/core/lib/surface/call.cc","file_line":1052,"grpc_message":"Not supported","grpc_status":13}"
>
2020-12-17 10:35:55,808 ERROR [forseti-security][2.25.2] google.cloud.forseti.scanner.scanner(run): Error running scanner: ConfigValidatorScanner: 'Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.25.2-py3.6.egg/google/cloud/forseti/scanner/scanners/config_validator_util/validator_client.py", line 196, in reset
    self.stub.Reset(validator_pb2.ResetRequest())
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 565, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
    status = StatusCode.INTERNAL
    details = "Not supported"
    debug_error_string = "{"created":"@1608201355.807504094","description":"Error received from peer ipv6:[::1]:50052","file":"src/core/lib/surface/call.cc","file_line":1052,"grpc_message":"Not supported","grpc_status":13}"
>

During handling of the above exception, anothe
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.25.2-py3.6.egg/google/cloud/forseti/scanner/scanners/config_validator_util/validator_client.py", line 196, in reset
    self.stub.Reset(validator_pb2.ResetRequest())
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 565, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
    status = StatusCode.INTERNAL
    details = "Not supported"
    debug_error_string = "{"created":"@1608201355.807504094","description":"Error received from peer ipv6:[::1]:50052","file":"src/core/lib/surface/call.cc","file_line":1052,"grpc_message":"Not supported","grpc_status":13}"
>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.25.2-py3.6.egg/google/cloud/forseti/scanner/scanner.py", line 119, in run
    scanner.run()
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.25.2-py3.6.egg/google/cloud/forseti/scanner/scanners/config_validator_scanner.py", line 203, in run
    for flattened_violations in self._retrieve_flattened_violations():
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.25.2-py3.6.egg/google/cloud/forseti/scanner/scanners/config_validator_scanner.py", line 178, in _retrieve_flattened_violations
    self.validator_client.reset()
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise
    raise value
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.25.2-py3.6.egg/google/cloud/forseti/scanner/scanners/config_validator_util/validator_client.py", line 203, in reset
    raise errors.ConfigValidatorResetError(e)
google.cloud.forseti.scanner.scanners.config_validator_util.errors.ConfigValidatorResetError: <_Rendezvous of RPC that terminated with:
    status = StatusCode.INTERNAL
    details = "Not supported"
    debug_error_string = "{"created":"@1608201355.807504094","description":"Error received from peer ipv6:[::1]:50052","file":"src/core/lib/surface/call.cc","file_line":1052,"grpc_message":"Not supported","grpc_status":13}"
>
2020-12-17 10:35:55,819 INFO [forseti-security][2.25.2] google.cloud.forseti.scanner.scanner(run): Scan completed!

Thanks

krab-skunk commented 3 years ago

Worth mentioning , in the constraints, i try both naming conventions ["organizations/**"] or ["organizations/XXXXXXXX"]

gkowalski-google commented 3 years ago

@krab-skunk Forseti v2.25.2 and earlier versions use a static binary of Config Validator that is included in the repo. On the current master branch, this has changed to use the docker image. Not all versions of CV are compatible with Forseti. If you want to use CV with Docker, then you'll need to use the master branch of Forseti and the Forseti Terraform module. There is a default version of CV (image tag) used by the module, but it should be safe to try the latest. I don't think there have been many updates to CV that would benefit Forseti though.

Example Terraform config:

module "Forseti" {
  source          = "git::github.com/forseti-security/terraform-google-forseti"
  forseti_version = "master"

  org_id     = "1234567890"
  domain     = "example.com"
  project_id = "my-project"
  ...
}
krab-skunk commented 3 years ago

@gkowalski-google dzięki for your answer. The thing is that i was desperately trying the master branch, as the one coming with terraform never worked for me and as per issue 156 (https://github.com/forseti-security/config-validator/issues/156 ), i'm not the only one :(

I did all my install of forseti using terraform module provided in the wiki, but none of them ever worked with CV, most likely CV has an issue :/

ps: i'd be more than happy to share my screen via zoom if required ;P

gkowalski-google commented 3 years ago

Can you try starting with a single constraint using master branch and see if that still produces the error?

krab-skunk commented 3 years ago

i do have only one constraint actually that i copied from the samples directory, and its this one

apiVersion: constraints.gatekeeper.sh/v1alpha1
kind: GCPRestrictedFirewallRulesConstraintV1
metadata:
  name: restrict-firewall-rule-world-open
  annotations:
    bundles.validator.forsetisecurity.org/forseti-security: v2.26.0
    bundles.validator.forsetisecurity.org/scorecard-v1: security
    description: Checks for open firewall rules allowing ingress from the internet.
spec:
  severity: high
  match:
    target: # {"$ref":"#/definitions/io.k8s.cli.setters.target"}
    - "organizations/xxxxxxxxxxxx" #where x is my org ID
  parameters:
    rules:
    - direction: "INGRESS"
      source_ranges:
      - "0.0.0.0/0"
      enabled: "true"
      rule_type: "allowed"
gkowalski-google commented 3 years ago

Perhaps something is wrong with the policy library being used? I just redeployed the master branch of Forseti, and included the restricted firewall world open constraint. No errors encountered. I am using the git-sync feature to sync the policy library from a private GitHub repo. It is finding violations as well.

krab-skunk commented 3 years ago

@gkowalski-google zajebisty!! All works on master branch :) Config validator on docker also find me perfectly all the issues. Thanks a ton for your help :) The trick was to to use indeed forseti on master branch :)

nkaravias commented 3 years ago

I'm trying to run this on GKE right now. Is there a matrix with the docker image sha to get a working forseti & config validator on Kubernetes?