search

forseti-security / helm-charts

Apache License 2.0
8 stars 11 forks source link

Error applying manifest - unknown field: fsGroup #57

Closed pdutch closed 4 years ago

pdutch commented 4 years ago

When trying to apply the manifest generated by the helm chart, get the following error:

error: error validating "forseti-apply.yaml": error validating data: ValidationError(Deployment.spec.template.spec.initContainers[0].securityContext): unknown field "fsGroup" in io.k8s.api.core.v1.SecurityContext; if you choose to ignore these errors, turn validation off with --validate=false

kubectl version:

Client Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.11-dispatcher", GitCommit:"2e298c7e992f83f47af60cf4830b11c7370f6668", GitTreeState:"clean", BuildDate:"2019-09-19T22:26:40Z", GoVersion:"go1.11.13", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.11-gke.14", GitCommit:"56d89863d1033f9668ddd6e1c1aea81cd846ef88", GitTreeState:"clean", BuildDate:"2019-11-07T19:12:22Z", GoVersion:"go1.12.11b4", Compiler:"gc", Platform:"linux/amd64"}

Let me know if you need any further info.

Tasks

kevensen commented 4 years ago

@pdutch thanks for opening this issue. Can you please provide the command you are using to apply the manifest as well as the version of Helm?

pdutch commented 4 years ago 
helm template --set production=true --set-string server.config.contents="$(cat \
  forseti_conf_server.yaml | base64 -)" --values=values-new.yaml --namespace \
  forseti-9605 forseti-security-2.1.0.tgz > forseti-apply-21.yaml

Then kubectl apply -f forseti-apply-21.yaml which is what produces the error at the top when it gets to the deployments.

helm version: helm version Client: &version.Version{SemVer:"v2.15.1", GitCommit:"cf1de4f8ba70eded310918a8af3a96bfe8e7683b", GitTreeState:"clean"}

kevensen commented 4 years ago

@pdutch Thanks for the additional info. I will investigate this.

kevensen commented 4 years ago

@pdutch Interestingly I am unable to re-produce this problem on a Linux client. I wonder if there is some nuance with kubectl on macOS.

Do you mind upgrading your kubectl client to the latest version? I think that should be ~1.16.

pdutch commented 4 years ago

Yep, just tried with the brew installed version of kubectl (before was using the gcloud components version):

Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3", GitCommit:"b3cbbae08ec52a7fc73d334838e18d17e8512749", GitTreeState:"clean", BuildDate:"2019-11-14T04:24:34Z", GoVersion:"go1.12.13", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.11-gke.14", GitCommit:"56d89863d1033f9668ddd6e1c1aea81cd846ef88", GitTreeState:"clean", BuildDate:"2019-11-07T19:12:22Z", GoVersion:"go1.12.11b4", Compiler:"gc", Platform:"linux/amd64"}

Running the apply:

kubectl apply -f forseti-apply-21.yaml 
configmap/release-name-server-conf created
serviceaccount/config-validator created
rolebinding.rbac.authorization.k8s.io/pod-deleter created
role.rbac.authorization.k8s.io/pod-deleter created
serviceaccount/forseti-orchestrator created
serviceaccount/forseti-server created
service/release-name-config-validator created
service/release-name-database created
service/release-name-server created
error: error validating "forseti-apply-21.yaml": error validating data: ValidationError(Deployment.spec.template.spec.initContainers[0].securityContext): unknown field "fsGroup" in io.k8s.api.core.v1.SecurityContext; if you choose to ignore these errors, turn validation off with --validate=false
kevensen commented 4 years ago

@pdutch IIUC from Slack, you were able to successfully deploy. Is it ok to close this issue or does it require further discussion?

pdutch commented 4 years ago

Yes I can deploy, I just need to comment out the fsGroup: lines in the Yaml. just wondering if there are any ramifications of commenting these lines out?

kevensen commented 4 years ago

It may. IIRC, the gsutil container will pull down the file and give it ownership of 65535:65535. The fsGroup is meant to allow the container process to read the files with that ownership.

Would you be willing to share your rendered yaml file with any sensitive data removed? I'd be happy to take a look.

Also, would you be willing to try the kubectl binary from the Kubernetes site?

pdutch commented 4 years ago

Right so this could be potentially affect my other issue with config validator not being able to find any violations ? Ie, the config validator service can't read the files written by the init container?

pdutch commented 4 years ago

Rendered Yaml:

---
# Source: forseti-security/templates/server/config-maps/configmap-conf.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: ConfigMap
metadata:
  name: release-name-server-conf
  labels:
    chart: "forseti-security-2.1.0"
    component: forseti-server
data:
  forseti_conf_server.yaml: |-

    global:

        dummy_key: this_is_just_a_placeholder_see_issue_2486

    ##############################################################################

    inventory:

        # You must set ONLY one of root_resource_id or composite_root_resources in
        # your configuration. Defining both will cause Forseti to exit with an
        # error.

        # Root resource to start crawling from, formatted as
        # <resource_type>/<resource_id>, (e.g. "organizations/12345677890")
        # Note: For root resource id at a project level, you will need to use
        # project number instead of project id, e.g. "projects/<project number>".
        root_resource_id: "organizations/xxxx"

        # Resources to be excluded during the inventory process.
        # Only organizations/<ORG_NUMBER>, folders/<FOLDER_NUMBER>,
        # projects/<PROJECT_ID> or projects/<PROJECT_NUMBER> are accepted.
        # The child resources under the excluded resources will also be excluded.
        #
        # Example:
        # excluded_resources: ['folders/1234', 'projects/my-project-123', 'projects/4321']
        excluded_resources: []

        # Composite root resources: combines multiple resource roots into a single
        # inventory, for use across all Forseti modules. Can contain one or more
        # resources from the GCP Resource Hierarchy in any combination.
        # https://cloud.google.com/resource-manager/docs/cloud-platform-resource-hierarchy
        #
        # All resources must grant the appropriate IAM permissions to the Forseti
        # service account before they can be included in the inventory.
        #
        # Forseti Explain is not supported with a composite root at this time.
        #
        # Resources can exist in multiple organizations.
        #
        #composite_root_resources:
        #    - "folders/<folder number>"
        #    - "projects/<project number>"
        #    - "organizations/<organization number>"
        composite_root_resources: []

        # gsuite access
        domain_super_admin_email: xxx@xxx.xxx

        api_quota:
            # We are not using the max allowed API quota because we wanted to
            # include some rooms for retries.
            # Period is in seconds.

            # Set disable_polling to True to disable polling that API for creation
            # of the inventory. This can speed up inventory creation for
            # organizations that do not use specific APIs. Defaults to False if
            # not defined.
            admin:
              max_calls: 14
              period: 1.0
              disable_polling: True
            appengine:
              max_calls: 18
              period: 1.0
              disable_polling: True
            bigquery:
              max_calls: 160
              period: 1.0
              disable_polling: True
            cloudasset:
              max_calls: 1
              period: 1.0
              disable_polling: False
            cloudbilling:
              max_calls: 5
              period: 1.2
              disable_polling: True
            compute:
              max_calls: 18
              period: 1.0
              disable_polling: True
            container:
              max_calls: 9
              period: 1.0
              disable_polling: True
            crm:
              max_calls: 4
              period: 1.2
              disable_polling: True
            groupssettings:
              max_calls: 5
              period: 1.1
              disable_polling: True
            iam:
              max_calls: 90
              period: 1.0
              disable_polling: True
            logging:
              max_calls: 9
              period: 1.0
              disable_polling: True
            servicemanagement:
              max_calls: 2
              period: 1.1
              disable_polling: True
            serviceusage:
              max_calls: 4
              period: 1.1
              disable_polling: True
            sqladmin:
              max_calls: 1
              period: 1.1
              disable_polling: True
            storage:  # Does not use API quota
              disable_polling: True

        cai:
            # The FORSETI_CAI_BUCKET needs to be in Forseti project.
            enabled: True
            gcs_path: gs://xxx-cai-export-xxx

            # Timeout in seconds to wait for the exportAssets API to return success.
            # Defaults to 3600 if not set.
            api_timeout: 3600

            # Path to the CAI dump files. This is used when you have access to the
            # dump files directly and would like forseti to parse them into the
            # gcp_inventory table to use scanner / explainer on top of the data.
            # E.g. cai_dump_file_gcs_paths: ['gs://my-bucket/123-iam-policy.dump',
            #                                'gs://my-bucket/123-resource.dump']
            # cai_dump_file_gcs_paths: []

            # Optional list of asset types supported by Cloud Asset inventory API.
            # https://cloud.google.com/resource-manager/docs/cloud-asset-inventory/overview
            # If included, only the asset types listed will be included in the
            # Forseti inventory. This can be used to reduce the size of the
            # inventory database to save on storage and reduce the time to complete
            # a pull of the inventory.
            #
            # If commented out then all currently supported asset types are
            # exported from Cloud Asset API. The list of default asset types is
            # in google/cloud/forseti/services/inventory/base/cloudasset.py

            #asset_types:
            #    - appengine.googleapis.com/Application
            #    - appengine.googleapis.com/Service
            #    - appengine.googleapis.com/Version
            #    - bigquery.googleapis.com/Dataset
            #    - bigquery.googleapis.com/Table
            #    - bigtableadmin.googleapis.com/Cluster
            #    - bigtableadmin.googleapis.com/Instance
            #    - bigtableadmin.googleapis.com/Table
            #    - cloudbilling.googleapis.com/BillingAccount
            #    - cloudkms.googleapis.com/CryptoKey
            #    - cloudkms.googleapis.com/CryptoKeyVersion
            #    - cloudkms.googleapis.com/KeyRing
            #    - cloudresourcemanager.googleapis.com/Folder
            #    - cloudresourcemanager.googleapis.com/Organization
            #    - cloudresourcemanager.googleapis.com/Project
            #    - compute.googleapis.com/Address
            #    - compute.googleapis.com/Autoscaler
            #    - compute.googleapis.com/BackendBucket
            #    - compute.googleapis.com/BackendService
            #    - compute.googleapis.com/Disk
            #    - compute.googleapis.com/Firewall
            #    - compute.googleapis.com/ForwardingRule
            #    - compute.googleapis.com/GlobalAddress
            #    - compute.googleapis.com/GlobalForwardingRule
            #    - compute.googleapis.com/HealthCheck
            #    - compute.googleapis.com/HttpHealthCheck
            #    - compute.googleapis.com/HttpsHealthCheck
            #    - compute.googleapis.com/Image
            #    - compute.googleapis.com/Instance
            #    - compute.googleapis.com/InstanceGroup
            #    - compute.googleapis.com/InstanceGroupManager
            #    - compute.googleapis.com/InstanceTemplate
            #    - compute.googleapis.com/Interconnect
            #    - compute.googleapis.com/InterconnectAttachment
            #    - compute.googleapis.com/License
            #    - compute.googleapis.com/Network
            #    - compute.googleapis.com/Project
            #    - compute.googleapis.com/RegionBackendService
            #    - compute.googleapis.com/Route
            #    - compute.googleapis.com/Router
            #    - compute.googleapis.com/SecurityPolicy
            #    - compute.googleapis.com/Snapshot
            #    - compute.googleapis.com/SslCertificate
            #    - compute.googleapis.com/Subnetwork
            #    - compute.googleapis.com/TargetHttpProxy
            #    - compute.googleapis.com/TargetHttpsProxy
            #    - compute.googleapis.com/TargetInstance
            #    - compute.googleapis.com/TargetPool
            #    - compute.googleapis.com/TargetSslProxy
            #    - compute.googleapis.com/TargetTcpProxy
            #    - compute.googleapis.com/TargetVpnGateway
            #    - compute.googleapis.com/UrlMap
            #    - compute.googleapis.com/VpnTunnel
            #    - container.googleapis.com/Cluster
            #    - dataproc.googleapis.com/Cluster
            #    - dataproc.googleapis.com/Job
            #    - dns.googleapis.com/ManagedZone
            #    - dns.googleapis.com/Policy
            #    - iam.googleapis.com/Role
            #    - iam.googleapis.com/ServiceAccount
            #    - k8s.io/Namespace
            #    - k8s.io/Node
            #    - k8s.io/Pod
            #    - pubsub.googleapis.com/Subscription
            #    - pubsub.googleapis.com/Topic
            #    - rbac.authorization.k8s.io/ClusterRole
            #    - rbac.authorization.k8s.io/ClusterRoleBinding
            #    - rbac.authorization.k8s.io/Role
            #    - rbac.authorization.k8s.io/RoleBinding
            #    - spanner.googleapis.com/Database
            #    - spanner.googleapis.com/Instance
            #    - sqladmin.googleapis.com/Instance
            #    - storage.googleapis.com/Bucket

        # Number of days to retain inventory data:
        #  -1 : (default) keep all previous data forever
        #   0 : delete all previous inventory data before running
        retention_days: -1

    ##############################################################################

    scanner:

        # Output path (do not include filename).
        # If GCS location, the format of the path should be:
        # gs://bucket-name/path/for/output
        output_path: gs://forseti-bucket/scanner_violations

        # Rules path (do not include filename).
        # If GCS location, the format of the path should be:
        # gs://bucket-name/path/for/rules_path
        # if no rules_path is specified, rules are
        # searched in /path/to/forseti_security/rules/
        rules_path: gs://forseti-bucket/rules

        # Enable the scanners as default to true when integrated for Forseti 2.0.

        scanners:
            - name: audit_logging
              enabled: false
            - name: bigquery
              enabled: false
            - name: blacklist
              enabled: false
            - name: bucket_acl
              enabled: false
            - name: cloudsql_acl
              enabled: false
            - name: config_validator
              enabled: true
            - name: enabled_apis
              enabled: false
            - name: firewall_rule
              enabled: false
            - name: forwarding_rule
              enabled: false
            - name: group
              enabled: false
            - name: groups_settings
              enabled: false
            - name: iam_policy
              enabled: false
            - name: iap
              enabled: false
            - name: instance_network_interface
              enabled: false
            - name: ke_scanner
              enabled: false
            - name: ke_version_scanner
              enabled: false
            - name: kms_scanner
              enabled: false
            - name: lien
              enabled: false
            - name: location
              enabled: false
            - name: log_sink
              enabled: false
            - name: resource
              enabled: false
            - name: service_account_key
              enabled: false

    ##############################################################################

    notifier:
        api_quota:
            securitycenter:
                max_calls: 14
                period: 1.0

        # Provide connector details
        # email_connector:
        #   name: sendgrid
        #   auth:
        #     api_key: {SENDGRID_API_KEY}
        #   sender: {EMAIL_SENDER}
        #   recipient: {EMAIL_RECIPIENT}
        #   data_format: csv

        # For every resource type you can set up a notification pipeline
        # to send alerts for every violation found
        resources:
            - resource: iam_policy_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations
                # Slack webhook pipeline.
                # Create an incoming webhook in your organization's Slack setting, located at:
                # https://[your_org].slack.com/apps/manage/custom-integrations
                # Add the provided URL in the configuration below in `webhook_url`.
                # - name: slack_webhook
                #   configuration:
                #     data_format: json  # slack only supports json
                #     webhook_url: ''

            - resource: audit_logging_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: blacklist_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: bigquery_acl_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: buckets_acl_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: cloudsql_acl_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: config_validator_violations
              should_notify: true
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: enabled_apis_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: firewall_rule_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: forwarding_rule_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: groups_settings_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: ke_version_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: ke_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: kms_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: groups_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: instance_network_interface_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: iap_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: lien_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: location_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: log_sink_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: resource_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: service_account_key_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

            - resource: external_project_access_violations
              should_notify: false
              notifiers:
                # Email violations
                # - name: email_violations
                # Upload violations to GCS.
                - name: gcs_violations
                  configuration:
                    data_format: csv
                    # gcs_path should begin with "gs://"
                    gcs_path: gs://forseti-bucket/scanner_violations

        violation:
          cscc:
            enabled: false
            # Cloud SCC uses a source_id. It is unique per
            # organization and must be generated via a self-registration process.
            # The format is: organizations/ORG_ID/sources/SOURCE_ID
            source_id:

        inventory:
          gcs_summary:
            enabled: true
            # data_format may be one of: csv (the default) or json
            data_format: csv
            # gcs_path should begin with "gs://"
            gcs_path: gs://forseti-bucket/inventory_summary
          email_summary:
            enabled: false

---
# Source: forseti-security/charts/configValidator/templates/service-accounts/config-validator.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: ServiceAccount
metadata:
  name: config-validator
  namespace: forseti
  annotations:
    iam.gke.io/gcp-service-account: forseti-client-gcp@forseti-project.iam.gserviceaccount.com
  labels:
    component: config-validator
    chart: configValidator-0.1.1
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-deleter
  namespace: forseti
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: pod-deleter
subjects:
  - kind: ServiceAccount
    name: config-validator
    namespace: forseti
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-deleter
  namespace: forseti
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["delete"]

---
# Source: forseti-security/templates/orchestrator/service-accounts.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: forseti-orchestrator
  namespace: forseti
  annotations:
    iam.gke.io/gcp-service-account: forseti-client-gcp@forseti-project.iam.gserviceaccount.com
  labels:
    component: forseti-orchestrator
    chart: forseti-security-2.1.0
---
# Source: forseti-security/templates/server/service-accounts.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: ServiceAccount
metadata:
  name: forseti-server
  namespace: forseti
  annotations:
    iam.gke.io/gcp-service-account: forseti-project-sa@forseti-project.iam.gserviceaccount.com
  labels:
    component: forseti-server
    chart: forseti-security-2.1.0

---
# Source: forseti-security/charts/configValidator/templates/service.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Service
metadata:
  name: release-name-config-validator
  labels:
    component: config-validator
    chart: configValidator-0.1.1
spec:
  type: ClusterIP
  ports:
  - port: 50052
    targetPort: 50052
    protocol: TCP
  selector:
    component: config-validator

---
# Source: forseti-security/templates/database/service.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: Service
metadata:
  name: release-name-database
  labels:
    component: forseti-database
    chart: forseti-security-2.1.0
spec:
  ports:
  - port: 3306
    targetPort: 3306
    protocol: TCP
  selector:
    component: forseti-database

---
# Source: forseti-security/templates/server/service.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Service
metadata:
  name: release-name-server
  labels:
    component: forseti-server
    chart: forseti-security-2.1.0
spec:
  type: ClusterIP
  ports:
  - port: 50051
    targetPort: 50051
    protocol: TCP
  selector:
    component: forseti-server

---
# Source: forseti-security/charts/configValidator/templates/deployment.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-name-config-validator
spec:
  selector:
    matchLabels:
      component: config-validator
  replicas: 1
  template:
    metadata:
      labels:
        component: config-validator
        chart: configValidator-0.1.1
    spec:
      serviceAccountName: config-validator      
      initContainers:
      - name: gsutil-init
        image: gcr.io/cloud-builders/gsutil
        args: ['cp', '-r', 'gs://forseti-bucket/policy-library/', '/policy-library/']
        env:
        - name: HOME
          value: /policy-library
        volumeMounts:
        - name: policy-library
          mountPath: /policy-library/
        securityContext:
          runAsUser: 65533
          runAsGroup: 65533
          # fsGroup: 65533
      containers:
      - name: config-validator
        image: "gcr.io/forseti-containers/config-validator:572e207"
        args: ["-policyPath", "/policy-library/policies", "-policyLibraryPath", "/policy-library/lib", "-port", "50052", "-alsologtostderr"]
        ports:
        - containerPort: 50052
        env:
        - name: GOGC
          value: "1000"
        volumeMounts:
        - name: policy-library
          mountPath: /policy-library/policies
          subPath: policy-library/policies
          readOnly: true
        - name: policy-library
          mountPath: /policy-library/lib
          subPath: policy-library/lib
          readOnly: true
      securityContext:
        runAsUser: 65532
        runAsGroup: 65532
        # fsGroup: 65533
      volumes:
      - name: policy-library
        emptyDir: {
            medium: "Memory"
        }

---
# Source: forseti-security/templates/database/deployment.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-name-database
spec:
  selector:
    matchLabels:
      component: forseti-database
  replicas: 1
  template:
    metadata:
      labels:
        component: forseti-database
        chart: forseti-security-2.1.0
    spec:
      serviceAccountName: forseti-server
      nodeSelector:
      containers:      
      - name: cloudsql-proxy
        image: gcr.io/cloudsql-docker/gce-proxy:latest
        command: ["/cloud_sql_proxy"]
        args:
        - -instances=forseti-project:xxx-xxx:forseti-project-sql=tcp:0.0.0.0:3306
        securityContext:
          runAsUser: 2
          allowPrivilegeEscalation: false

---
# Source: forseti-security/templates/server/deployment.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-name-server
spec:
  selector:
    matchLabels:
      component: forseti-server
  replicas: 1
  template:
    metadata:
      labels:
        component: forseti-server
        configValidator: client
        database: client
        chart: forseti-security-2.1.0
    spec:
      serviceAccountName: forseti-server      
      containers:
      - name: forseti-server
        image: "gcr.io/forseti-containers/forseti:v2.24.0"
        imagePullPolicy: Always
        ports:
        - containerPort: 50051
        args:
        - --run_server
        - --services
        - 'explain inventory model scanner server notifier'
        - --sql_host
        - release-name-database.forseti.svc.cluster.local
        - --sql_database_name
        -  forseti_security
        - --log_level
        -  info
        - --server_host
        - 0.0.0.0
        volumeMounts:
        - name: forseti-server-config-volume
          mountPath: /forseti-security
          readOnly: true
        env:
        - name: FORSETI_HOME
          value: /forseti-security
        - name: CONFIG_VALIDATOR_ENDPOINT
          value: release-name-config-validator.forseti.svc.cluster.local:50052
        livenessProbe:
          exec:
            command:
            - forseti
            - inventory
            - list
          initialDelaySeconds: 30
          periodSeconds: 30
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 3

      securityContext:
        runAsUser: 1000
        # fsGroup: 65532
      volumes:
      - name: forseti-server-config-volume
        configMap:
          name: release-name-server-conf

---
# Source: forseti-security/templates/orchestrator/forseti-cron-job.yaml
# Copyright 2019 The Forseti Security Authors. All rights reserved.
#
# Licensed under the Apache License, Versisn 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: release-name-orchestrator
  labels:
    chart: "forseti-security-2.1.0"
    component: forseti-orchestrator
spec:
  schedule: "*/60 * * * *"
  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            component: forseti-orchestrator
            chart: forseti-security-2.1.0
        spec:
          serviceAccountName: forseti-orchestrator
          containers:
          - name: release-name-orchestrator
            image: "gcr.io/forseti-containers/forseti:v2.24.0"
            imagePullPolicy: Always
            args:
            - --run_client
            - --server_host
            - release-name-server.forseti.svc.cluster.local
            env:
            - name: FORSETI_HOME
              value: /forseti-security/
            securityContext:
              runAsUser: 1000
          restartPolicy: Never
          schedulerName: default-scheduler
      completions: 1
  concurrencyPolicy: Forbid

---
pdutch commented 4 years ago

I'm just trying to comment out only the fsGroup: line in the init-container spec section, as it looks like setting fsGroup is only allowed at the Pod level, not within the container level. will let you know.

kevensen commented 4 years ago

@pdutch I think you are on to something.

  1. According to the Kubernetes API, fsGroup is a valid field in the pod's security context
  2. fsGroup is NOT a valid field in the container's security context.

When I apply the Helm charts, either directly (with or without Tiller) or via Terraform, I receive no validation errors. I do notice that the fsGroup field in the pod's security context has been dropped. My host is a Linux host. This tells me that Helm and/or Tiller are silently tolerating the invalid field.

Long story short, we (I) need to remove that field from the containers' security contexts. I appreciate your patience and willingness to investigate this issue with us.