Remove limitiation of mounitng SPIRL to GG/GE

PetrMc commented 1 month ago

Gloo Edge Product

Enterprise

Gloo Edge Version

1.17.2

Is your feature request related to a problem? Please describe.

When following the SPIRL Docs and Istio Integration Docs:

The integration functions correctly on its own.
The SPIRL configuration is successfully applied to applications running within Istio containers.
However, the SPIRL configuration (specifically volumes and volume mounts) is not applied to the sidecar running within the Gloo Gateway pod.

Attachments:

The following files are attached for reference:

app-pod.yaml: YAML configuration of the application pod where SPIRL configuration is applied successfully.
gw-pod.yaml: YAML configuration of the Gloo Gateway pod where the SPIRL configuration (volumes and volume mounts) is not applied to the sidecar.
app-config-dump.json: Configuration dump from the application pod, showing the applied SPIRL settings.

app-config-dump.json.txt app-pod.yaml.txt gw-pod.yaml.txt

Describe the solution you'd like

Discussed the issue with @npolshakova, and it appears to be addressable by adding the necessary volumes and volume mounts to the Helm charts. Additionally, a flag will be introduced to trigger gloo-proxy-http to be deployed with the mounts required by the SPIRL integration.

Describe alternatives you've considered

No response

Additional Context

No response

npolshakova commented 1 month ago

Based on the SPIRL config, there's a couple changes that will need to be made:

Add the workload-socket to the gateway deployment

The SPIRL docs have this config added to the istio-proxy container:

            - name: istio-proxy
              volumeMounts:
              - name: workload-socket
                mountPath: /run/secrets/workload-spiffe-uds
                readOnly: true
            volumes:
              - name: workload-socket
                csi:
                  driver: "csi.spiffe.io"
                  readOnly: true

Will need to add a helm value to enable this in the Gloo Gateway template and Gloo Edge template.

Need to either have the solo sds container support using the provided workload-socket (don't initialize sds) and then configure the gateway_proxy_sds to point to that address.

Currently our config is this in the bootstrap:

        - name: gateway_proxy_sds
          connect_timeout: 0.25s
          http2_protocol_options: {}
          load_assignment:
            cluster_name: gateway_proxy_sds
            endpoints:
              - lb_endpoints:
                - endpoint:
                    address:
                      socket_address:
                        address: 127.0.0.1
                        port_value: 8234

And the sds container looks for a specific file path here.

Istio's istio-proxy config looks like this:

        - name: sds-grpc
          type: STATIC
          connect_timeout: 0.25s
          http2_protocol_options: {}
          load_assignment:
            cluster_name: sds-grpc
            endpoints:
              - lb_endpoints:
                - endpoint:
                    address:
                      pipe:
                        path: ./var/run/secrets/workload-spiffe-uds/socket

Where the path is set by [WorkloadIdentitySocketFile](the https://github.com/istio/istio/blob/master/pkg/istio-agent/agent.go#L370) to support the default SDS or listening to a custom provided socket like in the SPIRL case.

We would need to change out sds server to have the same logic as the istio agent to support the custom csi.spiffe.io value for the workload-socket instead of always looking for the files.

Alternatively, we could rely on istio-proxy to handle this, but it would require 1) translating the gateway_proxy_sds to match the istio-proxy cluster endpoint in the bootstrap config 2) this would break glooMtls support 3) to completely remove the sds sidecar we'd need to run istio-proxy's envoy, otherwise the sds serving won't start

bleggett commented 1 month ago

Where the path is set by [WorkloadIdentitySocketFile](the https://github.com/istio/istio/blob/master/pkg/istio-agent/agent.go#L370) to support the default SDS or listening to a custom provided socket like in the SPIRL case.

Istio uses a hardcoded path for the workload SDS socket - only the socket filename is customizable. The path is fixed as a const to /var/run/secrets/workload-spiffe-uds/WORKOAD_IDENTITY_SOCKET_FILE

The SPIRE CSI driver (which I assume is identical to the SPIRL-documented one) can be configured to mount the socket to any path within the volume.

So this should work if we do (1) and then configure the SPIFFE CSI driver to mount to the same fixed path GG/GE expects?

That's all we do in Istio, really.

EDIT: Ah, I missed that GG is proxying SDS by having Istio do it, then copying the certs to a path, then re-serving those to GG via SDS again with a proxy. TY @npolshakova for explaining. Should be able to ditch most of that and just contact SPIRE SDS directly via GG, yep.

bleggett commented 2 days ago

@nfuden this isn't completed, I don't think?

Namely, this has not been done:

We would need to change out sds server to have the same logic as the istio agent to support the custom csi.spiffe.io value for the workload-socket instead of always looking for the files.

GG still does things here that istio gateways do not, that make it impossible/cumbersome to use with SPIFFE/SPIRL.

k8sgateway / k8sgateway