search

kubernetes-sigs / apiserver-network-proxy

Apache License 2.0
369 stars 176 forks source link

Agent to server communication through an egress proxy #127

Open jieyu opened 4 years ago

jieyu commented 4 years ago

Currently, the agent to server communication is based on gRPC streams. There're cases where an agent's egress has to go through an egress proxy. In many cases, the egress proxy does not support gRPC protocol directly.

Is there a recommended way to solve this problem? For example, has anyone tried tunnel the grpc http/2 traffic through an HTTP CONNECT based proxy? Is that a supported model?

cheftako commented 4 years ago

Several people have floated the idea of having an alternate to gRPC for Agent <-> Server communication. It seems clear some refactor will need to be done to make that happen. (It certainly took some of Client <-> Server communication). While I think everyone on the project is in support of making that work, it will require someone who actually has a use case to drive it.

jieyu commented 4 years ago

Looks like gRPC already supports HTTP CONNECT tcp tunnel (most proxies support that): https://github.com/grpc/grpc-go/blob/master/proxy.go#L140 https://github.com/grpc/proposal/blob/master/A1-http-connect-proxy-support.md

I tested this by launching a squid proxy using docker

docker run -v $(pwd)/squid.conf:/etc/squid/squid.conf -p 3128:3128 --rm -ti sameersbn/squid:3.5.27-2

squid.conf

http_port 3128
http_access allow all

Launch the proxy server

./bin/proxy-server --mode=http-connect --server-ca-cert=certs/master/issued/ca.crt --server-cert=certs/master/issued/proxy-master.crt --server-key=certs/master/private/proxy-master.key --cluster-ca-cert=certs/agent/issued/ca.crt --cluster-cert=certs/agent/issued/proxy-master.crt --cluster-key=certs/agent/private/proxy-master.key

Launch the proxy agent by setting HTTPS proxy env vars.

HTTPS_PROXY=127.0.0.1:3128 HTTP_PROXY=127.0.0.1:3128 ./bin/proxy-agent --ca-cert=certs/agent/issued/ca.crt --agent-cert=certs/agent/issued/proxy-agent.crt --agent-key=certs/agent/private/proxy-agent.key --proxy-server-host=10.0.1.120

(Note that you cannot use 127.0.0.1 for proxy-server-host as golang will default to not use proxy for localhost)

So this works as expected.

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

cheftako commented 3 years ago

/lifecycle frozen

cheftako commented 3 years ago

I believe this is being looked into with KEP https://github.com/kubernetes/enhancements/pull/2025