search

cloudflare / pingora

A library for building fast, reliable and evolvable network services.
Apache License 2.0
20.3k stars 1.1k forks source link

When upstream is a Kubernetes service IP, traffic almost go to just one endpoint #138

Closed rongyi closed 3 months ago

rongyi commented 3 months ago

we set one upstream IP of a k8s serviceIP, which in turn will LoadBalance traffic to two pods.

As we read the doc, there's connection pool in pingora, and we cancel the connection pool as doc said to do this:

        peer.options.idle_timeout = Some(Duration::from_secs(0));

but we still get high unbalanced traffic between two pods. Can some one help us?

Object905 commented 3 months ago

I've approached this from different perspective, by using https://github.com/kube-rs/kube. And turning my proxy into kind of static ingress. Where router listens for changes in endpointslices and updates backends with ServiceDiscovery in background service. Here's gist, maybe you will find this helpful https://gist.github.com/Object905/6cafd5e8e56dd60670149296411a407f

rongyi commented 3 months ago

you light my day @Object905 thank you!

rongyi commented 3 months ago

can you create an repo for this? I have some issue for the partial code. @Object905

Object905 commented 3 months ago

I will, in near future. Right now I'm migrating some varnish scripts at work to pingora, after I'll try to come up with nicer api for services/routing/registering, generic enough to be a library, not just internal tool.

oliverjumpertz commented 3 months ago

Although there has already been a solution proposed, I just wanted to add my 2c here:

When dealing with things like traffic distribution in k8s, I would highly suggest looking at things like service meshes to solve this problem. Istio or Linkerd would most probably solve your problem immediately, as their proxies do all the heavy lifting through their load-balancing algorithms, freeing you from having to add more custom code than necessary to your proxy.

You just save having to reimplement a lot of things you can get for free. :)

eaufavor commented 3 months ago

Just a note on the original question: connection pooling is not directly related to unbalanced requests. It is up to the user of the Pingora proxy to decide where a traffic to go to. After that, the connection pool can help save a new handshake if there is already a connection to reuse. Connection pooling has no influence on where a request should go to.

If you see unbalanced traffic, get some debug information around you upstream_peer to see what causes the uneven selection.

github-actions[bot] commented 3 months ago

This question has been stale for a week. It will be closed in an additional day if not updated.

github-actions[bot] commented 3 months ago

This issue has been closed because it has been stalled with no activity.