envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.95k stars 4.8k forks source link

Network::Utility::getLocalAddress performance is bad in multi-threads #35137

Closed soulxu closed 1 month ago

soulxu commented 3 months ago

When running benchmark with nighthawk on quic, it found that the quic client side spend a lot of time on the Network::Utility::getLocalAddress

image

The throughput can't be raised after concurrency 30 image

If the Network::Utility::getLocalAddress was removed, it get much better image

The flamegraph also changed. image

the nighthawk is running as below: nighthawk_client --rps 100 --connections 20000 --duration 30 --concurrency <concurrency> -v warn --protocol http3 --address-family v4 --request-body-size 0 --request-method GET --max-requests-per-connection 1 --timeout 120 --max-active-requests 100000 https://www.example.com:13333/

Thinking of there are two ways to fix this:

Looking for feedback on which way is better.

nezdolik commented 3 months ago

cc @jmarantz @wbpcode

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

github-actions[bot] commented 2 months ago

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

jmarantz commented 2 months ago

@RyanTheOptimist

RyanTheOptimist commented 2 months ago

@danzh2010 Did this issue get fix by your recent IP caching change?

danzh2010 commented 2 months ago

The expensive call is getifaddrs() which should be fixed by @abeyad 's ongoing work of moving away from getLocalAddress() and use connected UDP socket instead. My IP caching change is for a different issue where Address::Instance object creation is too expensive.

RyanTheOptimist commented 2 months ago

/assign @abeyad

abeyad commented 2 months ago

This issue should be fixed by https://github.com/envoyproxy/envoy/pull/35652, but @soulxu could you kindly verify?

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.