search

envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.82k stars 4.77k forks source link

Enhance envoy Tap to dump all message with high performance #34424

Closed coolg92003 closed 1 month ago

coolg92003 commented 4 months ago

Title: Enhance envoy transport Tap to dump all message

Description: Because of Network security, It can't read the package content because it is encrypted since all CDS connections will go to TLS even if dumping network traffic by tool tcpdump. Therefore, there is an urgent needed to dump all messages per CDS to Data collecting center if there is debugging and checking needs However, currently, no matter HTTP tap or transport tap, the performance is low

[optional Relevant Links:] There is a link about grpc https://www.youtube.com/@cncf

My thought: For dumping all messages per CDS, I think it is better to use UDP. As for message format, it based on configuration. The major reason: 1) in Live site, it could be 1k-10k HTTP request per second and UDP has high performance 2)UDP has little application and kernel cost in envoy, as it is known, envoy tap will be client, if use TCP, it has to main TCP connection, also, if envoy has 100 works thread, then it has to maintain 100 tcp connections. if all works put dumped package to another thread queue, then it has to lock/unlock. Therefore the cost is very low if sending package from each work thread by UDP. 3) why not use grpc? major reason is maintain cost, need tcp and tls 4)then how to solve data format and path security to Data collecting center? developer a local UDP server to solve them, or create a new CDS type to support like HEP3(maybe can do it later)

this is my current thought and welcome any suggestion! image

coolg92003 commented 4 months ago

include @yiyibaoguo

coolg92003 commented 3 months ago

any other thoughts?

coolg92003 commented 3 months ago

below sink: https://github.com/envoyproxy/envoy/issues/28838

coolg92003 commented 3 months ago

issue is reported in below https://github.com/envoyproxy/envoy/issues/32068

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

github-actions[bot] commented 1 month ago

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.