search

questdb / questdb

QuestDB is an open source time-series database for fast ingest and SQL queries
https://questdb.io
Apache License 2.0
14.29k stars 1.14k forks source link

Add ILP traffic volume metrics #4751

Open puzpuzpuz opened 2 months ago

puzpuzpuz commented 2 months ago

Is your feature request related to a problem?

We should add new "total bytes" metrics for ILP/TCP and ILP/HTTP to monitor the ingestion network traffic.

Describe the solution you'd like.

No response

Describe alternatives you've considered.

No response

Full Name:

Andrei Pechkurov

Affiliation:

QuestDB

Additional context

No response

sriram-V20 commented 1 month ago

Hi @puzpuzpuz,

I'd love to work on this issue. As I'm new here, it might take me some time to complete it. If you have any tips or guidance, it would be really helpful.

puzpuzpuz commented 1 month ago

Hi @sriram-V20

The new Counter metrics should be added to LineMetrics: line_tcp_recv_bytes for ILP/TCP and line_http_recv_bytes for ILP/HTTP. The counters should be incremented in LineTcpConnectionContext (ILP/TCP) and LineHttpProcessor (ILP/HTTP).

Finally, it'll be nice to have some tests around the new metrics.

sriram-V20 commented 1 month ago

@puzpuzpuz Thank you for the details.

For the new Counter metrics in LineMetrics, should I write tests that mock TCP/HTTP requests to verify the increment of line_tcp_recv_bytes and line_http_recv_bytes counters? Also, could you point me to any existing tests that handle similar scenarios?

puzpuzpuz commented 1 month ago

@sriram-V20 some tests wouldn't hurt. You could take a look at IODispatcherTest#testTriggerInternalCriticalCairoException() as it's somewhat similar. You could add a similar test for ILP/HTTP.