els0r / goProbe

High-performance IP packet metadata aggregation and efficient storage and querying of flows
GNU General Public License v2.0
12 stars 4 forks source link

[feature] Support for server sent events (SSE) in global-query API #337

Closed els0r closed 3 months ago

els0r commented 3 months ago

Background

This commit adds functionality to the global-query server: it provides an SSE (server-sent-events) endpoint for querying, which pushes partial results back to the client whenever a new result has been received by the distributed querier.

The premise of this change is that, for long-running queries involving many hosts, the user already gets feedback on progress.

Client

goQuery now has a flag --query.streaming which uses an SSE client capable of parsing the event stream provided by global-query's SSE endpoint. This client supports two callback functions:

to communicate progress. So far, these functions are hard-coded in root.go and don't do much beyond logging. They may be extended in the future.