search

gchq / stroom

Stroom is a highly scalable data storage, processing and analysis platform.
https://gchq.github.io/stroom-docs/
Apache License 2.0
424 stars 56 forks source link

Data Feed Keys #4312

Open gcdev373 opened 2 weeks ago

gcdev373 commented 2 weeks ago

Similar in concept to Stroom API keys, data feed keys would allow sources to send data to Stroom and especially Stroom Proxy.

A one-to-one mapping between data feed keys and feeds would give a good level of granularity of control over data supply. In this case there should be no need to specify FEED as a header arg to the data feed API. However, if going down this route, it would be necessary that every feed has an associated data feed key. So, it should be possible to view the data feed key for any feed that is visible in the UI.

Some thought needs to be given to how to run in a mixed mode where perhaps not all feeds are authenticated this way, but could use other mechanisms (certs or OIDC).

at055612 commented 2 weeks ago

My original thinking was that a datafeed key would be 1-* with feed, so a system with >1 feed would have one datafeed key to use for all their feeds. This would still ensure that a system holding a datafeed key could only send data to the feeds associated with that feed.

Going 1-1 means more keys for a client system (and stroom admin) to manage but like you say does remove the requirement for the feed to be sent.

Datafeed keys would also NOT be associated with a user, unlike API keys, and would not have any permissions on the API so can only be used for data feed on 1 (or maybe multiple feeds).