search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.37k stars 1.06k forks source link

Stream lookup #11353

Open ed-graylog opened 3 years ago

ed-graylog commented 3 years ago

What?

As a Graylog user I would like to have a stream lookup to help enrich values from another local stream.

Why?

This would help us immensely for example if we had Stream A which is local store a mac ID and compare it to the value of Stream B which is remote. This field can be IP address, agent id (if app was reinstalled), or any field really. Since this information is "live" this cannot be accomplished with simple lookup tables. Since employees come and go constantly, this would be a helpful tool. Would this be similar in functionality to the HTTP Jsonpath or is this something else all together?

This old plugin (which is no longer maintained) is a great example.

Please see HS-560796954 for additional context and examples.

dennisoelkers commented 3 years ago

Hey @edward-marrufo, I do not understand yet the idea behind this feature. Can you explain a little bit more about how it should work?

victorfeng19 commented 3 years ago

Hi @dennisoelkers

Please refer to https://github.com/billmurrin/graylog-plugin-slookup-function

SLookup facilitates the lookup of a local stream's field value on a remote stream field, and if it matches, returns the requested fields for enrichment in the source stream.

For example, say there are two streams, one contains some http logs with source IPs (E.g. src_ip) from internal hosts and the other stream contains information about the systems on the network such as IP address (E.g. ip_address), computer name (E.g. computer_name), MAC address (E.g. mac_address), OU, make/model, etc.

In the example above, you might want to return the computer_name and mac_address fields where the value of src_ip matches ip_address.

The thought behind this function is to implement a similar functionality to the VLOOKUP function in Excel.

Victor