flant / loghouse

Ready to use log management solution for Kubernetes storing data in ClickHouse and providing web UI.
Apache License 2.0
924 stars 76 forks source link

Parse json keys in log #161

Open kiaplayer opened 3 years ago

kiaplayer commented 3 years ago

Improvement for #115. Allows to parse inner json objects (only one level deep).

For example, we have this log message:

{
    "message": "some message",
    "someData1": {
        "someKey1": "someValue1",
        "someKey2": "someValue2"
    },
    "someData2": {
        "someKey3": "someValue3",
        "someKey4": "someValue4"
    }
}
We'll get this data in Clickhouse: string_fields.names string_fields.values
['message', 'someData1.someKey1', 'someData1.someKey2', 'someData2.someKey3', 'someData2.someKey4'] ['some message', 'someValue1', 'someValue2', 'someValue3', 'someValue4']

Now we can search records in Clickhouse by inner fields:

SELECT
    arrayElement(`string_fields.values`, indexOf(`string_fields.names`, 'someData1.someKey1')) AS searchValue
FROM logs
WHERE searchValue == 'someValue1'
ORDER BY timestamp DESC;