dCache / logstash4dcache

logstash parser and config for integration with dCache
5 stars 2 forks source link

Tell if the entry is a pool: or a door: #3

Open martbhell opened 10 years ago

martbhell commented 10 years ago

Would be nice to be able to filter/populate a field depending on if the billing line is a pool: or a door: line (and only the first pool:). Like:

06.12 23:59:59 [pool:csc_fi_38:transfer] 06.12 23:59:59 [door:Xrootd-madhatter@xrootd-madhatterDomain:request]

One would need to update the CELL_AND_TYPE but I couldn't get the right syntax without messing things up..

martbhell commented 9 years ago

I have something working, but it's not optimal (would be good to always populate a cell_type field) so not pushing the update.

Method:

0 remove %{BILLING_TIME} from all the grok patterns (keep it by itself) 1 a new grok filter before the primary that only matches to %{BILLING_TIME} and overwrites message field (excluding billing_time) 2 make the %{CELL_TYPE} and %{CELL_AND_DOMAIN} populate a field called cell_and_type 3 finally two mutate filters that have (replace pool with door for door..):

if [cell_and_type] =~ "^\[pool" {
  mutate {
    add_field => { "cell_type" => "pool" }
    remove_field => [ "cell_and_type" ]
  }
kofemann commented 9 years ago

Hi Johan,

we will definitely look at it, Thanks for the suggestions.

Tigran.