logsearch / logsearch-filters-seo

Apache License 2.0
2 stars 0 forks source link

Strip out spaces from robots field #2

Open mrdavidlaing opened 10 years ago

mrdavidlaing commented 10 years ago

"robots", stores the sent Robots-X-Tag HTTP headers. Is there a way to simply remove the spaces from these? Right now it shows noindex, follow and noindex,follow as different entities. See screenshot: http://uploads.yoast.nl/Kibana_3_-_Googlebot_19471DBC.png

dpb587 commented 10 years ago

Maybe something like this...

mutate {
  gsub => [ "robots", "\s+", "" ]
  split => [ "robots", "," ]
}

Which would result in something like (an array may be more beneficial than a primitive string, although indexing-wise they can be fairly similar)...

{
  "robots": [
    "noindex",
    "follow"
  ]
}

If a string is preferred (may have further issues on noindex, follow vs follow, noindex), should probably do something like this which will ensure there's always a space...

mutate {
  gsub => [ "robots", ",\s*", ", " ]
}