logsearch-filters-{name} or logstash-filters-{name}

[mrdavidlaing]

We seems to have two competing names for the filters

• logsearch-filters-common
• logstash-filters-cf

Choose and standardise, dernit.

[mrdavidlaing]

@dpb587, @wdneto, @sopel - your 2p please

[mrdavidlaing]

my 2p goes for logsearch-filters-{name}

[dpb587]

FWIW, I initially named it logsearch-filters-common because it maintains conventions which are a subset of logstash (e.g. that @type will have the log message type).

[aristotelesneto]

Being new, I'm not sure why they are separate repo's or the significance of the numbering scheme (other than the order they are concatenated). The numbering scheme made me think of the very well organised owasp mod security rule set: https://github.com/SpiderLabs/owasp-modsecurity-crs/. I'd love to be enlightened :)

Having said that though, I'd vote for logsearch-*, regardless of whether they are stock standard logstash filters or not unless there was a pre-existing ecosystem of rules elsewhere that could just be dropped-in to logsearch - in which case it could make sense to maintain a distinction between logsearch-specific or not.

[dpb587]

In theory, the separate repos would make it easier for users to pick and choose the filters they need instead of having one big conglomerate of a filter repo. We have a private repo as well with our business-specific rules. It might be worth reconsidering the separate repo approach in the case of cf and common.

The numbering scheme allows us to be explicit about the order in which filters are processed. For example, we have some global "pre-filters" that we apply to messages which trim or drop empty messages, or filters to ensure consistency with type/@type and message/@message. Or "post-filters" which could perform further lookups on specific "shared" keys after they've been parsed, like geoip whenever ip exists.

[aristotelesneto]

Thanks, needing an ability to merge other similarly structured filters sounds reasonable.

I'm aware of the effect of the numbering - I trying to establish whether there was a scheme for which numbers to use for which purpose filter - or is 75-, 100- arbitrary?

[dpb587]

Ah, good question. No, they're fairly arbitrary with the current usage following:

• range 00 thru 99
• our internal repos currently have our log filters named at 50
• the open source filters are 75

I think we'll need to formalize and reconsider though. The open source ones as 75 don't really make sense - they'll get applied regardless.

[mrdavidlaing]

I think we're all in agreement that it should be logsearch-filters-{name}

[aristotelesneto]

I just noticed that this means renaming both common as well as cf filters as currently only the repository for the common filters is named after 'logsearch-*'.

I say we still move ahead with it, but just thought I point it out.

[mrdavidlaing]

@wdneto, @dpb587 - No advantage to delaying the pain then. I've renamed all repos to logsearch-filters-{name}

Please expect (and fix) the resulting breakages

[mrdavidlaing]

Renaming done. Let the breakages and fixes flow :scream: