Add array_remove_duplicate.

[adamjt]

This PR addresses comments at:

https://github.com/logstash-plugins/logstash-filter-unique/issues/1

[jordansissel]

Code looks good.

Docs: Can you also add docs to docs/index.asciidoc for this new setting?

[jordansissel]

Also looks like there's a git conflict that needs resolving.

[shoito]

@adamjt I hope this to be merged🙏

[adamjt]

@shoito Whoops, I'll fix this up this week!

[adamjt]

@jordansissel @shoito I've updated the merge conflict and asciidoc - Travis is failing on something seemingly unrelated. invalid level[OFF] for logger[]

[ekho]

@adamjt could you push something to PR to trigger travis build? Issue with invalid level has been fixed at https://github.com/elastic/logstash/issues/8518

[adamjt]

@ekho All set, thanks!

[ekho]

@jordansissel all problems has been fixed. Could you merge PR ?

[jordansissel]

In future PRs, please disable your editor's "trim whitespace" feature as it makes the diff very noisy and hard to review.

[jordansissel]

Ahh, turns out we already have a filter to do this feature: https://github.com/logstash-plugins/logstash-filter-unique.

Let's use that filter instead.

[adamjt]

@jordansissel This PR was opened based off of https://github.com/logstash-plugins/logstash-filter-unique/issues/1

[ekho]

@jordansissel to use logstash-filter-unique my pr https://github.com/logstash-plugins/logstash-filter-unique/pull/2 should be merged.

[andrewvc]

@ekho apologies for the delay, we've just published a gem with your patch to the unique filter! Thanks for all the great work :) I'm going to close this PR as a result.