Heads up: latest logstash-core will break devutils for some plugins

[OliverGavin]

Hi guys,

Just a heads up; I was testing out devutils using the latest logstash-core bundled with logstash (6.x) rather than the old version (logstash-core-5.6.4) that gets pulled with logstash-core-plugin-api-2.1.28-java. There is a regression that will appear in devutils when the logstash-core-plugin-api gets updated that causes plugins to be registered twice. This causes some plugins to break as this is unexpected behaviour.

4 years ago (https://github.com/elastic/logstash-devutils/blame/master/lib/logstash/devutils/rspec/logstash_helpers.rb#L50) A commit was made to devutils to register the filter plugins.

1 year ago (https://github.com/elastic/logstash/blame/master/logstash-core/lib/logstash/pipeline.rb#L487) A commit was made to Logstash (released under 6.x) which registers the filter plugins. The result is that the registration now occurs twice. More complex plugins such as logstash-filter-augment break when registered twice (how/why is irrelevant).

I know that logstash-core-plugin-api is only updated every so often (I'm guessing only when there are relevant changes to the api) but the recent change proves that the api has in fact changed since it causes this issue.

[IIIEII]

Also have same problem with Logstash 6.5.3 I'm creating some automatic tests for my complex logstash pipelines (according to this instruction: https://gquintana.github.io/2016/09/07/Testing-Logstash-configuration.html). I can see that every filter from pipeline is registered twice:

1. logstash/devutils/rspec/logstash_helpers.rb:

   def sample(sample_event, &block)
   ...
       pipeline.instance_eval { @filters.each(&:register) }
   ...
   end

2. logstash-core/lib/logstash/pipeline.rb:

   def filter(event, &block)
   maybe_setup_out_plugins
   ...
   end
   def maybe_setup_out_plugins
   if @outputs_registered.make_true
     ...
     register_plugins(@filters)
   end
   end

   For some filters with global context it's a problem. Currently I should use a hack. I override filter's register method and prevent second registration like this:

   
   require "logstash/filters/somefilter"
   class LogStash::Filters::Somefilter
   
   alias_method :reg, :register
   
   @@ids = []
   
   def register
   if @@ids.include? @id
     puts "Somefilter Filter already registered id:#{@id}"
   else
     puts "Somefilter Filter register id:#{@id}"
     @@ids << @id
     reg
   end
   end

end

However I want my tests to work normal without any hacks.