[plugin manager] Unused plugins aren't removed from the `local_gems` or the bundle's gems directory.

[gquintana]

# find logstash-2.4.1  -name 'kafka.rb'
logstash-2.4.1/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-2.0.9/lib/logstash/inputs/kafka.rb
logstash-2.4.1/vendor/bundle/jruby/1.9/gems/logstash-output-kafka-2.0.5/lib/logstash/outputs/kafka.rb
logstash-2.4.1/vendor/local_gems/facb65f6/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb
logstash-2.4.1/vendor/local_gems/78d8d5c6/logstash-output-kafka-5.1.1/lib/logstash/outputs/kafka.rb
logstash-2.4.1/vendor/local_gems/fe38c577/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb
logstash-2.4.1/vendor/local_gems/92fa9e22/logstash-output-kafka-5.1.1/lib/logstash/outputs/kafka.rb
logstash-2.4.1/vendor/local_gems/9ba87163/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb
logstash-2.4.1/vendor/local_gems/3f6473d0/logstash-output-kafka-5.1.1/lib/logstash/outputs/kafka.rb

For all general issues, please provide the following details for fast resolution:

• Version: 2.4.1
• Operating System: RHEL7
• Steps to Reproduce: Run multiple times:

  logstash-plugin install --no-verify --local /tmp/logstash-input-kafka-5.1.0.gem

  This command was idempotent in Logstash 2.3

[ph]

I've update the title with the right description.

What actually happen here is that unused plugins are not correctly cleanup when you install a new one. Only the latest installed will be actually loaded into logstash correctly.

You can verify that by doing cat Gemfile*.lock | grep kafka

This was removed in https://github.com/elastic/logstash/pull/6340, I will try to find better solution compatible with the changes with did for xpack.

[cameronkerrnz]

This is also an issue in 6.3.0. Thanks for the Gemfile.log trick; I was trying to figure out a way to find out where a plugin was coming from, so that helps my documentation at least.

[cameronkerrnz]

This also happens when doing a logstash-plugin update (updating to 7.16.1; the release notes called out updating plugins as a thing I need to do (which surprised me, seing as the plugins came in the RPM.

Thing to point out though, that with log4j vulnerabilities being all the rage these days, folks that use vulnerability scanners are going to still see a lot of vulnerable systems. At least Tenable (if you ask it to do a 'thorough' search) will look for log4j instances in the filesystem.

Is there a way we can clean up the old on-disk legacy of older plugins?

[cameronkerrnz]

For users of logstash installed from RPM, here is a command that could tell you which files (logstash plugins) are not a part of the current logstash RPM owned files. It doesn't include local_gems though, so might only be useful for instances of updating plugins supplied by the RPM.

comm -13 \
  <(rpm -ql logstash | grep /usr/share/logstash/vendor/bundle/ | sort ) \
  <(find /usr/share/logstash/vendor/bundle/ -type f | sort)

A better strategy would be to run logstash in a container though; this is one area is where immutability clearly trumps idempotency; maybe I should look to run logstash in podman.