Logstash keep restarting after plugin installation

ISSUE TYPE

Bug Report

COMPONENT NAME

logstash-output-dynatrace

OS / ENVIRONMENT

CentOS Linux release 8.4.2105 4.18.0-305.25.1.el8_4.x86_64

CONFIGURATION

SUMMARY

I followed the steps from available documentation and together with Dynatrace support successfully installed the plugin.

[root@ip-xxx centos]# /usr/share/logstash/bin/logstash-plugin list | grep dynatrace logstash-output-dynatrace After that Logstash service kept on restarting in infinite loop. Here is the log:

Nov 25 23:18:52 ip-xxx.ap-southeast-2.compute.internal systemd[1]: Started logstash. Nov 25 23:18:52 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: Using bundled JDK: /usr/share/logstash/jdk Nov 25 23:18:52 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be remove> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: [FATAL] 2021-11-25 23:19:04.504 [main] Logstash - Logstash stopped processing because of an error: (PathError) The > Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: org.jruby.exceptions.StandardError: (PathError) The path/root/logstash-output-dynatracedoes not exist. Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.sour> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.sour> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.sour> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.defi> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1820) ~[jruby-complete-9.2.19.0.jar:?] Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.spec> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.defi> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.defi> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.defi> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.defi> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.runt> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.bundler_minus_2_dot_2_dot_30.lib.bundler.setu> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.lib.bootstrap.bundler.setup!(/usr/share/logstash/lib/bootstrap/bundler.rb:79) ~[?:?] Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1839]: at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89)> Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal systemd[1]: logstash.service: Main process exited, code=exited, status=1/FAILURE Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal systemd[1]: logstash.service: Failed with result 'exit-code'. Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal systemd[1]: logstash.service: Service RestartSec=100ms expired, scheduling restart. Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal systemd[1]: logstash.service: Scheduled restart job, restart counter is at 7. Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal systemd[1]: Stopped logstash. Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal systemd[1]: Started logstash. Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1891]: Using bundled JDK: /usr/share/logstash/jdk Nov 25 23:19:04 ip-xxx.ap-southeast-2.compute.internal logstash[1891]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be remove>

According to this log:

The path/root/logstash-output-dynatracedoes not exist.

But the directory is in place:

[root@ip-xxx centos]# ll -lth /root/logstash-output-dynatrace total 44K -rwxrwxrwx. 1 root root 4.6K Nov 12 03:51 Gemfile.lock -rwxrwxrwx. 1 root root 2.2K Nov 12 03:39 logstash-output-dynatrace.gemspec drwxrwxrwx. 3 root root 43 Nov 12 03:39 spec -rwxrwxrwx. 1 root root 83 Nov 12 03:39 CHANGELOG.md drwxrwxrwx. 2 root root 28 Nov 12 03:39 docs -rwxrwxrwx. 1 root root 991 Nov 12 03:39 Gemfile drwxrwxrwx. 3 root root 22 Nov 12 03:39 lib -rwxrwxrwx. 1 root root 12K Nov 12 03:39 LICENSE -rwxrwxrwx. 1 root root 712 Nov 12 03:39 Rakefile -rwxrwxrwx. 1 root root 2.9K Nov 12 03:39 README.md -rwxrwxrwx. 1 root root 5 Nov 12 03:39 VERSION [root@ip-xxx centos]# ll -lth /root/ total 24K -rwxrwxrwx. 1 root root 301 Nov 12 05:09 20-dynatrace-output.conf drwxrwxrwx. 7 root root 4.0K Nov 12 03:51 logstash-output-dynatrace -rw-------. 1 root root 6.2K Dec 4 2020 anaconda-ks.cfg -rw-------. 1 root root 5.9K Dec 4 2020 original-ks.cfg

STEPS TO REPRODUCE

Install the plugin on Centos using provided documentation.

This looks like a bug to me. I tried to change the ownership of the folder recursively with logstash user but it didn't help.

dynatrace-oss / logstash-output-dynatrace

Logstash keep restarting after plugin installation #14