CSM BDS logstash dependency fixes

[williammorrison2]

Purpose

This fixes the dependency/compatibility issues with logstash that we are testing with logstash-7.5.1-1_ol001.el8.ppc64le.rpm

additional prerequisite: java-11 is also needed.

Needed dependencies while setting up:

• logstash-codec-plain -v 3.0.6
• jar-dependencies -v 0.4.0

The csm_big_data/logstash/plugins/csm_event_correlator/build.sh script would try to pull in the latest versions of

• logstash-codec-plain 3.1.0
• jar-dependencies 0.4.1

This was the error message while trying to install the CSM BDS RPMS ibm-csm-bds-, ibm-csm-bds-logstash-

ERROR: An error occured when installing the: file:///opt/ibm/csm/bigdata/logstash/plugins/logstash-filter-csm-event-correlator.zip, to have more information about the error add a DEBUG=1 before running the command., message: You have requested:
  logstash-codec-plain = 3.1.0

The bundle currently has logstash-codec-plain locked at 3.0.6.
Try running `bundle update logstash-codec-plain`

If you are updating multiple gems in your Gemfile at once,

How to Test

1. Modify and create the appropriate files:
   • csm_big_data/logstash/CMakeLists.txt (added in the jruby patch script as an executable)
   • csm_big_data/logstash/patches/csm_logstash_patch_jruby_9.2.8.sh (added this script The user might need to patch the jruby jar to a new level due to the included jar file having a back level)
   • csm_big_data/logstash/plugins/csm_event_correlator/Gemfile (added in gem 'jar-dependencies', '~> 0.4.0')
   • csm_big_data/logstash/plugins/csm_event_correlator/build.sh (added in following lines)

     gem fetch logstash-codec-plain -v 3.0.6
     gem install jar-dependencies -v 0.4.0

2. Build CSM against the changes that were implemented.
3. Install the CSM and BDS environment
4. Run some CSM allocations through the pipeline to ensure the whole ELK stack is working.
5. View the details of the allocations in the Kibana GUI. Here are some useful links:

   Logstash

   logstash-7-5-1 download logstash-7-5-1 guide

   RubyJems

   logstash-codec-plain 3.0.6 jar-dependencies 0.4.0 jruby-jars 9.2.8.0

Open Questions and Pre-Merge TODOs

• [ ] @besawn to review code.
• [x] @thanh-lam to review code (IST Test).
• [x] @williammorrison2 to review testing and CSM FVT regression.