CSM BDS fixes related to date-time format issues

[thanh-lam]

Purpose

Fixes for issues #992 #995. The mutate filter plugin gsub in logstash changes the date field it reads into the desired format before writing it to Elasticsearch index: cast-allocation. With this change in date format, those python scripts that use the old format have to be changed to use the new format:

    modified:   csm_big_data/logstash/config/logstash.conf
    modified:   csm_big_data/python/findJobKeys.py
    modified:   csm_big_data/python/findJobMetrics.py
    modified:   csm_big_data/python/findJobTimeRange.py

Approach

Updated logstash.conf file and modified python scripts in /opt/ibm/csm/csm_big_data/python/.

Origin

992

995

How to Test

For the logstash.conf file:

1. Copy CAST/csm_big_data/logstash/config/logstash.conf to /etc/logstash/conf.d/logstash.conf
2. Restart logstash: "systemctl restart logstash"

For the scripts, just try to run them in usual tests.

Open Questions and Pre-Merge TODOs

• [ ] Assign @williammorrison2 to review (So they can check regression)
• [ ] Assign 1 developer to @besawn

[thanh-lam]

@besawn , I checked in an "urgent" fix late yesterday for changing TIME_SEARCH_FORMAT in cast_helper.py

#TIME_SEARCH_FORMAT = "epoch_millis"
TIME_SEARCH_FORMAT = "strict_date_optional_time_nanos"

This fixes the problem of Elasticsearch query by time range. Part of this time range query not working was due to the search time format in epoch_millis but the date+time format of "begin_time" and "end_time" are in nano seconds.

@williammorrison2 and I tested this fix yesterday and it worked very well. So we didn't want to miss this PR. Changing TIME_SEARCH_FORMAT also led to eliminating the call to convert_time(). Another effect of this fix is that: User has to input "complete" time format for --starttime and --endtime. For example:

"2021-03-08T16:00:00" instead of "2021-03-08T16".

I'll add comments into #992 to document this fix.