Open vaishali-kare opened 10 months ago
Here is an example text coming from events that we are missing entirely:
"could not find object for source"
Hey @vaishali-kare thanks for opening this issue and I am glad to help. It seems to me that some rows have issue but some don't. For example, the 5th row beginning with wordReinit
has no issue. If scripted field is the problem then we should expect all rows fail. Do you think you could share the mapping of the index, and some data failed to use scripted field and some that have no issue?
The index mapping is as below: {fw-logs-2023.11.20": { "mappings": { "LogMessage.keyword": { "full_name": "LogMessage.keyword", "mapping": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }
Working data: ReInit channel[nspos-resync-fw-0@mdm-server-0.mdm-server-svc-headless.svc.cluster.local@30000] scheduled for MDM mdm-server-0, address=mdm-server-0.mdm-server-svc-headless.svc.cluster.local:30000
unique_log_message__ ReInit channel[nspos-resync-fw--@mdm-server--.mdm-server-svc-headless.svc.cluster.local@-] scheduled for MDM mdm-server--, address=mdm-server--.mdm-server-svc-headless.svc.cluster.local:-
Not working one: could not find object for source ={scheme=app, namespace=mdm-ami, fdn={"NE":"22","classId":"openconfig-network-instance:/network-instances/network-instance/protocols/protocol/isis/interfaces/interface/levels","network-instance_name":"default","protocol_name":"1","protocol_identifier":"openconfig-policy-types:IS","interface_interface-id":"INT-R5"}} , class=/openconfig-network-instance:network-instances/network-instance/protocols/protocol
Any update on this issue?
Ou observation is:
When Length of LogMessage is 175 which is less than 256 and hence respective unique_log_message is displayed successfully.
When Length of LogMessage higher than 256 and the respective unique_log_message is not displayed or shows as blank.
What is the correct method to increase the ignore_above value from 256?
Describe the bug
With v 2.10.0 version, unique_log_message scripted field per OpenSearch dashboard doesn't reliably and consistently work.
Purpose of unique_log_message scripted field is having few panels that attempt at unifying LogMessage by padding any integer in the LogMessage with the purpose of providing a summary report showing for each error type (LogMessage after padding integers) and number of occurrences of such error type. With that, we try to save user from having to navigate thousands of events and avoid missing any particular error type because of high volume of events, a lot of which could be duplicates of the same error type, it just that we have so many instances of the same error.
To Reproduce Steps to reproduce the behavior:
if (doc.containsKey('LogMessage.keyword') && doc['LogMessage.keyword'].size()>0) { return /\d+/.matcher(doc['LogMessage.keyword'].value).replaceAll('-'); }
Expected behavior The OpenSearch dashboard should display accurate results for the unique_log_message scripted field.
OpenSearch Version v 2.10.0
Dashboards Version v 2.10.0
Screenshots
Host/Environment (please complete the following information): OS: Windows Browser and version: Microsoft Edge for Business Version 119.0.2151.44 (Official build) (64-bit)