Undefined method has_key

[aarunkushwahaa]

2019-09-23 16:23:36 -0400 [warn]: #3 emit transaction failed: error_class=NoMethodError error="undefined method has_key?' for \"3001211\":String" location="/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-brevity-control-0.0.2/lib/fluent/plugin/filter_brevity_control.rb:30:inblock (3 levels) in filter_stream'" tag="system" 2019-09-23 16:23:36 -0400 [warn]: #3 suppressed same stacktrace 2019-09-23 16:23:36 -0400 [error]: #3 unexpected error on reading data host="127.0.0.1" port=58762 error_class=NoMethodError error="undefined method `has_key?' for \"3001211\":String"

My configuration

@type brevity_control interval 15s num 2 attr_keys log.threadname.to_s, service, process max_slot_num 100000 stats_msg_fields host, log.threadname, service, process, system, systemid stats_msg_tag company.logging.tmplog all_keys_used true @type brevity_control interval 15s num 2 attr_keys alarm.id.to_s, alarm.severity max_slot_num 100000 stats_msg_fields host, alarm.key.to_s, alarm.id.to_s, alarm.name, alarm.data stats_msg_tag company.logging.log all_keys_used true

[nanaal]

There are two configurations for brevity_control, which one is used in this issue? Please make sure whether all the keys which are configured in "attr_keys" are contained in your record. It will be better if you can send a sample of your record. Thanks.

`@type brevity_control interval 15 snum 2 attr_keys log.threadname.to_s, service, process max_slot_num 100000 stats_msg_fields host, log.threadname, service, process, system, systemid stats_msg_tag company.logging.tmplog all_keys_used true

@type brevity_control interval 15 snum 2 attr_keys alarm.id.to_s, alarm.severity max_slot_num 100000 stats_msg_fields host, alarm.key.to_s, alarm.id.to_s, alarm.name, alarm.data stats_msg_tag company.logging.log all_keys_used true`

[aarunkushwahaa]

LOG sample {"host":"host-0","level":"WARN","log":{"classname":"com.common.server.monitoring.notifier.ServiceStateMonitorImpl$ServiceInstanceStateExpiryChecker:1738","message":"State expired for monitored service instance 'BaseMonitoredServiceInstance{instanceName='sps-0-spssc', serviceName='selfcare', state='DOWN', properties={}}'. Current time 'Fri Sep 06 01:24:35 EDT 2019', last updated time 'Fri Sep 06 01:24:15 EDT 2019', time-to-live '10'. Service instance being marked as down.","stacktrace":"","threadname":"ServiceStateMonitorImpl-State-Expiry-Audit-1"},"process":"run","service":"SPS","time":"2019-09-06T01:24:35.625Z","timezone":"America/Toronto","type":"log","system":"IG_19_9_R1_I73","systemid":"IG_19_9_R1_I73"}

@type brevity_control interval 60 num 0 attr_keys log.message, service, process max_slot_num 100000 stats_msg_fields host, service, process stats_msg_tag nokia.logging.tmplog all_keys_used true

ALARM sample: {"level":"NOTICE","time":"2019-09-04T08:59:26.033Z","alarm":{"name":"FluentdServiceInactive","task":"NOTIFY","key":"MOC018","event-type":"11","text":"Fluentd is inactive","probable-cause":"714","id":"3001217","severity":"MAJOR","data":"sps-ig-oame-0:8001: Fluentd is inactive"},"type":"alarm","host":"sps-ig-oame-0:8001","timezone":"America/Toronto","system":"IG_19_9_R1_I70","systemid":"CBAM-14fb88490394425b9ae564c92279"}

@type brevity_control interval 3600 num 2 attr_keys alarm.id, alarm.severity max_slot_num 100000 stats_msg_fields host, alarm.key, alarm.name, alarm.id stats_msg_tag nokia.logging.log all_keys_used true

These two cases for logs and alarms..

My questions:

1. Can you have num = 0 ?? I mean just stop all messages of similar pattern and display on brevity message.
2. Can you have interval = 0 ?? Though it doesn't make sense.
3. Since interval denotes time, is it always in seconds? I mean plugin says that 'interval' is of type 'integer' but not of type 'time', so it's not possible to write interval as 1m or 1h rather we need to write 60 for 60s , etc? 4.If any attribute in stats_msg_fields iff not present in record , will it throw error??
4. attr_keys needs string values as said in plugin. But my attribute alarm.id is integer. So does the plugin internally converts it to string?? I don't think so because i saw plugin. If not, how can we convert it to string. I can't change the plugin. So please advice what changes i can make inside td-agent.conf file. .. though i read about 'types' section inside parse which helps to convert to whatever data type we need. can you give sample to convert?
5. If i pass message field of my record as stats_message_field, it overwrites the message field of brevity. I think this is also a bug because stats_msg_field means these fileds to be appended in brevity record but not to overwrite the message section of brevity.

[aarunkushwahaa]

Issue was related to brevity data type conversion https://github.com/nokia/fluent-plugin-brevity-control/issues/2 . the issue is resolved now