One of our engineers was frustrated that they could not develop an aggregated Kibana visualisation as they had hoped to. They wanted to look at the frequency of errors grouped by their message. They could not use the message field as it shows as non-aggregatable in Elastic (presumably due to its length). They could not use the short_message field since it is empty when the message field is not truncated.
It took me some time to track this back to this conditional in the plugin code, as I initially assumed it was the Logstash GELF library that is used by the client that was leaving the short_message out. The docs for this plugin do not indicate that the remap functionality will actually remove the short_message in the event it matches the message field.
It seems to me that the short_message should be preserved, or that behaviour should be properly documented.
Version: 6.2.2
Operating System: CentOS
Config File (if you have sensitive info, please remove it):
{
"version": "1.1",
"host": "example.org",
"short_message": "This is less than 250 characters.",
"full_message": "This is less than 250 characters.",
"timestamp": 1385053862.3072,
"level": 1,
"_user_id": 9001,
"_some_info": "foo",
"_some_env_var": "bar"
}
Steps to Reproduce:
Feed the sample data to the gelf input plugin and observe that the output does not contain short_message.
One of our engineers was frustrated that they could not develop an aggregated Kibana visualisation as they had hoped to. They wanted to look at the frequency of errors grouped by their message. They could not use the
message
field as it shows as non-aggregatable in Elastic (presumably due to its length). They could not use theshort_message
field since it is empty when themessage
field is not truncated.It took me some time to track this back to this conditional in the plugin code, as I initially assumed it was the Logstash GELF library that is used by the client that was leaving the
short_message
out. The docs for this plugin do not indicate that the remap functionality will actually remove theshort_message
in the event it matches themessage
field.It seems to me that the short_message should be preserved, or that behaviour should be properly documented.
Feed the sample data to the gelf input plugin and observe that the output does not contain
short_message
.