Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.32k stars 1.05k forks source link

Ability to Round numbers #8620

Open nick-piz opened 4 years ago

nick-piz commented 4 years ago

Please can we have the ability to Round numbers in Pipelines.

Beats reports all memory and file system sizes in bytes. I want to convert these to KiB, MiB, GiB and TiB (it's similar for uptime in ms where I don't want to round to a datetime but number of days, hours, minutes or seconds).

That should be as simple as dividing each value in turn by 1024. This can give very large decimal place values that are difficult to display with dashboards.

to_round(value, decimal places) would be ideal (perhaps even an option for up or down, floor ceiling etc.).

An example:

metricbeat_system_filesystem_total 42303746048

I want to convert and store this value as KiB, MiB, GiB and TiB so I write the following Pipeline rule.


when

has_field("metricbeat_system_filesystem_total")

then

//Maths

let size_bytes = to_double($message.metricbeat_system_filesystem_total);
let size_KiB = size_bytes / 1024.00;
let size_MiB = size_KiB / 1024.00;
let size_GiB = size_MiB / 1024.00;
let size_TiB = size_GiB / 1024.00;

The values should be as follows:

41312252 40343.99609375 39.39843368530273 0.0384750328958035

I need to get these to two decimal places so my only option at the moment is to split the number on ".", convert the two halves of the number to strings and then substring the second half of the number to two characters. Then concat them back together and convert to double.


//Round Numbers

let size_KiB_split = split("\\.", to_string(size_KiB)); //split KiB by .
let size_KiB = to_double(size_KiB_split[0]);
let size_KiB1 = concat(to_string(size_KiB_split[0]), "."); 
let size_KiB = to_double(concat(to_string(size_KiB1), (substring(to_string(size_KiB_split[1]), 0,2))));

let size_MiB_split = split("\\.", to_string(size_MiB)); //split KiB by .
let size_MiB1 = concat(to_string(size_MiB_split[0]), "."); 
let size_MiB = to_double(concat(to_string(size_MiB1), (substring(to_string(size_MiB_split[1]), 0,2))));

let size_GiB_split = split("\\.", to_string(size_GiB)); //split KiB by .
let size_GiB1 = concat(to_string(size_GiB_split[0]), "."); 
let size_GiB = to_double(concat(to_string(size_GiB1), (substring(to_string(size_GiB_split[1]), 0,2))));

let size_TiB_split = split("\\.", to_string(size_TiB)); //split KiB by .
let size_TiB1 = concat(to_string(size_TiB_split[0]), "."); 
let size_TiB = to_double(concat(to_string(size_TiB1), (substring(to_string(size_TiB_split[1]), 0,2))));

//Create and set new fields

set_field("metricbeat_system_filesystem_total_kebibytes", size_KiB);
set_field("metricbeat_system_filesystem_total_mebibytes", size_MiB);
set_field("metricbeat_system_filesystem_total_gebibytes", size_GiB);
set_field("metricbeat_system_filesystem_total_tebibytes", size_TiB);

end

This gives me the result

41312252 (correct in this case) 40343.99 (should be 40344.00) 39.39 (should be 39.40) 0.038 (should be 0.04)

ryanholden8 commented 3 years ago

Would love to see this, app insights has this ability and improved our analytics reporting by making it much more human readable.

drewmiranda-gl commented 1 year ago

This would be very useful to have :)

boosty commented 1 year ago

Is it important for your use cases that a number gets already converted at processing time (and stored like that), or would it also be ok to convert it at display time?

StarMonkey1 commented 1 year ago

Ideally an option to do both.

Displayed at processing time is a nice to have for quick and easy metrics.

Convert and store at prcessing time is absolutely required so we could use the mathematical functions within Dashboards and Pipeline rules.

drewmiranda-gl commented 1 year ago

My thinking, at a minimum, a round function in pipeline rules would be nice.