rsyslog / liblognorm

a fast samples-based log normalization library
http://www.liblognorm.com
GNU Lesser General Public License v2.1
99 stars 64 forks source link

Support for comma-separated numbers? #319

Open saurabhnanda opened 5 years ago

saurabhnanda commented 5 years ago

Is there any way to parse 1,234,567 as a number?

Use-case: I'm trying to parse the output of rsync --stats, which looks like the following:

Number of files: 2,954,826 (reg: 1,178,981, dir: 1,775,845)
Number of created files: 29 (reg: 16, dir: 13)
Number of deleted files: 0
Number of regular files transferred: 25
Total file size: 138,567,893,245 bytes
Total transferred file size: 21,814,234 bytes
Literal data: 3,923,368 bytes
Matched data: 17,890,866 bytes
File list size: 10,947,374
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 1,865,173
Total bytes received: 111,947,536

sent 1,865,173 bytes  received 111,947,536 bytes  1,089,116.83 bytes/sec
total size is 138,567,893,245  speedup is 1,217.51
kyparisisg commented 4 years ago

Hello @saurabhnanda,

I am not sure if that helps but you can probably read the whole number by simply using this %field_name:word% (One or more characters, up to the next space (\x20), or up to end of line.). For more complicated situations, keep in mind that you can also use: %field_name:char-sep:\x20% OR %field_name:char-to:\x20% (\x20 is hex and represents whitespace, it can be replaced with other punctuation characters, decimal point, etc )

Let me know if that helps.

davidelang commented 4 years ago

no, liblognorm v2 doesn't do number conversion, and rsyslog doesn't have a string-to-number() function that could interpret such things (and would probably also need to handle the european equivalent 1.234.567,89 (reverse use of, and . compared to the US)

I think such a function would be useful to have and I would encouage you to file an enhancement request for such a function in rsyslog (and consider sponsoring getting it added if you aren't able to code)

David Lang

On Thu, 4 Apr 2019, Saurabh Nanda wrote:

Is there any way to parse 1,234,567 as a number?

Use-case: I'm trying to parse the output of rsync --stats, which looks like the following:

Number of files: 2,954,826 (reg: 1,178,981, dir: 1,775,845)
Number of created files: 29 (reg: 16, dir: 13)
Number of deleted files: 0
Number of regular files transferred: 25
Total file size: 138,567,893,245 bytes
Total transferred file size: 21,814,234 bytes
Literal data: 3,923,368 bytes
Matched data: 17,890,866 bytes
File list size: 10,947,374
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 1,865,173
Total bytes received: 111,947,536

sent 1,865,173 bytes  received 111,947,536 bytes  1,089,116.83 bytes/sec
total size is 138,567,893,245  speedup is 1,217.51