hoytech / vmtouch

Portable file system cache diagnostics and control
https://hoytech.com/vmtouch/
BSD 3-Clause "New" or "Revised" License
1.79k stars 210 forks source link

Machine readable output kvp-style #91

Closed rdhatt closed 3 years ago

rdhatt commented 3 years ago

I have the desire to track file cache diagnostics across many machines on regular basis, so having machine-friendly output is a must. My logging system can parse either KVP or JSON records.

My first attempt used sed and awk. KVP is easy to read/write with these tools.

$ vmtouch /home/foo | sed -r 's/^  Resident Pages: ([0-9]+)\/([0-9]+)\s\s([0-9\/]+[KMG]?)\/([0-9\/]+[KMG])\s\s([0-9]+)%/   ResidentPages: \1\n      TotalPages: \2\n    ResidentSize: \3\n       TotalSize: \4\n ResidentPercent: \5/g' | sed 's/ seconds//' | awk 'BEGIN { FS = ": "; OFS="="; ORS=" ";} { sub(/^[ \s]+/, ""); print $1,$2}'; echo
Files=37 Directories=1 ResidentPages=17885956 TotalPages=17885956 ResidentSize=68G TotalSize=68G ResidentPercent=100 Elapsed=0.01825

This gets me most of what I want however the size units are a pain to deal with. I would need to figure out how to get numfmt/units involved or rewrite in perl or the like. This was getting complicated quickly.

I realized just adding my own printf inside vmtouch would be much simpler, so here we are. I realize JSON is the trending format these days, however I just stuck with kvp since it's simpler and it works for me.

Example output produced by -k switch:

Files=37 Directories=1 ResidentPages=17885956 TotalPages=17885956 ResidentSize=73014444032 TotalSize=73014444032 ResidentPercent=100 Elapsed=0.01825

This addresses the first item in TODO 🙂.

hoytech commented 3 years ago

Thank you for this! Just for future compatibility, what are your thoughts on making different output modes? For example, if we used -o then your output could be -o kv. That way if somebody did want to add JSON output they could do -o json. This way we wouldn't have to use up a letter for each output mode, and wouldn't have to worry about what happens when a user specifies multiple output modes at the same time.

rdhatt commented 3 years ago

I was taking the "do the simplest thing" approach , especially since I haven't written C since the last millennium. 😄

I think I managed to do what you asked, let me know what you think. Also remembered to update the help text this time.

hoytech commented 3 years ago

I found a couple minor issues, like that it fails if you don't specify "-o". But I'll fix them up in a subsequent commit. Thank you!