search

lzap / pcp-mmvstatsd

Statsd agent for Performance Co-Pilot
GNU General Public License v3.0
4 stars 0 forks source link

Import data directly rather than via MMV #2

Open lzap opened 7 years ago

lzap commented 7 years ago

There is an API to write directly to logs: http://pcp.io/docs/lab.importdata.html

This sounds better than sampling, but there are no golang bindings. Perl prototype could be easy tho, but this should be rather complementary mode.

lzap commented 7 years ago

@natoscott can you give me hint with this one? I very much like the idea of "offline" data collection. I can write a super simple code that will persist all statsd measurements in an (archive) file and then Perl script that will import this into PCP. This could be the default setting for "ad-hoc" monitoring when performance issue appears in our project to let users quickly record "something" we can easily investigate with PCP.

Now, I can see PCP already have some tools to import data (sar2pcp, sheet2pcp), to not reinvent the wheel, I was thinking to store it in some already known (and efficient) format. What would be your recommendation? Thanks

lzap commented 7 years ago

I quickly hacked together a simple statsd seriazer based on MessagePack binary format which I can easily pick up in C or Perl. The PMI API does not seem to be complex, it might be possible to write a Ruby FFI wrapper, not sure how safe is this to do from application context tho.