search

performancecopilot / speed

A Go implementation of the PCP instrumentation API
MIT License
37 stars 6 forks source link

Added pcp-mmvstatsd to README #38

Closed lzap closed 7 years ago

lzap commented 7 years ago

If that makes any sense.

coveralls commented 7 years ago

Coverage Status

Coverage remained the same at 78.037% when pulling 681009d016202f1af76906470c157b0681083a38 on lzap:doco into 36cbb4812a3228baa2501c8e4cd37089466f4f99 on performancecopilot:master.

natoscott commented 7 years ago

@lzap can I suggest the name "speed-daemon" as Speed is the project doing the heavy lifting here? @suyash perhaps consider merging this daemon into Speed itself, if @lzap is open to that option?

Parfait is likely to end up with a daemon mode also, which will also be an MMV-based daemon, but more Java focused of course, so this distinction will be helpful for users I think.

lzap commented 7 years ago

I was not going to propose that as my code is ugly and hacky, also I am not golang expert at all, but if you prefer I'd happily to move it here. I plan to write systemd unit for it and make RPM as I'd like to ship it in the project and then product later on. If you don't mind accepting my PRs here and there as I will be improving this (if the Ruby part is accepted into the project) then go ahead.

I don't have problem with any name at all, whatever works.

suyash commented 7 years ago

@natoscott Looking at the project it seems pretty extensive to be an example. I would favor merging this and adding a minimal example for a daemon mode in examples/ separately. Your call.

natoscott commented 7 years ago

@suyash @lzap I recommend only merging it if its something that is planned to be maintained in the long-term. At this stage it seems like it maybe a bit experimental. OTOH its an interesting development and may open new possibilities for Speed deployments, and we've seen similar requests for Parfait (hence proxying mode now exists in Parfait, and no doubt further work will be done in that area of Parfait over time).

So if it is merged I'd suggest getting it into a shape everyone is comfortable with (noone thinks its quite ready for that just yet), and making it a supported component of Speed rather than just example code (e.g. as a speed/daemon peer of speed/mmvdump in the repo, perhaps). But until then, using separate repos makes most sense I think.

suyash commented 7 years ago

@natoscott I agree. Merging this for now, should probably track creating a daemon mode by creating an issue in the issue tracker.

lzap commented 7 years ago

Cool, let's see how folks from my team like it first, if the integration part gets merged and gets its way into Satellite, I will be maintaining the project including packaging and stuff. My only concern is golang in RHEL7 - I would like to get it into RHEL7.

fche commented 7 years ago

FWIW name toolshedding: "speed-daemon" or calling out this statsd-consumer's dependence on golang into its own name seems to focus too much on small stuff. The nature & value of the statsd consumer is that it turns statsd packets into mmv data, not that it is written in go, nor that it uses the "speed" golang bindings.

@lzap , As to one bit about the tool's behaviour: it could make sense for it to retain information about its network peer, for propagation into the mmv output namespace. Instead of just ignoring where the packet came from, it could pass the address or hostname into the pcp namespace somehow. One way would be to prefix the statsd metrics with that info - another way would be to name the mmv "client" files on a per-network-peer basis.

Conceptually, did you consider making this an archive importer rather than an mmv pmda-flavoured data source? The former model could preserve the timing & content of each statsd packet. The mmv model requires an external tool (pmlogger?) to poll periodically, which by necessity loses timing & individual content. Just wondering whether you considered that.

lzap commented 7 years ago

So you want to have remote IP address (or hostname) as part of the PCP namespace? I can do that, sure. https://github.com/lzap/pcp-mmvstatsd/issues/1

I haven't considered that at all as my knowledge of PCP is almost zero (I just discovered the project week ago). I very much like the idea of retaining all the packets, compared to that sampling is poor's man solution. I understand there are no golang bindings. Perl prototype could be easy tho, but this should be rather complementary mode. I like the possibility of live monitoring (aka top). https://github.com/lzap/pcp-mmvstatsd/issues/2

natoscott commented 7 years ago

@lzap just to clarify - you're gonna have a bad time if you try to put hostnames or IP addresses in the (dot-separated) PCP namespace - see the metric namespace naming requirements in the pmns(4) man page 'SYNTAX' section. Metric instance domains is a much better suited place for that sort of thing.