search

ryandotsmith / l2met

Convert a formatted log stream into metrics
http://r.32k.io/l2met-introduction
GNU General Public License v3.0
284 stars 26 forks source link

Allow router source to be manually specified/include summation metric #96

Closed BRMatt closed 11 years ago

BRMatt commented 11 years ago

Each of our users has their own subdomain in our multitenant app, and each router metric creates a source for each of these domains. While it's nice to be able to compare traffic between different sites, having a large number of sources for each metric prevents us from getting a single overview metric which we can evaluate/act on.

It'd be handy to have a separate source that amalgamates all of the router measurements. As it's useful to have stats for individual sites, it probably isn't necessary to stop l2met creating extra sources per domain. Maybe make it a configuration flag to help keep costs down? All it needs is for someone to send 1 request per minute to a few thousand subdomains and you'll have racked up a fair bill.

ryandotsmith commented 11 years ago

Great issue. Allow me to disclose that I hadn't really thought of the ramifications of per-domain sourcing on router metrics. I would definitely be up for picking something that is more reasonable.

ryandotsmith commented 11 years ago

If we wanted to remove this "feature", we could do so by removing this line. I am not sure how many people depend on this. I suspect it is mostly annoying.

BRMatt commented 11 years ago

It's useful if you want to see which site is producing the most traffic, but if we were to allow the specification of source in log lines it'd be trivial to write a small app specific metric that'd recreate the feature with safe guards(e.g. only log for sites that exist).

ryandotsmith commented 11 years ago

Are you aware that you can specify the source in the log lines? Checkout the wiki for the details.

BRMatt commented 11 years ago

Whoops, no I must've missed that! In that case this should definitely be removed On 8 Aug 2013 02:19, "Ryan Smith" notifications@github.com wrote:

Are you aware that you can specify the source in the log lines? Checkout the wiki https://github.com/ryandotsmith/l2met/wiki/Usage#source for the details.

— Reply to this email directly or view it on GitHubhttps://github.com/ryandotsmith/l2met/issues/96#issuecomment-22296408 .

aseemk commented 10 years ago

Hey guys, I just came across this, after I'd just asked @josephruscio about this. We miss seeing the hostname as part of the source in our router metrics.

Our use case is that we serve both an HTML website and a JSON API from the same Heroku app (using vhosting), and it'd be great to differentiate stats between the two.

I'd appreciate if this could be reconsidered, as something that could be configurable. (Ideally, via the query string rather than a server config, so we could continue to use Librato's hosted l2met service.)

aseemk commented 10 years ago

Addendum: after reading https://github.com/ryandotsmith/l2met/issues/107#issuecomment-22908928 ("We should modify the Heroku router to read [dyno=web.X] and use it as the source"), I'm no longer sure how the host could/would get incorporated into the source. Tricky.