Running carbonzipper with carbonserver on custom ports not working ?

[szibis]

Carbonzipper and carbonserver from latest master.

I would like to test this setup next to standard graphite-web setup, but i have some problems.

2015/08/30 13:27:14 starting carbonzipper (development version)
2015/08/30 13:27:14 setting GOMAXPROCS= 2
2015/08/30 13:27:14 Using graphite host loadbalancer-graphite-instances:2013
2015/08/30 13:27:14 listening on :8080
2015/08/30 13:27:14 singleGet: error querying  10.1.16.128:8080 / /metrics/find/?format=protobuf&query=%2A : unsupported protocol scheme ""
2015/08/30 13:33:04 singleGet: error querying  10.1.16.128:8080 / /render/?format=protobuf&from=1440940588&now=1440941188&target=diamond.fluentd.%2A.%2A.processcpu.fluentd&until=1440941188 : unsupported protocol scheme ""
2015/08/30 13:33:04 render: error querying backends for: /render/?target=diamond.fluentd.*.*.processcpu.fluentd&format=pickle&from=1440940588&until=1440941188&now=1440941188 backends: [10.1.16.128:8080]

Config for carbonzipper running on graphite-web/api instances:

{
  "Backends": [
    "10.1.16.128:8080"
  ],
  "GraphiteHost": "loadbalancer-graphite-instances:2013",
  "MaxProcs": 2,
  "Port":     8080,
  "Buckets":  10,
  "TimeoutMs": 10000,
  "TimeoutMsAfterAllStarted": 2000,

  "MaxIdleConnsPerHost": 100
}

Communication is working to carbonserver from carbonzipper hosts:

telnet 10.1.16.128 8080
Trying 10.1.16.128...
Connected to 10.1.16.128.
Escape character is '^]'.

And nothing in carbonserver logs:

2015/08/30 13:12:21 starting carbonserver (development build)
2015/08/30 13:12:21 reading whisper files from: /opt/graphite/storage/whisper
2015/08/30 13:12:21 maximum brace expansion set to: 10
2015/08/30 13:12:21 set GOMAXPROCS=2
2015/08/30 13:12:21 listening on :8080 

Carbonserver running with such parameters on graphite instances with whipser files - different instances then carbonzipper:

/usr/bin/carbonserver -logdir=/var/log/carbonserver/ -maxexpand=10 -maxprocs=2 -p=8080 -scanfreq=0 -w=/opt/graphite/storage/whisper -stdout=true -vv=true

From what i was able to get from code, configs should looks like i describe. What is wrong with that setup ? Any suggestions ?

[dgryski]

Need actually to prefix the URLS with http://. Docs are lacking. Sorry :(

[gmlwall]

Hello, ive been using carbon-zipper for almost a week now, taking a mirror or all prod traffic, and it looks to be working very well, so many thanks :) next week ill start trying to break things.

anyway, as per these comments, would it be possible to get a list of arguments that the app takes?

ive only used:

-c=./zipper.json -p=8080 -stdout -d=3

even if its just a quick list it would be a great help to me, many thanks.

[szibis]

@dgryski thank, yes with http it is working now. Using curl all looks to work well, but i have no time to check if graphite-web will work with single local metrics endpoint.

[dgryski]

The only command line flag you're missing is -logdir, which sets the output directory for the log files.

Also interesting are the values available to be set in the config file, which I've annotated with some comments:

var Config = struct {
    Backends []string
    MaxProcs int   // GOMAXPROCS
    Port     int        // port to listen on
    Buckets  int    // number of buckets for timings aggregations

    TimeoutMs                int        // timeout to wait for a response from the backend
    TimeoutMsAfterAllStarted int  // timeout to wait after all goroutines to the remote backends for a single metric have started

    GraphiteHost string  // the destination host to send graphite metrics to

    MaxIdleConnsPerHost int // number of idle connections to keep open to carbon backends

    ConcurrencyLimitPerServer int  // maximum number of concurrent requests for a single server

[dgryski]

@szibis graphite does work with a single local backend ,but it needs to be patched to accept localhost. You need the tiny patch listed on http://www.iamsysadmin.ninja/2014/12/graphite-scaling-and-my-evaluation-of.html

[dgryski]

Closing. But opening another bug to improve documentation.