go-graphite / carbonzipper

proxy to transparently merge graphite carbon backends
Other
104 stars 29 forks source link

format=pickle returns empty data set #29

Closed reyjrar closed 6 years ago

reyjrar commented 7 years ago

Pointing graphite-web at carbonzipper via CLUSTER_SERVERS in local_settings.py

2017/01/26 20:53:06 request:  /metrics/find/?local=1&format=pickle&query=%2A
2017/01/26 20:53:06 querying servers= [http://10.20.14.4:8080 http://10.20.28.2:8080] uri= /metrics/find/?format=protobuf&local=1&query=%2A

Receiving an empty response and no debugging information. If I manually query the carbonserver instances for format={pickle,protobuf,json} I get valid data.

If I change:

2017/01/26 20:53:06 request:  /metrics/find/?local=1&format=pickle&query=%2A

TO

2017/01/26 20:53:06 request:  /metrics/find/?local=1&format=json&query=%2A

I get data.

I just pulled the latest master and built using golang 1.7.4 from golang.org.

dgryski commented 7 years ago

Weird. It should handle this case. I'll investigate this tomorrow.

dgryski commented 7 years ago

When I query carbonzipper with /metrics/find/?local=1&format=pickle&query=%2, I do get data back. Maybe look at the responses that are coming back from carbonserver via wireshark (or enable debug logging with -d=2). However, any errors should be logged, and I don't see those in what your pasted above.

reyjrar commented 7 years ago

Testing with -d=2 now.

reyjrar commented 7 years ago

This is the only information I get in the logs for the graphite-web tree requests when setting the carbonapi server as the graphite-web cluster server:

2017/02/01 01:43:04 request:  /metrics/find/?format=protobuf&query=%2A
2017/02/01 01:43:04 querying servers= [http://10.20.14.4:8080 http://10.20.28.2:8080] uri= /metrics/find/?format=protobuf&query=%2A

So this is [frontend node: graphite-web -> carbonapi -> carbonzipper] -> [storage nodes: carbonserver ]

Civil commented 7 years ago

Can you try with 0.70+ and with debug loglevel?

reyjrar commented 7 years ago

OK, will rebuild and test

Civil commented 7 years ago

Have you tried with 0.80+?

Civil commented 6 years ago

no response for almost a year, I will assume that it's fixed now. Please reopen if it's not.