Closed Knud3 closed 1 year ago
Tried with different combinations:
test/foo-{0}/total_alloc.wsp
test.*.total_alloc
INFO access request served {"data": {"handler":"find","carbonapi_uuid":"cbfaf034-7681-4e7b-94bc-6622eac34c10","url":"/metrics/find","host":"carbonapi:8081","metrics":["test.*.total_alloc"],"runtime":0.001640143,"http_code":200,"uri":"/metrics/find","from_cache":false,"used_backend_cache":false,"zipper_requests":1,"total_metrics_count":1,"request_headers":{}}}
test.foo-{0}.total_alloc
WARN zipper find request failed when resolving globs {"type": "broadcastGroup", "groupName": "root", "metric_name": "test.foo-{0}.total_alloc", "error": "metric not found"}
INFO access request served {"data": {"handler":"render","carbonapi_uuid":"c4574dd1-f7b3-46f2-b9d3-2f6395aafb03","url":"/render","host":"carbonapi:8081","format":"json","use_cache":true,"targets":["test.foo-{0}.total_alloc"],"cache_timeout":600,"metrics":["test.foo-{0}.total_alloc"],"have_non_fatal_errors":true,"runtime":0.00222413,"http_code":200,"carbonapi_response_size_bytes":2,"from":1676306580,"until":1676328180,"max_data_points":2128,"from_raw":"1676306602","until_raw":"1676328204","uri":"/render","from_cache":false,"used_backend_cache":false,"request_headers":{"X-Panel-Id":"2"}}}
test.foo-\\{0\\}.total_alloc
WARN zipper find request failed when resolving globs {"type": "broadcastGroup", "groupName": "root", "metric_name": "test.foo-\\{0\\}.total_alloc", "error": "metric not found"}
test/foo-\\{0\\}/total_alloc.wsp
test.*.total_alloc
INFO access request served {"data": {"handler":"find","carbonapi_uuid":"73e0486a-0b12-41a7-bed6-590f20b9aa34","url":"/metrics/find","host":"carbonapi:8081","metrics":["test.*.total_alloc"],"runtime":0.001691782,"http_code":200,"uri":"/metrics/find","from_cache":false,"used_backend_cache":false,"zipper_requests":1,"total_metrics_count":1,"request_headers":{}}}
WARN zipper find request failed when resolving globs {"type": "broadcastGroup", "groupName": "go-carbon", "metric_name": "test.foo-\\\\{0\\\\}.total_alloc", "error": "no metrics in the Response"}
ERROR access request failed {"data": {"handler":"render","carbonapi_uuid":"e5019610-f1c7-46e7-904f-780012a120b0","url":"/render","host":"carbonapi:8081","format":"json","use_cache":true,"targets":["test.*.total_alloc"],"cache_timeout":600,"runtime":0.001764256,"http_code":500,"reason":"no metrics in the Response","from":1676305560,"until":1676327160,"from_raw":"1676305601","until_raw":"1676327203","uri":"/render","from_cache":false,"used_backend_cache":false,"request_headers":{"X-Panel-Id":"2"}}}
test.foo-{0}.total_alloc
INFO access request served {"data": {"handler":"render","carbonapi_uuid":"e9c8f3f9-d125-4e86-89fa-41853b10c369","url":"/render","host":"carbonapi:8081","format":"json","use_cache":true,"targets":["test.foo-{0}.total_alloc"],"cache_timeout":600,"metrics":["test.foo-{0}.total_alloc"],"have_non_fatal_errors":true,"runtime":0.002544316,"http_code":200,"carbonapi_response_size_bytes":2,"from":1676305860,"until":1676327460,"max_data_points":2128,"from_raw":"1676305894","until_raw":"1676327496","uri":"/render","from_cache":false,"used_backend_cache":false,"request_headers":{"X-Panel-Id":"2"}}}
INFO access request served {"data": {"handler":"find","carbonapi_uuid":"67f85b0c-0ec4-427f-b97b-de9278489b1d","url":"/metrics/find","host":"carbonapi:8081","metrics":["test.foo-{0}.total_alloc"],"runtime":0.00130966,"http_code":200,"reason":"metric not found","uri":"/metrics/find","from_cache":false,"used_backend_cache":false,"zipper_requests":1,"request_headers":{}}}
test.foo-\\{0\\}.total_alloc
INFO access request served {"data": {"handler":"render","carbonapi_uuid":"2f145320-2189-44a6-b538-ef417c5f6fa1","url":"/render","host":"carbonapi:8081","format":"json","use_cache":true,"targets":["test.foo-\\\\{0\\\\}.total_alloc"],"cache_timeout":600,"metrics":["test.foo-\\\\{0\\\\}.total_alloc"],"have_non_fatal_errors":true,"runtime":0.001477726,"http_code":200,"carbonapi_response_size_bytes":2,"from":1676305980,"until":1676327580,"max_data_points":2128,"from_raw":"1676306022","until_raw":"1676327624","uri":"/render","from_cache":false,"used_backend_cache":false,"request_headers":{"X-Panel-Id":"2"}}}
Also these are from go-carbon
INFO access find success {"handler": "find", "url": "/metrics/find/?format=carbonapi_v3_pb", "carbonzipper_uuid": "2661225a-4a17-43f9-8372-b93d606c7257", "carbonapi_uuid": "2661225a-4a17-43f9-8372-b93d606c7257", "query": ["test.*.total_alloc"], "format": "carbonapi_v3_pb", "runtime_seconds": 0.000056104, "Files": 1, "find_cache_enabled": true, "from_cache": true, "http_code": 200, "lookups": 28}
ERROR access find failed {"handler": "find", "url": "/metrics/find/?format=carbonapi_v3_pb", "carbonapi_uuid": "392061dd-26da-4eb3-a111-0e779d6265b8", "carbonzipper_uuid": "392061dd-26da-4eb3-a111-0e779d6265b8", "query": ["test.foo-{0}.total_alloc"], "format": "carbonapi_v3_pb", "runtime_seconds": 0.000054932, "reason": "Not Found", "error": "Not Found", "http_code": 404}"
ERROR access find failed {"handler": "find", "url": "/metrics/find/?format=carbonapi_v3_pb", "carbonapi_uuid": "2d827c60-60eb-4337-a725-4bced1883725", "carbonzipper_uuid": "2d827c60-60eb-4337-a725-4bced1883725", "query": ["test.foo-\\{0\\}.total_alloc"], "format": "carbonapi_v3_pb", "runtime_seconds": 0.000312228, "reason": "Not Found", "error": "Not Found", "http_code": 404}
Maybe it is because expandGlobBraces
funtion in Go-Carbon carbonserver
? https://github.com/go-graphite/go-carbon/blob/master/carbonserver/carbonserver.go#L1434-L1477
Hi @Knud3,
As far as I remember, curly braces are part of Graphite globs, so they were never allowed characters in metrics, not in go-graphite, neither in python implementation. I'm not sure it it's theoretically possible even - how you distinguish glob from real bracket in filename?
Actually in graphite-web there is a functionality of escaping special characters (and it allows any kind of characters, excluding dots in the name as far as I remember) which is semi-working in carbonapi, but can be in non-working state in go-carbon and graphite-clickhouse (I've never checked to be honest)
More I've looked into it, more it's started to look like you said. Not sure why, but it is working in render when maxBatchSize
is set to 0
and curly brackets are replaced with asterisk in query. e.g.:
filename: test/foo-{0}/total_alloc.wsp
series query: test.foo-*.total_alloc
I'm no Grafana user myself, but trying to provide platform to our devs. So, I was no familiar with globs before this.
That use case comes from our dev.
HTTP clients and servers that use path parameters often represent these path parameters as
{variable}
, e.g./reports/{date}
What I know now, I try to shift them to use something else than curly brackets, because I'd like to set maxBatchSize
for decreased render times.
Thank you for pointing it out! If you do not see easy workaround, I'm ok to close this issue.
Difference is that with maxBatchSize=0 - carbonapi sends request as-is to backend ("give me metrics"). With maxBatchSize > 0 - it first sends find query and then iterates over result to split one big request into smaller one that are not more than maxBatchSize metrics per request.
Basically that means that backend have a fallback path for /render request that tries to fetch data anyhow, but with /metrics/find it tries to expand it and if that fails - returns nothing.
If not 0, carbonapi will do find request to determine how many metrics matches criteria and only then will fetch them, not more than maxBatchSize per request.
Not sure if it is find request or does
maxBatchSize
do something else for render request.Maybe related to #29 and/or #746 and/or #661.
Have tried with
Tried with escaping curly brackets in directory names
Bad Request: Bad Request Target : foo.bar\{topic\}.* Parsed so far : foo.bar\{topic\ Could not parse : }.*
HTTP clients and servers that use path parameters often represent these path parameters as
{variable}
, e.g./reports/{date}
. It would be great to be able to use this same format in the graphs to refer to these URL endpoints.Settings in use with newest
v0.16.0-patch2
and go-carbonv0.16.2
: