chop-dbhi / prometheus-sql

Service that exposes Prometheus metrics for a SQL result set.
BSD 2-Clause "Simplified" License
202 stars 55 forks source link

Panic when there is a query execution related error #54

Closed jurgis closed 4 years ago

jurgis commented 4 years ago

If one query fails for whatever reason the application panics and quits.

Sample error output:

panic: a previously registered descriptor with the same fully-qualified name as Desc{fqName: "query_result_test", help: "Result of an SQL query", constLabels: {instance="test"}, variableLabels: []} has different label names or a different help string

goroutine 78 [running]:
app/vendor/github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister(0xc420012540, 0xc42012c180, 0x1, 0x1)
        /go/src/app/vendor/github.com/prometheus/client_golang/prometheus/registry.go:404 +0x9e
app/vendor/github.com/prometheus/client_golang/prometheus.MustRegister(0xc42012c180, 0x1, 0x1)
        /go/src/app/vendor/github.com/prometheus/client_golang/prometheus/registry.go:152 +0x53
main.(*QueryResult).RegisterMetrics.func1(0xc420512240, 0x34, 0x91f280, 0xc420012240)
        /go/src/app/set.go:158 +0x13e
main.(*QueryResult).RegisterMetrics(0xc420250330, 0xc4205140f0)
        /go/src/app/set.go:162 +0x27c
main.(*Worker).SetMetrics(0xc42016db90, 0xc420128100, 0x1, 0x4)
        /go/src/app/worker.go:46 +0xf0
main.(*Worker).Fetch(0xc42016db90, 0x7ffd304a5f36, 0x17, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/src/app/worker.go:117 +0x900
main.(*Worker).Start.func1()
        /go/src/app/worker.go:124 +0x45
main.(*Worker).Start(0xc42016db90, 0x7ffd304a5f36, 0x17)
        /go/src/app/worker.go:131 +0x77
created by main.main
        /go/src/app/main.go:93 +0x70e

I'm working on a patch.

jurgis commented 4 years ago

The error is only when value-on-error is set. For example to see when there was an error for all queries I have following entry: prometheus-sql.yml

defaults:
  query-value-on-error: -1
jurgis commented 4 years ago

Pull request created: https://github.com/chop-dbhi/prometheus-sql/pull/55