cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.09k stars 3.8k forks source link

sql: pgweb fails with error 500 upon inspecting crdb_internal.node_statement_statistics #33199

Closed knz closed 5 years ago

knz commented 5 years ago

Note: this may not be a problem inside CockroachDB and instead a bug in pgweb. Unsure.

However since pgweb is rather consistently good at viewing CockroachDB we should do moderate effort at ensuring the experience remains smooth.

Describe the problem

When (trying to) inspecting crdb_internal.node_statement_statistics the inspection fails and pgweb returns an internal error 500 instead, with the following stack trace:

Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: nl,en-US;q=0.7,en;q=0.3
Connection: keep-alive
Cookie: G_ENABLED_IDPS=google; id-token=eyJhbGciOiJSUzI1NiIsImtpZCI6IjhkN2JmNzIxODgzMjA0N2RlYTNmNzQwMTZmZTQ1ZmQwZDlkNDJhMjkiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJhY2NvdW50cy5nb29nbGUuY29tIiwiYXpwIjoiNDMxODgxNjE1NDEyLTY3NTg2OGpkZHB1aXZ0NGRpOHBtcW5tN2syYnFtMmpsLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwiYXVkIjoiNDMxODgxNjE1NDEyLTY3NTg2OGpkZHB1aXZ0NGRpOHBtcW5tN2syYnFtMmpsLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwic3ViIjoiMTE1MTI2MTA4NjgwMzYyMzM0OTQ0IiwiaGQiOiJjb2Nrcm9hY2hsYWJzLmNvbSIsImVtYWlsIjoia256QGNvY2tyb2FjaGxhYnMuY29tIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsImF0X2hhc2giOiJCV3BrbHpZVjZ4MElmendNV2ljYnp3IiwibmFtZSI6IlJhcGhhZWwgJ2tlbmEnIFBvc3MiLCJwaWN0dXJlIjoiaHR0cHM6Ly9saDYuZ29vZ2xldXNlcmNvbnRlbnQuY29tLy1TWG83M0xUOXZDNC9BQUFBQUFBQUFBSS9BQUFBQUFBQUFEby9QbGg2REtWQjlhNC9zOTYtYy9waG90by5qcGciLCJnaXZlbl9uYW1lIjoiUmFwaGFlbCAna2VuYSciLCJmYW1pbHlfbmFtZSI6IlBvc3MiLCJsb2NhbGUiOiJlbiIsImlhdCI6MTU0NDg5NTEyMywiZXhwIjoxNTQ0ODk4NzIzLCJqdGkiOiJkYjcwMTJhMjk1MTBmM2I3YjZhMTI5Y2ZlOGQyYzlhNjI4MWY5Zjc5In0.jcejmoiZZcSRlKQJ1UAWlSM98Qu8r25mkt_eFXHPtUgqPQTD4TS_CNO_fXI8ynmAMicw99MCw19CpqQAvOmm1hGN7cCuZ2BUlgNWzWcE2DAS3ns6q6XYrupm6GNiR3ulb4ZWAzqBicRVr2QfKUptJ2ZhP_MWKsCHK8HS2kqkXYxOr7UK4O_FGM__B9qIhYNev8Z2iVqxhcJr2s9Otx7tXrNC2ubVyPmgJIDE0qDTvESemGJVaYkNzCoKHCunYP4UIS8TrkZMsvWHSo3Sr6dC5LIekevEXlJ1ideG7-fWEtIIY_aqhwKger4MTwAVBVoX0nLHSgxBBCdAc9zT1U_SFA
Dnt: 1
Referer: http://localhost:7648/
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0
X-Requested-With: XMLHttpRequest
X-Session-Id: f58be225-b79f-c8ad-f2e2-3dd65d9d3316

json: unsupported value: NaN
/usr/local/go/src/runtime/panic.go:513 (0x42c2e8)
        gopanic: reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/render/json.go:46 (0x8cf9df)
        JSON.Render: panic(err)
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/context.go:713 (0x8d79b2)
        (*Context).Render: if err := r.Render(c.Writer); err != nil {
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/context.go:756 (0x8d7d7a)
        (*Context).JSON: c.Render(code, render.JSON{Data: obj})
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/context.go:756 (0x8d7d7a)
        (*Context).JSON: c.Render(code, render.JSON{Data: obj})
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/pkg/api/middleware.go:77 (0x9b2117)
        serveResult: c.JSON(200, result)
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/pkg/api/api.go:364 (0x9af75b)
        GetTableRows: serveResult(res, err, c)
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/context.go:108 (0x8d46c2)
        (*Context).Next: c.handlers[c.index](c)
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/pkg/api/middleware.go:31 (0x9b36cb)
        dbCheckMiddleware.func1: c.Next()
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/context.go:108 (0x8d46c2)
        (*Context).Next: c.handlers[c.index](c)
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/recovery.go:48 (0x8e5a99)
        RecoveryWithWriter.func1: c.Next()
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/context.go:108 (0x8d46c2)
        (*Context).Next: c.handlers[c.index](c)
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/logger.go:84 (0x8e4da1)
        LoggerWithWriter.func1: c.Next()
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/context.go:108 (0x8d46c2)
        (*Context).Next: c.handlers[c.index](c)
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/gin.go:361 (0x8dce6a)
        (*Engine).handleHTTPRequest: c.Next()
/data/home/kena/src/go/src/github.com/sosedoff/pgweb/vendor/github.com/gin-gonic/gin/gin.go:326 (0x8dc691)
        (*Engine).ServeHTTP: engine.handleHTTPRequest(c)
/usr/local/go/src/net/http/server.go:2741 (0x69366a)
        serverHandler.ServeHTTP: handler.ServeHTTP(rw, req)
/usr/local/go/src/net/http/server.go:1847 (0x68f8c5)
        (*conn).serve: serverHandler{c.server}.ServeHTTP(w, w.req)
/usr/local/go/src/runtime/asm_amd64.s:1333 (0x459010)
        goexit: BYTE    $0x90   // NOP

To Reproduce

  1. start crdb server
  2. start pgweb
  3. navigate to the internal table

(nothing happens in the browser)

  1. check the pgweb console, observe stack trace

Expected behavior

either a successful visualization or a better error message.

knz commented 5 years ago

cc @mjibson thoughts? Is this a symptom of pgweb not supporting NaNs when it should, or is this a case of CockroachDB stuffing an incompatible value (NaN) in a type where clients don't expect it?

(In either case perhaps we can move the issue forward by replacing NaNs with NULLs for the internal table)

maddyblue commented 5 years ago

show create table crdb_internal.node_statement_statistics indicates these columns are all floats, so NaN is a valid value here, suggesting this is a pgweb bug. Presumably we are returning NaN there because you can't have variance with only one value. I'm not convinced NULL is a better choice than NaN, from a math perspective.

knz commented 5 years ago

I see thanks. I'll try a regular table with a nan in it and see what it says.

knz commented 5 years ago

I have verified that pgweb indeed fails with a 500 error on regular tables containing a float column with a NaN value. So this is non-actionable.