search

aslakknutsen / almighty-performance

Apache License 2.0
0 stars 1 forks source link

pq complains about too many clients #2

Open hectorj2f opened 7 years ago

hectorj2f commented 7 years ago 
INFO[2017-05-29 17:09:45] completed                                     action=List bytes=124 ctrl=WorkitemtypeController req_id=TKPMk5eZBW-23886 status=500 time=478.13588ms
INFO[2017-05-29 17:09:45] an error occurred in our api                  err="pq: sorry, too many clients already" error_message="pq: sorry, too many clients already" pkg=jsonapi
ERRO[2017-05-29 17:09:45] uncaught error detected in ErrorHandler       err="pq: sorry, too many clients already\ngithub.com/almighty/almighty-core/application.Transactional\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/application/transaction.go:18\ngithub.com/almighty/almighty-core/controller.(*WorkitemtypeController).List\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/controller/workitemtype.go:147\ngithub.com/almighty/almighty-core/app.MountWorkitemtypeController.func2\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/app/controllers.go:3805\ngithub.com/almighty/almighty-core/app.handleWorkitemtypeOrigin.func1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/app/controllers.go:3868\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa.(*Controller).MuxHandler.func1.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/service.go:254\ngithub.com/almighty/almighty-core/space/authz.InjectAuthzService.func1.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/space/authz/authz.go:197\ngithub.com/almighty/almighty-core/login.InjectTokenManager.func1.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/login/service.go:859\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware.Recover.func1.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware/recover.go:37\ngithub.com/almighty/almighty-core/jsonapi.ErrorHandler.func1.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/jsonapi/error_handler.go:37\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware/gzip.Middleware.func2.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware/gzip/middleware.go:98\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware.LogRequest.func1.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware/log_request.go:75\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware.RequestIDWithHeaderAndLengthLimit.func1.1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/middleware/request_id.go:63\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa.(*Controller).MuxHandler.func1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/service.go:287\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa.(*mux).Handle.func1\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/mux.go:59\ngithub.com/almighty/almighty-core/vendor/github.com/dimfeld/httptreemux.(*TreeMux).ServeHTTP\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/dimfeld/httptreemux/router.go:200\ngithub.com/almighty/almighty-core/vendor/github.com/goadesign/goa.(*mux).ServeHTTP\n\t/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/vendor/github.com/goadesign/goa/mux.go:85\nnet/http.(*ServeMux).ServeHTTP\n\t/home/hector/.gvm/gos/go1.8/src/net/http/server.go:2238\nnet/http.serverHandler.ServeHTTP\n\t/home/hector/.gvm/gos/go1.8/src/net/http/server.go:2568\nnet/http.(*conn).serve\n\t/home/hector/.gvm/gos/go1.8/src/net/http/server.go:1825\nruntime.goexit\n\t/home/hector/.gvm/gos/go1.8/src/runtime/asm_amd64.s:2197" file="/home/hector/rh_repo/gopath/src/github.com/almighty/almighty-core/jsonapi/error_handler.go" func="github.com/almighty/almighty-core/jsonapi.ErrorHandler.func1.1" line=66 msg=&{[0xc421503f40]} pid=6415 pkg=jsonapi.ErrorHandler.func1 req_id=TKPMk5eZBW-23863
hectorj2f commented 7 years ago

I am checking why I get this error from pq when I apply some high workloads

hectorj2f commented 7 years ago

FYI: @aslakknutsen I set our ENV vars MAX_OPEN and MAX_IDLE and nothing changes. However if I set 

            db.DB().SetConnMaxLifetime(0)
            db.DB().SetMaxOpenConns(90)

it easily handles the workload