search

google-code-export / ords

Automatically exported from code.google.com/p/ords
1 stars 0 forks source link

Too many connections to ORDS crashes system #518

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Have lots of open connections (these still seem to build up over time)
2. Try to view/edit/query a db
3. Get the following error message:

HTTP Status 500 - javax.servlet.ServletException: 
org.postgresql.util.PSQLException: FATAL: sorry, too many clients already
________________________________________
type Exception report
message javax.servlet.ServletException: org.postgresql.util.PSQLException: 
FATAL: sorry, too many clients already
description The server encountered an internal error 
(javax.servlet.ServletException: org.postgresql.util.PSQLException: FATAL: 
sorry, too many clients already) that prevented it from fulfilling this request.
exception 
org.apache.jasper.JasperException: javax.servlet.ServletException: 
org.postgresql.util.PSQLException: FATAL: sorry, too many clients already
        org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:549)
        org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:455)
        org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
        org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
        uk.ac.ox.oucs.ords.servlets.filters.ParamFilter.doFilter(ParamFilter.java:24)
root cause 
javax.servlet.ServletException: org.postgresql.util.PSQLException: FATAL: 
sorry, too many clients already
        org.apache.jasper.runtime.PageContextImpl.doHandlePageException(PageContextImpl.java:911)
        org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:840)
        org.apache.jsp.queryBuilder_jsp._jspService(queryBuilder_jsp.java:1230)
        org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
        org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
        org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
        org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
        uk.ac.ox.oucs.ords.servlets.filters.ParamFilter.doFilter(ParamFilter.java:24)
root cause 
org.postgresql.util.PSQLException: FATAL: sorry, too many clients already
        org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:293)
        org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
        org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
        org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
        org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
        org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
        org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32)
        org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
        org.postgresql.Driver.makeConnection(Driver.java:393)
        org.postgresql.Driver.connect(Driver.java:267)
        java.sql.DriverManager.getConnection(DriverManager.java:571)
        java.sql.DriverManager.getConnection(DriverManager.java:187)
        uk.ac.ox.oucs.ords.database.QueryRunner.initialiseConnection(QueryRunner.java:132)
        uk.ac.ox.oucs.ords.database.QueryRunner.<init>(QueryRunner.java:80)
        uk.ac.ox.oucs.ords.ordsProjectNode.utilities.database.DBUtils.<init>(DBUtils.java:77)
        uk.ac.ox.oucs.ords.ordsProjectNode.utilities.database.PostgresDBUtils.<init>(PostgresDBUtils.java:68)
        org.apache.jsp.queryBuilder_jsp._jspService(queryBuilder_jsp.java:384)
        org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
        org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
        org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
        org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
        uk.ac.ox.oucs.ords.servlets.filters.ParamFilter.doFilter(ParamFilter.java:24)
note The full stack trace of the root cause is available in the Apache 
Tomcat/7.0.28 logs.

What is the expected output? What do you see instead?
Ideally we shouldn't be getting this error message at all. If we can't 
completely avoid it we should at least be catching it and giving a 
human-readable error (and ideally an action they can take to alert us that it 
is happening).

Please use labels and text to provide additional information.

This is occurring on App 1.0.1 - 3485

Original issue reported on code.google.com by jajwil...@gmail.com on 11 Aug 2014 at 1:46

GoogleCodeExporter commented 9 years ago

Original comment by jajwil...@gmail.com on 11 Aug 2014 at 1:47

GoogleCodeExporter commented 9 years ago 
Just working with the Migrations Project team and it crashed after performing 
about 20 queries. Wasn't doing much else so it definitely appears that querying 
is leaving connections open. This is going to hit every researchers using the 
ORDS sooner rather than later, so raising to critical.

Original comment by jajwil...@gmail.com on 27 Aug 2014 at 10:46

GoogleCodeExporter commented 9 years ago

Original comment by thest...@gmail.com on 4 Sep 2014 at 7:54

GoogleCodeExporter commented 9 years ago 
I have added more defensive code around the db code that will hopefully fix 
this problem. Obviously re-raise as critical if the problem persists.

Original comment by thest...@gmail.com on 8 Sep 2014 at 7:21