PostgreSQL Error: sorry, too many clients already - while testing

[dalu93]

Hi all,

In our project, we are executing endpoint tests: when running them, the memory usage keeps increasing, allocated thread's number grows and PostgreSQL raises the exception sorry, too many clients already.

The code we are using is quite simple and is taken from here: https://github.com/raywenderlich/vapor-til/tree/master/Tests/AppTests

To let you identify the issue simply, I ran the project from Tim (the one in the link above) and I noticed the same exact issue, even if the restricted amount of tests doesn't raise the exception in Postgre. In addition, the code above is calling the method app.shoutdownGracefully() which is not deallocating the app instance, while replacing it with the assignment app = nil does it.

So, this is our BaseTestCase

class BaseTestCase: XCTestCase {

    var app: Application!
    var conn: PostgreSQLConnection!

    override func setUp() {
        try! Application.reset()
        app = try! Application.testable()
        conn = try! app.newConnection(to: .psql).wait()
    }

    override func tearDown() {
        super.tearDown()
        conn.close()
        app = nil
    }
}

And this is our Application+testable.swift

import Vapor
import Backend

extension Application {
    static func testable(envArgs: [String]? = nil) throws -> Application {
        var config = Config.default()
        var services = Services.default()
        var env = Environment.testing

        if let environmentArgs = envArgs {
            env.arguments = environmentArgs
        }

        try Backend.configure(&config, &env, &services)
        let app = try Application(config: config, environment: env, services: services)

        return app
    }

    static func reset() throws {
        let revertEnvironment = ["vapor", "revert", "--all", "-y"]
        var app: Application! = try Application.testable(envArgs: revertEnvironment)
        try app.run()
        app = nil

        let migrateEnvironment = ["vapor", "migrate", "-y"]
        app = try Application.testable(envArgs: migrateEnvironment)
        try app.run()
        app = nil
    }
}

As you can see in the screenshots below, the memory keeps increasing and the number of allocated threads is huge.

Is there any way how to overcome this issue? We would really keep connecting to real database while testing to verify the connection to the db server.

[zdnk]

also tested with

    override func setUp() {
        super.setUp()

        try! Application.reset()
        app = try! Application.testable()
        conn = try! app.newConnection(to: .psql).wait()
    }

    override func tearDown() {
        conn.close()
        try! app.syncShutdownGracefully()

        super.tearDown()
    }

[zdnk]

Above plus this:

    static func reset() throws {
        let revertEnvironment = ["vapor", "revert", "--all", "-y"]
        let app1 = try Application.testable(envArgs: revertEnvironment)
        try app1.asyncRun().wait()
        try app1.syncShutdownGracefully()
        let migrateEnvironment = ["vapor", "migrate", "-y"]
        let app2 = try Application.testable(envArgs: migrateEnvironment)
        try app2.asyncRun().wait()
        try app2.syncShutdownGracefully()
    }

Seems to get rid of the too many clients issue. However there is still a lot of spawned threads and memory keeps increasing linearly during tests execeution.