MatteoGioioso / serverless-pg

A package for managing PostgreSQL connections at SERVERLESS scale
MIT License
321 stars 16 forks source link

Client has encountered a connection error and is not queryable #75

Closed ddobryakredknee closed 2 years ago

ddobryakredknee commented 2 years ago

We are having error: Client has encountered a connection error and is not queryable

There is a suggestion to set idleTimeoutMillis: 0 https://github.com/brianc/node-postgres/issues/2512

How to do that using serverless-pg?

MatteoGioioso commented 2 years ago

I think that option is only for Pool which this library is not using. Can you post a code snippet of what you are doing?

ddobryakredknee commented 2 years ago
import QueryStream from 'pg-query-stream'
import ServerlessClient from 'serverless-postgres'
import { Readable } from 'stream'

// some imports of my classes

const postgres = new ServerlessClient({
  connUtilization: DEFAULT_ALLOWED_MAX_CONNECTIONS_PERCENT / 100.0,
  minConnectionIdleTimeSec: 0
})

export class RdsPostgreService extends RdsService {
  private constructor (params: ActionInputParams) {
    super()

    postgres.setConfig({
      host: params.host,
      database: params.dbInstanceIdentifier,
      user: params.username,
      password: params.password,
      port: params.port,
      statement_timeout: params.maxExecutionTime
    })
  }

  public static newInstance (params: ActionInputParams): RdsPostgreService {
    return new RdsPostgreService(params)
  }

  protected async * runQuery (paramQuery: QueryParameters): AsyncGenerator<unknown> {
    try {
      await postgres.connect()
      const dataStream: Readable = await postgres.query(new QueryStream(paramQuery.query, paramQuery.parameters))
      for await (const item of dataStream) {
        yield { item }
      }
    } finally {
      await postgres.clean()
    }
  }
}

Then I am calling runQuery to get the result

ddobryakredknee commented 2 years ago

also I am using "serverless-postgres": "1.9.x",

MatteoGioioso commented 2 years ago

How are you using runQuery? clean is supposed to be used only one time, I am not sure how it will behave if used multiple times, ex:

const result = await runQuery("...")

await runQuery("...")
ddobryakredknee commented 2 years ago

@MatteoGioioso should it be executed one time per lambda execution? I am asking because it is possible that the same image will be reused accross multiple lambda executions. Maybe it make sense to not call it at all?

MatteoGioioso commented 2 years ago

It .connect and .clean should be used one time per lambda execution, at least that's how I design it to work. Maybe is not really clear in the docs.

ddobryakredknee commented 2 years ago

most probably this is the case. I will try to change the code accordingly