jeremydaly / serverless-mysql

A module for managing MySQL connections at SERVERLESS scale
MIT License
1.2k stars 83 forks source link

Not working in PM2 Application #76

Closed bpr-acn closed 4 years ago

bpr-acn commented 4 years ago

We have implemented this and is working in two Lambda's but in the ETL code that is running in PM2 I saw it logging 20,000 open connections on a 4 ACU Aurora MySQL Serverless with 270 connections. It seemed like there was 1 connection that was working for a while but each time .query() was called it looked like a new connection with no cleanup. Is there something to run to cleanup connections during a run? Or was this just built for very short lived Lambda's?

jeremydaly commented 4 years ago

Thanks for the note. Connection management should work the same way (i.e. open connections should stay persistent until cleaned up). Are you running the mysql.end() command after each query run? Also, you might want to set the zombie timeout much higher since the likelihood of zombies would be low with a PM2.

Any code samples would be helpful.

bpr-acn commented 4 years ago

Is this what we should be doing? We were not calling .end() after each call. This code is shared with Lambda/PM2 with your proposed change: async query(sql: string, args = [], paginated: boolean = false): Promise<any | any[]> { const queryArgs = paginated ? [...args, ...args] // repeat args for totalCount & nodes : args; const result = await this.mysql.query(sql, queryArgs.filter((arg) => arg !== undefined)); await this.mysql.end(); return camelizeKeys(result); }`

bpr-acn commented 4 years ago

We have also seen decrease in query performance as now each one is making a new connection versus the pool we were previously using. Is there a way to reuse connections within this package?

jeremydaly commented 4 years ago

This looks right. The library is defined for short-lived, ephemeral compute instances, so it doesn't manage connection pooling, but instead creates a new connection when needed. However, it should maintain the connection across requests.

I would suggest using the standard mysql library if connection pooling is required.

bpr-acn commented 4 years ago

Thanks for the information!