juanjoDiaz / serverless-plugin-warmup

Keep your lambdas warm during winter. ♨
MIT License
1.11k stars 114 forks source link

Getting 500 error due to no mongo connection #42

Closed pizzarob closed 7 years ago

pizzarob commented 7 years ago

When using this plugin for some reason it breaks my database connection. Any ideas why?

{"error":{"name":"MongoError","message":"no connection available"}}

I use the following function to connect to mongo before running my lambda functions.

const mongoose = require('mongoose');

mongoose.Promise = global.Promise;
let cachedDB = null;

module.exports = fn => (...args) => {
  const [, context] = args;

  context.callbackWaitsForEmptyEventLoop = false;

  if (cachedDB && cachedDB.readyState != 0 && cachedDB.readyState != 3) {
    fn(...args);
  } else {
    mongoose.connect(process.env.MONGO_URI);

    mongoose.connection.on('error', err => {
      console.log('Connection Error');
      console.log(err);
    });

    mongoose.connection.once('open', () => {
      cachedDB = mongoose.connection;
      fn(...args);
    });
  }
};
juanjoDiaz commented 7 years ago

Not sure what your issue is. But it's good practice to keep the handler as small as possible and move the logic to external classes.

The documentation used to state that db connections, etc. should be declared out of the handler scope as singletons.

I do that with few mongo projects and works just fine :)

pizzarob commented 7 years ago

I’m not sure what you mean. This is a higher order function that allows the database connection to be kept open across lambda executions. @juanjoDiaz. It guarantees a connected database before running code that is dependent on the connection.

goncaloneves commented 7 years ago

I don't see how this has any relation to the warmup plugin.

@realseanp how can you guarantee a permanent connection if you are using a lambda function? That connection will close once the container is dropped ~15 minutes on AWS or if you do it inside the handler function after your defined execution timeout, to a max of 300 seconds.

What @juanjoDiaz suggested is what you should do to avoid re-establishing DB connection every time your lambda executes and only once on container initialisation.

In the meantime I will close this. If you bring an assertion to what is the issue on the warmup plugin, feel free to bring it back. Until then you should rethink your code logic within a serverless architecture.

pizzarob commented 7 years ago

@goncaloneves @juanjoDiaz any Node.js variable that is declared outside the handler method remains initialized across calls, as long as the same container is reused. I am caching the DB connection outside of the handler. I am also utilizing context.callbackWaitsForEmptyEventLoop = false to request AWS Lambda to freeze the process soon after the callback is called, even if there are events in the event loop (open db connection). So if subsequent calls to the lamba functions are made before the container is dropped the connection will be re-used. @goncaloneves if you look closely at the code you will see I am not re-establishing connection everytime, only once. I don't see how I should "rethink [my] code logic within a serverless architecture." when there's not a better way to handle the situation.

goncaloneves commented 7 years ago

This feels like an AWS support ticket. 🤔 😅

From what you shared and what @juanjoDiaz stated, you should call mongoose.connect(process.env.MONGO_URI); in the global scope and not call it inside a function. As you did it, your connection exists inside your function scope memory. This is not specific to lambda. Database connections should always be declared in the highest scope.

Helpers to improve code logic: https://docs.aws.amazon.com/lambda/latest/dg/vpc-rds-deployment-pkg.html https://forum.serverless.com/t/how-to-properly-close-the-database-connection-in-a-lambda-function/2545

goncaloneves commented 7 years ago

And another thing, that mustn't go unmentioned, the next time you open an issue like this one: Say thanks when someone helps you and you expect them to solve your code problem.

Open source tech is about politeness, being thankful and contribute. For all the rest, is a well paid service.

pizzarob commented 7 years ago

I'll keep debugging. Thanks. FYI I am caching DB connection in global scope ... @goncaloneves @juanjoDiaz

code

pizzarob commented 7 years ago

@goncaloneves sorry you took it that way. condescending comments about how I'm doing it wrong don't warrant a thank you in my book. I won't be using this plugin.