Open ericsj opened 2 years ago
@ericsj Thanks for posting! We'll take a look as soon as possible.
In the mean time, there are a few ways you can help speed things along:
Please remember: never post in a public forum if you believe you've found a genuine security vulnerability. Instead, disclose it responsibly.
For help with questions about Sails, click here.
@ericsj Without looking at the code for this app, I think the most likely scenario here is your app has an action that is fetching too much of the database into memory at once. i.e. needs to be paginated (limit
). This can sometimes come up indirectly, for example when trying to respond with a list of red cars with any current passengers who are wearing shoes. An antipattern is to look up all red cars first. But the problem is, there might be too many red cars to fit into memory. Another, simpler possibility is that a primary piece of data intended to be used by the action is just too big for your server's available RAM-- for example an action that returns an unpaginated list of millions of records.
@ericsj Without looking at the code for this app, I think the most likely scenario here is your app has an action that is fetching too much of the database into memory at once. i.e. needs to be paginated (
limit
). This can sometimes come up indirectly, for example when trying to respond with a list of red cars with any current passengers who are wearing shoes. An antipattern is to look up all red cars first. But the problem is, there might be too many red cars to fit into memory. Another, simpler possibility is that a primary piece of data intended to be used by the action is just too big for your server's available RAM-- for example an action that returns an unpaginated list of millions of records.
I searched the bug source and found it by moving through the commit history while supervising the chrome inspector. The commit added a simple endpoint with a model.find({}, ...) operation without limit or pagination and called it twice. I solved it adding limit. Surprisingly the table of that model has only 22mb (see print attached), so I still don't understand the bug completely. Thank you by your help anyway. Old endpoint:
ProviderInformation.find({}, function(err, records) {...
New:
ProviderInformation.find({ where: { networkBuild: buildProvider.networkBuild }, limit: 1000 })
.exec(function(err, providerInformationInSameBuild) { ...
@ericsj thanks for posting this, we are having a similar problem and we suspect a memory leak somewhere in the code or the sails framework. Our projects are running inside a docker containers. We are noticing that the sails process i.e. the docker container process, is keep on getting the memory gradually. We have observed this for quite some time and now we are sure that there is some memory leak somewhere. When we start the container afresh, it starts with less memory, and everyday it increases by a few MBs of memory, even without getting it freed by GC. Therefore after every 3-4 weeks we have to manually restart the process by restarting the container and then thing goes back to normal, and this process is repeated. If we do not do this activity then the entire system performance slows down. Note that these are just REST APIs without any frontend, gulp or grunt tasks.
I need help with - 1) how do we find memory leaks in sails project? 2) How can we be sure if memory leak is due to the source code or the sails framework itself?
@mikermcneil any help/suggestion would be highly appreciated.
You can find memory leaks using the inspector:
node --inspect index.js
Open in the browser the link: chrome://inspect To find the source of the leak, navigated through the commit history and noted that the code above introduced the memory leak, by the memory usage graphs, and the only way to discover if the memory leak is caused by sails framework is changing the source code, but I guess 99% of the cases the memory leak is caused by the source code, so I suggest trying to find which commit introduced the leak and testing it
Node version: v12.22.9 Sails version (sails): 0.12.14 ORM hook version (sails-hook-orm): 1.0.9 Sockets hook version (sails-hook-sockets): 0.13.11 Organics hook version (sails-hook-organics): -- Grunt hook version (sails-hook-grunt): -- Uploads hook version (sails-hook-uploads): -- DB adapter & version (e.g. sails-mysql@5.55.5): sails-postgresql@0.11.4 Skipper adapter & version (e.g. skipper-s3@5.55.5): skipper@0.7.6
I manage a Node.js server that tipically occupies ~65MB of RAM. After opening a certain page, the memory usage increases drastically, until it freezes the PC or throws a FATAL ERROR with a heap related error. In the second screenshot we can note an array emerging and occupying a large portion of memory. I found an object that is really present in the code (that purple providerinformation in the inferior part). But it doesn't tell much, it's a just a model name and page name. In an older version of the code that is running on prd server it doesn't happen so I don't think it is caused by Sails, since I didn't update sails (at least not in package.json) and I didn't find memory leak issues in my node version (the server runs an older version). How can I proceed? Notes:
Error message: