search

GLab / ToMaTo

German-Lab ToMaTo
http://www.tomato-lab.org
32 stars 21 forks source link

Memory usage #1417

Open dswd opened 8 years ago

dswd commented 8 years ago

Sometimes ToMaTo backend-core uses huge amounts of memory and continues to grow until it crashes. I suspect that there is a data structure that (under some circumstances) grows infinitely and something is storing new data in a hot loop.

We should check the code for a) Potentially hot loops that could generate huge amounts of data in short time b) Unbounded data structures that can grow infinitely over time

I think we might find something in the error handling code.

a-gerhard commented 8 years ago

the dump limit may have fixed that, lets wait until a new version is rolled out on the server.

a-gerhard commented 8 years ago

We could try to use sys.getsizeof to determine the size of the cache, the dumps library's dumps, and some other stuff. Maybe that could give us some clues...