dcramer / django-devserver

A drop-in replacement for Django's runserver.
BSD 3-Clause "New" or "Revised" License
1.27k stars 158 forks source link

"Memory usage was increased by" message is misleading #41

Closed aht closed 13 years ago

aht commented 13 years ago

I see that you are using self.heapy.domisize for this number, but from guppy, this really is

The dominated size of the set x. The dominated size of x is the total size of memory that will become deallocated, directly or indirectly, when the objects in x are deallocated.

which is a mouthful.

"Memory usage was increased by X Kb" is incorrect and it might freak people out. It certainly freaked me out as I saw the console output and thought some memory must have been reclaimed by GC, it can't just increase monotonically like that.

Perhaps "X KB of memory could have been reclaimed if all newly allocated objects are deallocated", but I think not.

So maybe you could use self.heapy.size (or self.heapy.indisize) and simply say "X KB of memory was allocated".

aht commented 13 years ago

To do what I think you'd want, I came up with the following patch and use a single heapy instance to measure how much is allocated and deallocated between requests (no need to use heapy.setref() as the guppy heap() instance support set difference operation)

diff --git a/devserver/modules/profile.py b/devserver/modules/profile.py
index 156ec75..609356b 100644
--- a/devserver/modules/profile.py
+++ b/devserver/modules/profile.py
@@ -56,22 +56,18 @@ else:
         """
         logger_name = 'profile'

-        def process_init(self, request):
-            from guppy import hpy
-        
-            self.usage = 0
-
-            self.heapy = hpy()
-            self.heapy.setrelheap()
+        def __init__(self, request):
+            super(MemoryUseModule, self).__init__(request)
+            self.hpy = hpy()
+            self.oldh = self.hpy.heap()
+            self.logger.info('heap size is %s', filesizeformat(self.oldh.size))

         def process_complete(self, request):
-            h = self.heapy.heap()
-        
-            if h.domisize > self.usage:
-                self.usage = h.domisize
-        
-            if self.usage:
-                self.logger.info('Memory usage was increased by %s', filesizeformat(self.usage))
+            newh = self.hpy.heap()
+            alloch = newh - self.oldh
+            dealloch = self.oldh - newh
+            self.oldh = newh
+            self.logger.info('%s allocated, %s deallocated, heap size is %s', *map(filesizeformat, [alloch.size, dealloch.size, newh.size]))
dcramer commented 13 years ago

@aht thats awesome -- would you want to send that as a pull request so I can let github give you credit :)

aht commented 13 years ago

https://github.com/dcramer/django-devserver/pull/42

hey it's a 42!