ZipImporter called on every request resulting in high CPU usage

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Build a 0.96 Django project, apply AppEngine patch, upload to GAE -
beyond that I don't know.

What is the expected output? What do you see instead?

The problem is unexpected high CPU usage
08-08 11:12AM 11.077 / 200 2501ms 3008cpu_ms 124api_cpu_ms 0kb
...
zipimporter('/base/data/home/apps/MY_APP_NAME/5.335434339950952156/common/zip-pa
ckages/django-1.0.2.zip',
'')

What version of the product are you using? On what operating system?
Django 0.96, Python 2.5, GAE

Please provide any additional information below.

Original issue reported on code.google.com by isaac.n....@gmail.com on 9 Aug 2009 at 12:48

[GoogleCodeExporter]

Google keeps the imports cached for a certain amount of time (maybe a few 
seconds).

If you are getting hits at a lower frequency, like once every minute, the 
zipimport
has to happen again, and you'll get the CPU hit. This looks very scary, because 
the
CPU usage on your site will start creeping towards the free CPU quota.

The good news is, when traffic goes up, this problem goes away, so don't worry 
too
much about it. When your site is getting a few hits a second or more, the 
imports
stay cached pretty much all the time, and CPU usage actually goes way down.

To prove this is happening, you can put this at the top of views.py:

import logging

logging.info("re-imported views.py (and re-imported django.zip)")

That log message will happen only when views.py is re-imported (due to it not 
being
cached anymore). You'll notice that the above log message will happen in 
lock-step
with the big CPU warnings.

Original comment by secor...@gmail.com on 13 Aug 2009 at 5:17

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

OK, thanks for the information!

Original comment by isaac.n....@gmail.com on 13 Aug 2009 at 2:25

• Added labels: ****
• Removed labels: ****