Deferred tasks sometimes do not find modules

[GoogleCodeExporter]

1. set up a deferred task, with `from google.appengine.ext import deferred`;

2. have the deferred task import a module from a non-standard python path (i.e. 
not found on the PYTHONPATH environment variable), but which is found on a path 
added to `sys.path` by the application;

3. run the appserver and call the deferred task.

One would expect the deferred task to consistently run. However, I get the 
following more often than not:

Traceback (most recent call last): File "/Users/bmh/dev/vendors/appengine-devappserver2-experiment/google/appengine/ext/deferred/deferred.py", line 309, in post self.run_from_request() File "/Users/bmh/dev/vendors/appengine-devappserver2-experiment/google/appengine/ext/deferred/deferred.py", line 304, in run_from_request run(self.request.body) File "/Users/bmh/dev/vendors/appengine-devappserver2-experiment/google/appengine/ext/deferred/deferred.py", line 144, in run raise PermanentTaskFailure(e) PermanentTaskFailure: No module named flask

Obviously in this case I am trying to import flask. Flask is in my 
APP_ROOT/vendors directory, which python searches because I have code like this 
at the beginning of my `application.py` handler (which is a different file from 
the one containing the deferred task):

```python
import sys
sys.path.append(path.join(settings.APP_ROOT, 'vendors/'))

In the file containing the deferred task I will have at the head, among other things, something like:

from flask import request #, ...

The deferred task is called in the usual way with something like:

def deferred_todo():
   # ... 

def handle_request():
   # ... do something based on a HTTP request
   deferred.defer(_deferred_todo)

Sometimes the deferred task will run, indicating some sort of race condition.

The workaround I have found is to hard-code my add-on paths into the file containing the deferred call, like this:

import sys
sys.path.append("contrib/")
from flask import request #, ...

I hope the above is helpful.

Original issue reported on code.google.com by `brianmh...@gmail.com` on 21 Jan 2013 at 5:51

[GoogleCodeExporter]

Original comment by bquin...@google.com on 22 Jan 2013 at 11:25

• Changed state: Accepted

[GoogleCodeExporter]

Thanks for accepting.

I am linking related googleappengine Issue 5701: modify python paths with 
app.yaml

    http://code.google.com/p/googleappengine/issues/detail?id=5701

A fix for Issue 5701 would presumably also address the issue here. 

Original comment by brianmh...@gmail.com on 22 Jan 2013 at 2:39

[GoogleCodeExporter]

This looks like it's working as intended. If a deferred request is the first 
request to hit an instance, your path setup won't have run. devappserver2 is 
correctly emulating production behavior.

You may be able to work around this by doing your path setup in your 
appengine_config.py.

Original comment by sa...@google.com on 22 Jan 2013 at 11:19

• Changed state: WontFix

[GoogleCodeExporter]

Good to know, thanks.

It would be wonderful if the tip on putting python path into 
`appengine_config.py` were documented (I couldn't seem to find it). It's very 
handy.

Original comment by brianmh...@gmail.com on 23 Jan 2013 at 1:28