Deferred should wait for async operations

[GoogleCodeExporter]

deferred.defer doesn't wait for async operations, I think the function should 
be wrapped around a @ndb.toplevel internally

Original issue reported on code.google.com by KaanSo...@gmail.com on 25 Feb 2013 at 8:41

[GoogleCodeExporter]

Why?  If you need defer to wait for something, you should do the waiting 
yourself.  Any time I don't yield on an async operation it's on purpose.

Original comment by jim@twist.com on 7 Mar 2013 at 4:58

[GoogleCodeExporter]

Why? Simple. If you don't wait, the operation gets discarded, you lose data. 
deferred doesn't wait.
For example, you can item.put_async() and forget about it if @ndb.toplevel is 
there, it will be put, however with deferred, you have to manually .wait() for 
it at the end, so unpretty
There are many solutions obviously, I'm not looking for work-arounds.

Original comment by KaanSo...@gmail.com on 7 Mar 2013 at 6:35

[GoogleCodeExporter]

The deferred package has very different responsibilities than NDB.  I think 
coupling the two in any way is wrong.

I also fail to understand your use case.  Is the async call in the function 
passed to defer() or in the code running before calling defer()?  And why would 
the async call be discarded?  It may not run before the deferred function runs, 
but it's not clear if that's what you have in mind.

Original comment by gvanrossum@gmail.com on 25 Apr 2013 at 3:26

• Added labels: Priority-Low
• Removed labels: Priority-Medium

[GoogleCodeExporter]

The async calls are from the function passed to defer()

Let's say we deferred function F, and function F, created A,B,C and 
put_async()'d them.

Since we don't explicitly wait for put_async()'s any/all of A,B,C might not 
exist after F's execution

You can't simply wrap F with @ndb.toplevel, so you either have to create 
another function G, wrap it with @ndb.toplevel, and call G within F and do all 
async operations inside G -- or -- wait for everything explicitly

I currently defer a wrapper function with the actual function name, and wrap 
the actual functions with @ndb.toplevel, so my defer calls are like:
deferred.defer(function_wrapper,"F")

the function_wrapper calls F from globals()

Original comment by KaanSo...@gmail.com on 25 Apr 2013 at 10:03

[GoogleCodeExporter]

Ok, that clarifies things.  I don't know if anyone from Google is following 
this tracker, and since I don't work for Google any more I can't do anything 
about the deferred library.  I recommend that you file a bug report in the 
general App Engine tracker and explain things there.

Original comment by gvanrossum@gmail.com on 25 Apr 2013 at 3:18

[GoogleCodeExporter]

Original comment by pcoste...@google.com on 27 Nov 2013 at 5:33

• Changed state: Started

[GoogleCodeExporter]

Original comment by pcoste...@google.com on 27 Nov 2013 at 5:38

• Added labels: log-11634414

[GoogleCodeExporter]

Fixed in ndb release 1.0.10 (SDK release 1.8.9). This release allows 
defereed.defer to properly differ a function that has been wrapped with 
@ndb.toplevel, which will ensure your async calls get completed.

Original comment by pcoste...@google.com on 11 Jan 2014 at 11:51

• Changed state: Fixed