Possible out-of-memory due to unlimited cache _type_done_callbacks if dynamically generated types are used

[freininghaus]

We found that an application which has functions with local classes that end up in Promise resolutions can go out of memory if it runs sufficiently long.

The reason is that the type of the local classes is different every time the functions are called, such that the lookup in the dict promise.promise._type_done_callbacks fails, and the new type is inserted into the dict. This prevents that the local classes are garbage collected, and causes a continuous increase of the memory usage.

I will submit a pull request with a fix and unit test soon.

Example to reproduce the problem:

import promise
import time

# Uncomment the following two lines to fix the OOM issue:
# import weakref
# promise.promise._type_done_callbacks = weakref.WeakKeyDictionary()

def function_with_local_type():
    class A:
        # Attach a lot of data to the type 'A' to make the OOM issue show up faster
        data = [0] * 1024 * 1024

    a = A()

    # Resolving a Promise with 'a' causes the type 'A' to end up
    # in the dict promise.promise._type_done_callbacks...
    assert a == promise.Promise.resolve(a).get()

# Watch the script's memory usage, e.g., in htop, while the loop runs...
for _ in range(32 * 1024):
    function_with_local_type()
    time.sleep(0.01)

[syrusakbary]

There is a new version of Promise 2.3.0 that ships with the fix done in #76.

Closing the issue