slezica / python-frozendict

A simple immutable mapping for python
MIT License
111 stars 22 forks source link

Possible memory leak with Python 3.9 #38

Open o-khytrov opened 2 years ago

o-khytrov commented 2 years ago

When frozendict is converted to string in python 3.9, it causes allocations of memory which can not be released with gc.collect()

import gc
import psutil
from frozendict import frozendict

def rss():
    return psutil.Process().memory_info().rss

def process(body):
    str(body)

if __name__ == '__main__':
    initial_memory = rss()
    for i in range(100000):
        payload = frozendict({"text": "Hello", "category": "electronics", "keyword": "electronics"})
        # payload = {"text": "Hello", "category": "electronics", "keyword": "electronics"}
        process(payload)

    print("memory before: ", initial_memory)
    gc.collect()
    print("memory after:  ", rss())

Result with frozendict

memory before:  13094912
memory after:   27693056

Result with builtin dictionary

memory before:  13086720
memory after:   13086720
Marco-Sulla commented 2 years ago

This repo is unmantained. Please look at https://github.com/Marco-Sulla/python-frozendict/issues/59