taichi-dev / taichi

Productive, portable, and performant GPU programming in Python.
https://taichi-lang.org
Apache License 2.0
25.35k stars 2.27k forks source link

Memory increases when creating and destroying the FieldsBuilder repeatedly #7242

Open lin-hitonami opened 1 year ago

lin-hitonami commented 1 year ago

Describe the bug The memory increases when running the snippet below by ~5KB/iter. From https://forum.taichi-lang.cn/t/topic/3839 To Reproduce

import taichi as ti
ti.init()

def w():
     i = 0
     while i < 1000000:
         particle_per_cell = 1000000
         num_particle_in_cell = ti.field(int)
         fb1 = ti.FieldsBuilder()
         fb1.dense(ti.i, particle_per_cell).place(num_particle_in_cell)
         fb1_snode_tree = fb1.finalize()
         fb1_snode_tree.destroy()
w()
jim19930609 commented 1 year ago

For this specific issue, the memory leak comes from pytaichi.global_vars which holds a expr_field corresponding to the place-SNode of the field. Upon SNodeTree destruction, the cached expr_field won't get poped from the global_vars, resulting in the memory leak.

截图 2023-04-03 14-07-34

The bigger issue is that there's no lifetime management for the global variables associated with PyTaichi, therefore similar memory leak will still appear if we only fix this single issue. I'd suggest fixing this PR along with the Type System Refactor, where we can implement a more systematic mechanism to manage the global variables & caches at Python scope.