exaloop / codon

A high-performance, zero-overhead, extensible Python compiler using LLVM
https://docs.exaloop.io/codon
Other
15k stars 517 forks source link

【Codon】With repeated processing, memory consumption increases, Killed is displayed, and processing stops. #551

Open icst005 opened 6 months ago

icst005 commented 6 months ago

I was not sure if there was a report applicable to the same case, so I am reporting it just in case.

In the following environment and execution of the code in codon, the memory usage increases and the process becomes Killed and stops. When executing the code in Python, there is no increase in memory usage and the process completes to the end. ("owari" is displayed.  「np07b = None」 is not working in codon?)

If you can get the Log by some means and need to provide it, please contact me. I will try to help.

Windows 10 Pro 22H2 Windows Feature Experience Pack 1000.19055.1000.0

Core(TM) i7-7700HQ memory:64GB

codon:v0.16

From Windowshell, execute the following command 「codon run -release test01.py」

-------------------- test01.py ------------------ import time import datetime from python import numpy as np from python import pandas as pd from python import random

def main(aa000):

t00 = time.time()

for aa001 in range(1000000): 

    np07b = np.zeros((99999, 1) , dtype = 'int32')

    np.random.seed()
    np07b[0: 99999, 0] = np.random.randint(0, 9, 99999)

    np07b = None

t01 = time.time()
t02 = t01 - t00
print(t02)
print("owari")

 #------------------------------------------------------------                           
 #------------------------------------------------------------ 

t00 = time.time()

for aa000 in range(1):

main(aa000)
arshajii commented 5 months ago

Thanks for issue @icst005 -- we'll look into it. Creating Python objects in Codon allocates memory tracked by the GC, but the GC should be collecting the temporary objects so memory growth definitely shouldn't be unbounded.

icst005 commented 5 months ago

Thank you for your comment.

If there is anything else I need to provide, such as additional information, please do not hesitate to let me know.