search

jfowkes / pycutest

Python interface to CUTEst
https://jfowkes.github.io/pycutest/
GNU General Public License v3.0
27 stars 10 forks source link

Release of memory #60

Closed yihang-gao closed 1 year ago

yihang-gao commented 1 year ago

Hi, it seems that the pycutest still occupy much CPU memory and will not release if we didn't terminate the python program. 

problems_name = ["HS7", "HS6", "HS56", "HS48", "HS60"]
tracemalloc.start()
for name in problems_name:
    prob = pycutest.import_problem(name, drop_fixed_variables=True)
    for i in range(2000):
        x = prob.x0
        g = prob.objcons(x)

    del prob
    gc.collect()

    snapshot = tracemalloc.take_snapshot()
    display_top(snapshot)

It is reported that the CPU memory is increasing... Looking forward to your help, thanks!

yihang-gao commented 1 year ago

Hi, it seems that the pycutest still occupy much CPU memory and will not release if we didn't terminate the python program.

problems_name = ["HS7", "HS6", "HS56", "HS48", "HS60"]
tracemalloc.start()
for name in problems_name:
    prob = pycutest.import_problem(name, drop_fixed_variables=True)
    for i in range(2000):
        x = prob.x0
        g = prob.objcons(x)

    del prob
    gc.collect()

    snapshot = tracemalloc.take_snapshot()
    display_top(snapshot)

It is reported that the CPU memory is increasing... Looking forward to your help, thanks!

although I see some related issues #42 and #33 ...

jfowkes commented 1 year ago

Due to the way Python works there will still be some memory taken up, but this is a) much smaller than before, and b) due to memory leaks present in NumPy and the Python C API itself, and therefore not something we can do anything about. If you run valgrind on a Python debug instance you will see that we do not leak any memory (the memory is leaked by NumPy and Python).

yihang-gao commented 1 year ago

Thanks so much for your response.

yihang-gao commented 1 year ago

Due to the way Python works there will still be some memory taken up, but this is a) much smaller than before, and b) due to memory leaks present in NumPy and the Python C API itself, and therefore not something we can do anything about. If you run valgrind on a Python debug instance you will see that we do not leak any memory (the memory is leaked by NumPy and Python).

I commented "del" in problem_class.py, but nothing changed. The detected CPU memory occupied keeps the same..

jfowkes commented 1 year ago

I commented "del" in problem_class.py, but nothing changed. The detected CPU memory occupied keeps the same..

You need to test on a large problem (choose a problem whose sifParams give a very high dimension) as you won't see much effect on small toy problems that use little memory.

yihang-gao commented 1 year ago

I commented "del" in problem_class.py, but nothing changed. The detected CPU memory occupied keeps the same..

You need to test on a large problem (choose a problem whose sifParams give a very high dimension) as you won't see much effect on small toy problems that use little memory.

thanks so much for your reply