search

almostearthling / pyclips

PyCLIPS - a Python module to integrate CLIPS into Python
Other
35 stars 19 forks source link

[ENVRNMNT8] Environment data not fully deallocated. #4

Open gitttt opened 7 years ago

gitttt commented 7 years ago

I get the above message every time I use pyclips. What is it about? Should it be fixed or can it be safely ignored? Are there cases in which it may pose a problem?

almostearthling commented 7 years ago

Hi. It might occur when you exit the Python interpreter and thus PyCLIPS: in this case you can ignore it, although it's not a good sign, and the OS will take care of deallocating all data that Python, PyCLIPS and the underlying CLIPS library have allocated. In case you receive it when you deallocate an environment (eg. you delete the corresponding Python container) it means that the underlying CLIPS environment object has still some data in it, and it's probably still alive somewhere in the heap. In other words this message kindly reports a severe memory leak. CLIPS is sometimes quite quirky in the way it handles data, and although I tried to use the Python-provided memory allocators and deallocators, there might be some point in the CLIPS code where this option is ignored and the normal malloc() and free() functions are called, making memory handling more difficult and prone to errors.

almostearthling commented 7 years ago

Sorry for the late answer: I've been away for some weeks with limited network access!

gitttt commented 7 years ago

No problem. I am thankful that you are working on this open source project!

For example, I ran the testsuite\tests.py and got:

Ran 124 tests in 9.500s

OK

[ENVRNMNT8] Environment data not fully deallocated.

[ENVRNMNT8] MemoryAmount = 88.

[ENVRNMNT8] MemoryCalls = 1.

[ENVRNMNT8] Environment data not fully deallocated.

[ENVRNMNT8] MemoryAmount = 88.

[ENVRNMNT8] MemoryCalls = 1.

What does this tell me? Is it a CLIPS or PyCLIPS leak?

almostearthling commented 7 years ago

Maybe it's a CLIPS leak, but I actually cannot say it precisely: PyCLIPS, knowing that CLIPS does a lot of memory allocation and deallocation, tries to be somewhat conservative about memory by tagging some of the objects that are passed to CLIPS as "borrowed" - I preferred to risk a leak than a segmentation fault, actually, as I couldn't be sure how long a CLIPS resulting value (especially in dynamic data types such as multifields) could be kept in Python. However in PyCLIPS I tried to substitute the default allocator (malloc) with the Python provided one, as well as related functions for reallocation and disposal. This should ensure that all memory is freed when the Python interpreter exits; however with modern OSs this is not the issue, because they would free the main process memory anyway on exit: the problem is that a program that uses environments dynamically would suffer of such leaks in the long run.

If I had the time to work on it I'd prefer to completely remove the current environment/other environments dichotomy and just leave environments as main objects to work with - without a current environment at all. This would simplify memory management too, I think.