Severe memory leak with LSP plugin usage

[DUOLabs333]

To be fair, this could be due to have large files open (I have 2 3MB files open), or my LSP (is there a way to disable LSP and syntax highlighting for large files?), but after ~5 hours, if I keep CudaText open, the process takes up 3GB of memory.

EDIT: It is not due to the LSP --- memory is creeping upwards, but none of the LSP servers are running.

[DUOLabs333]

I meant here: AppPython.RunModuleFunction.

[Alexey-T]

I meant here: AppPython.RunModuleFunction

maybe someone can improve it?

ah, we have one associated resource, in Py, but it lays not in pascal level. it is in cudatext.py

def _timer_proc_callback_proxy(tag='', info=''):
    if info in _live:
        return _live[info](tag)    

LSP calls timer_proc like this

                ct.timer_proc(ct.TIMER_START, self.timer_callback, self._last_period)  
...
        timer_proc(TIMER_START_ONE, self._scroll_to_end, 50, tag='')  
...
        timer_proc(TIMER_START_ONE, Hint.hide_check_timer, 750, tag='initial') 

so it is using callback-proxy list. ie _live list. maybe this list, _live, grows too much? can you debug it please?

[DUOLabs333]

There doesn't seem to be a memory leak in _live.

[Alexey-T]

i mean _live is dict.

to avoid usind _live, app must supply STRING form of timer callback. LSP plugin gives callable form of timer callback so its using _live.

[DUOLabs333]

But _live isn't the problem.

[Alexey-T]

What other associated (with timer) Py resources do you know?

[DUOLabs333]

The issues might be stemming from a conflict with QTimers being used, since after a while, cudatext crashes :

Crash Log

QObject: Cannot create children for a parent that is in a different thread. (Parent is QApplication(0x1625ac40), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject::startTimer: Timers can only be used with threads started with QThread QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject::setParent: Cannot set parent, new parent is in a different thread QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::ScrollBarEngine(0x16361340), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x163a22c0), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x163a22c0), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject::installEventFilter(): Cannot filter events for objects in a different thread. QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639e650), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639e650), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639e650), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639e650), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) QObject: Cannot create children for a parent that is in a different thread. (Parent is Breeze::WidgetStateEngine(0x1639fe60), parent's thread is QThread(0x1625ac20), current thread is QThread(0xffff6c0b4e10) [FORMS.PP] ExceptionOccurred Sender=EInvalidOp Exception=Invalid floating point operation Stack trace: $0000FFFFA4327EE0 $0000FFFFA3F07C7C $0000FFFF9EA2F234 $0000FFFF9EA2E5A4 $0000FFFF9EA16E38 $0000FFFFA3EABBD0 $0000FFFFA4622FD8 $0000FFFFA4623778 $0000FFFFA46335B8 $000000000081023C $00000000008AA9B4 $0000000000700F68 $0000000000701604 $0000000000492DF4 $000000000043B3B4 $0000000000708A04 $00000000006F9E9C [FORMS.PP] ExceptionOccurred QObject::killTimer: Timers cannot be stopped from another thread QObject::killTimer: Timers cannot be stopped from another thread QObject::killTimer: Timers cannot be stopped from another thread

[Alexey-T]

I tried to change TTimer class (in API timers) to TFPTimer (i did not study its code, maybe it's more light?)

beta: linux x64 qt5 uvviewsoft.com/c/ filename cudatext-qt5-12.zip

it crashes too?

[DUOLabs333]

I'm on ARM64, not x64.

[Alexey-T]

http://uvviewsoft.com/c/ now has file for ARM64 Qt5.

[DUOLabs333]

I get the same problem.

[Alexey-T]

it is caused by timer was created too many times? this changed timer_proc (py/cudatext.py) runs 'print' which shows counter in Console.

_timer_cnt = 0

def timer_proc(id, callback, interval, tag=''):
    global _timer_cnt
    if id in [TIMER_START, TIMER_START_ONE]:
        _timer_cnt += 1
        print('timer_cnt:', _timer_cnt)
    if callable(callback):
        sid_callback = str(callback)
        _live[sid_callback] = callback
        callback = 'module={};func=_timer_proc_callback_proxy;info="{}";'.format(__name__, sid_callback)
    return ct.timer_proc(id, callback, interval, tag)

[DUOLabs333]

Yeah, you're right --- this happens if you press Ctrl too much.

[Alexey-T]

If I print timer_cnt on TIMER_START and do decrement and print on TIMER_STOP/TIMER_DELETE, I get still increasing counter (with decrements). with LSP. timer starts are unbalanced?

[DUOLabs333]

Yeah, I'm seeing the same problem. This could be the leak --- timers are being created without being destroyed. For example, in the last 10 minutes, I'm up to 18 timers.

It seems to occur on any scroll and save of the LSP files.

[DUOLabs333]

Oh, I found a bug/oversight --- when a timer started with TIMER_START_ONE has finished, it should be deleted. This isn't currently done.

[Alexey-T]

But TIMER_DELETE is for that! plugin may want to reuse that timer for more TIMER_START_ONE calls so he don't want to delete it.

[Alexey-T]

Tried to do TIMER_DELETE in n places of LSP:

dlg.zip

'counter' is better now.

[Alexey-T]

here is new beta - timer is DELETED on tick after TIMER_START_ONE. http://uvviewsoft.com/c/

[DUOLabs333]

this changed timer_proc (py/cudatext.py) runs 'print' which shows counter in Console.

This doesn't account for the fact that timers are cached by their callback function. If you account for this, the counter stabilizes at ~6.

[Alexey-T]

what change do you suggest, I don't understand?

[DUOLabs333]

When a new timer is created, add sid_callback to a set. Before incrementing the timer, check if it's in the set.

[Alexey-T]

• so, no deleting of timer after TIMER_START_ONE is needed ? I gave the beta above. it is not needed?
• so, leak is gone now?

[DUOLabs333]

1. No, not really.
2. Unfortunately, also no.

[DUOLabs333]

Could you add a print statement in the Pascal code after every api_timer_proc that prints out the current length of the Timer list? I want to see if it is not caching callbacks correctly.

[Alexey-T]

added; beta at http://uvviewsoft.com/c/

[Alexey-T]

it does not 'print to Console' but 'writeln' to Terminal.

[DUOLabs333]

Ok, so there doesn't seem to be a memory leak in the Timers list.

[DUOLabs333]

Can you add a line that prints Py_REFCNT(Obj) right after Py_DECREF(Obj) in DoPyCallbackFromAPI? I want to see if maybe the reference count should be lowered a couple more times.

[DUOLabs333]

I see what the problem is with my threading rewrite --- threading only runs when GUI event is happening (eg, moving mouse, scrolling, clicking, saving, etc.).

[Alexey-T]

Can you add a line that prints Py_REFCNT(Obj) right after Py_DECREF(Obj) in DoPyCallbackFromAPI

no such Id - Py_REFCNT. do you mean this:

function TPyObject.Get_ob_refcnt: NativeInt;
begin
  Result := GetSelf^.ob_refcnt;
end;  

[DUOLabs333]

Yes.

[Alexey-T]

added 'writeln'. here: uvviewsoft.com/c/

[DUOLabs333]

Could you try adding Py_SET_REFCNT(obj, 0), or whatever is the equivalent for Pascal?

[Alexey-T]

It is bad style, usually code must call only IncDec / DecRef... why is it needed?

[DUOLabs333]

I just want to see if it's truly an issue with CudaText holding on to something longer than it should.

[Alexey-T]

added Py_CLEAR(Obj); at end of callback handler. beta: http://uvviewsoft.com/c/

[DUOLabs333]

Py_clear has the exact same effect as decref.

[DUOLabs333]

Could you add some lines to print out the reference count of the ParamsDic and Params in TAppPython.RunModuleFunction?

[Alexey-T]

Could you add some lines to print out the reference count of the ParamsDic and Params in TAppPython.RunModuleFunction?

added writeln lines for 2 objects: py Params refcnt: NNN. beta at http://uvviewsoft.com/c/

[DUOLabs333]

Could you comment out line 9178 in formmain_py_api.inc? I want to see if the issue is with the creation of the timers, or the actual execution of the Python function.

[Alexey-T]

I will make the beta, in a few days, after your attempts to compile Cud. write here 27-29 of May, can you compile?

note, I will be on vacation in June.

[DUOLabs333]

The snippet

for i:=0 to Length(FParamObjs)-1 do
    begin
        WriteLn('refcnt: ', FParamObjs[i]^.ob_refcnt);
    Py_DECREF(FParamObjs[i]);
    WriteLn('refcnt1: ', FParamObjs[i]^.ob_refcnt);
end;

errors out with Identifier not found "Py_DECREF", which is kind of weird, as it is used a few lines further down with Py_DECREF(Obj), which works fine.

[DUOLabs333]

I fixed the issue, and I tested CudaText both with and without my patch to make sure that I actually did fix the leak.

The root cause was that the reference to the objects in FParamObjs were not released after the module function was run, so it was never garbage collected by Python.

[Alexey-T]

errors out with Identifier not found "Py_DECREF", which is kind of weird, as it is used a few lines further down with Py_DECREF(Obj), which works fine.

Then try to write it as FEngine.Py_DecRef or GetPythonEngine.Py_DecRef.

[Alexey-T]

I fixed the issue, and I tested CudaText both with and without my patch to make sure that I actually did fix the leak.

The root cause was that the reference to the objects in FParamObjs were not released after the module function was run, so it was never garbage collected by Python.

That is good! Can you share your fix, pls?

[DUOLabs333]

PR #5535 is open.

[Alexey-T]

By analogy, I made few more fixes: https://github.com/Alexey-T/CudaText/commit/2680c0e6cade02f876d5832015cad79b47bb465d

Found places by search for func name: