QuantCrimAtLeeds / PredictCode

Standardised implementations of prediction techniques in the literature
Other
20 stars 14 forks source link

Garbage collection seems to force tkinker meltdown #6

Open MatthewDaws opened 7 years ago

MatthewDaws commented 7 years ago

On linux, it is quite easy to generate the following scary messages in the console:

Exception ignored in: <bound method Variable.__del__ of <tkinter.IntVar object at 0x7f70ec5f59e8>>
Traceback (most recent call last):
  File "/nfs/see-fs-02_users/matmdpd/anaconda3/lib/python3.6/tkinter/__init__.py", line 329, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
_tkinter.TclError: out of stack space (infinite loop?)

To recreate this: do anything which displays a lot of graphs (e.g. run an analysis, then view it and click to view a lot of different days). I explored this and came up with a minimal example:

https://stackoverflow.com/questions/44781806/tkinter-objects-being-garbage-collected-from-the-wrong-thread

My belief is that it's caused by the garbage collector running on a thread different to the main GUI thread, and calling the __del__ finaliser method on certain tk objects which then causes the error, as Tk on linux seems strict about which thread it runs on. (On Windows, by adding logging to the tkinter library code, I can verify that the same thing is happening; just there is not an ultimate error message, presumably because Tk on Windows is more tolerant of being called on the wrong thread).

This seemed to be benign, but I have now observed it killing sub-processes: the symptom is that an analysis run will take forever to run, and before this, you'll notice that Python is only using n-1 (or n-2 etc.) processes, instead of the usual n (= number of cores on your machine).

I have mitigated this by adding an explicit GC collection before critical points in the code.

MatthewDaws commented 7 years ago

If there are any Python / tkinter experts out there, I would of course prefer a better solution...

MatthewDaws commented 7 years ago

I've now verified that this seems to affect the latest Ubuntu release (so scorching hopes that it was due to me running an old linux install...)