dsoprea / PyInotify

An efficient and elegant inotify (Linux filesystem activity monitor) library for Python. Python 2 and 3 compatible.
GNU General Public License v2.0
242 stars 73 forks source link

Exception during shutdown because of __del__ #101

Open TD22057 opened 1 year ago

TD22057 commented 1 year ago

I have a complicated/large python program that is failing with an inotify exception when another error occurs. The problem is that in Inotify.del it uses the logger to send a debugging message. But - that variable doesn't exist in some cases. In my program, the python garbage collector has already deleted that variable and then it runs the del operator on the Inotify object contained in my class.

Python: 3.9.15 Inotify version: 0.2.10

The error looks like this:

Traceback (most recent call last):
[... my traceback removed... ]
    assert job.manifest
AttributeError: 'Manifest' object has no attribute 'manifest'
Exception ignored in: <function Inotify.__del__ at 0x7f1a93921b80>
Traceback (most recent call last):
  File "/home/trd/venv-test/lib/python3.9/site-packages/inotify/adapters.py", line 78, in __del__
AttributeError: 'NoneType' object has no attribute 'debug'

So my app is throwing an assert exception in a case I was testing but then another error happens in del. I've tried recreating this in a small script but can't - the order is undefined because at shutdown everything is being cleaned up and there is no way to explicitly control the order. My object has a member variable that is the Inotify object - and since my object is being deleted after the inotify package, this error is triggering.

In my experience accessing variables (in this case _LOGGER) outside the class in del can cause problems like this because there are scenarios like this one where python is shutting down and has already removed the variable. It's not a huge deal since everything is exiting anyway but it does lead to confusing errors for someone who doesn't understand what's happening.

TD22057 commented 1 year ago

Update: in my application, this happens whether there is an error or not. I have what will be a long running python app that watches directories. I build a class that stores an Inotify object as a member variable. When the program exits, I get the exception shown above because in my setup, my object is being cleaned up after the inotify module (maybe?) is being cleaned up. I haven't been able to make a simpler test case for this but it happens every time right now in my bigger application.

jimkimball commented 1 month ago

We see the same issue. Every single time.