Crash when deleting a file

[letorbi]

Hi,

now its index new files added to watched folders, but if i delete file from this folders watcher crash with,

2020-02-23 21:28:12,260 ERROR An uncaught exception occurred Traceback (most recent call last): File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/synoindexw atcher/main.py", line 224, in start() File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/synoindexw atcher/main.py", line 199, in start for event in inotify.read(): File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/inotifyrec ursive/inotifyrecursive.py", line 145, in read self.add_watch_recursive(path, mask, info["filter"], event.name, event.wd) File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/inotifyrec ursive/inotifyrecursive.py", line 93, in add_watch_recursive for entry in os.listdir(path): FileNotFoundError: [Errno 2] No such file or directory: "/volume1/Filmy/Testovac katorentu/test.avi__DpXyQU"

Thx for reply.

Originally posted by @sousasov in https://github.com/letorbi/synoindexwatcher/issues/33#issuecomment-590122897

[letorbi]

Hi @sousasov,

thanks for reporting the crash. Unfortunately I cannot reproduce it. Could you please tell me the exact steps how to reproduce the crash? Also the output of python3 -m pip list would be helpful.

Regards, Torben

[sousasov]

It happens with transmission. When transmission delete files, it generates this bug.

I try delete file with DSM file manager and watcher dont crash. Immeadetly after that (file is deleted) i delete torrent and files in transmission(file is missing i deleted it through filem manager) transmission delete torrent a after that watcher crashed with this.

2020-02-23 23:52:56,216 ERROR An uncaught exception occurred Traceback (most recent call last): File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/synoindexwatcher/main.py", line 224, in start() File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/synoindexwatcher/main.py", line 199, in start for event in inotify.read(): File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/inotifyrecursive/inotifyrecursive.py", line 145, in read self.add_watch_recursive(path, mask, info["filter"], event.name, event.wd) File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/inotifyrecursive/inotifyrecursive.py", line 79, in add_watch_recursive wd = inotify_simple.INotify.add_watch(self, path, mask | flags.IGNORED | flags.CREATE | flags.MOVED_FROM | flags.MOVED_TO) File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/inotify_simple.py", line 105, in add_watch return _libc_call(_libc.inotify_add_watch, self.fileno(), fsencode(path), mask) File "/volume1/@appstore/py3k/usr/local/lib/python3.5/site-packages/inotify_simple.py", line 38, in _libc_call raise OSError(errno, strerror(errno)) OSError: [Errno 0] Success

python3 -m pip list Package Version

---

inotify-simple 1.3.2 inotifyrecursive 0.3.1 pip 20.0.2 setuptools 45.2.0 synoindexwatcher 0.10.1 wheel 0.34.2

[letorbi]

@chrisjbillington The log above shows that inotify_simple throws an OSError with [Errno 0] Success. This seems somewhat strange to me (an error with a positive message!?). Could it be that the exception should not be thrown at all?

[chrisjbillington]

I got a bug report like this once:

https://github.com/chrisjbillington/inotify_simple/issues/7

But was not able to reproduce it.

Since inotify_simple is just calling the underlying inotify API, this implies that inotify_add_watch returned -1.

inotfiy_add_watch's documentation says that:

On error, -1 is returned and errno is set appropriately.

So this is a bug in something...

This is the code in inotify_simple that calls the underlying C functions:

def _libc_call(function, *args):
    """Wrapper which raises errors and retries on EINTR."""
    while True:
        rc = function(*args)
        if rc != -1:
            return rc
        errno = get_errno()
        if errno != EINTR:
            raise OSError(errno, strerror(errno))

I'm not seeing any way that could be wrong other than:

• A bug in inotify_add_watch itself, it fails to set the errno
• A bug in Python's get_errno() function
• Python calling other code in the same thread in the meantime that clears the errno (garbage collection...?)

Errno is supposed to be a thread-local thing, so muilti-threaded code shouldn't have the power to mess with this.

I'll google around to see if anyone else has encountered unexpected errno shenanigans in Python.

[chrisjbillington]

Oh and of course a minimal example that reproduces the problem would be great too!

[chrisjbillington]

I can reproduce the issue.

Very simple. Something is quite wrong! This should obviously be a ENOENT

>>> from inotify_simple import masks, INotify
>>> i = INotify()
>>> i.add_watch('sdfsdf', masks.ALL_EVENTS)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/inotify_simple.py", line 105, in add_watch
    return _libc_call(_libc.inotify_add_watch, self.fileno(), fsencode(path), mask)
  File "/usr/lib/python3.8/site-packages/inotify_simple.py", line 38, in _libc_call
    raise OSError(errno, strerror(errno))
OSError: [Errno 0] Success

Might test it in C to see if it's a Pytohn-specific problem.

[chrisjbillington]

Fixed in inotify_simple 1.3.3

turns out the get_errno() function in ctypes doesn't get the underlying errno directly, it gets a special one stored in ctypes, which isn't set unless you set a flag when you open the shared library.

[chrisjbillington]

Of course, all the fix in inotify_simple does it turn OSError: Succes into FileNotFoundError. Then it's presumably a bug in synoindexwatcher or inotifyrecursive that it is attempting to add a watch on a deleted file (or just a race condition - there's always a chance a file will be deleted before you do what you were gonna do).

[letorbi]

Thanks Chris for the fast analysis and fix - really awesome :+1:

inotifyrecursive already handles FileNotFoundError exceptions, so I'd say there's nothing to be done on my side apart from updating the inotify_simple dependency to version 1.3.3.

[letorbi]

@sousasov Can you confirm that the bug is fixed, when you upgrade inotify_simple to version 1.3.3?

[sousasov]

@letorbi Hi, after update inotify_simple to version 1.3.3 everything works fine. Thx for your time.

Have a nice day.

[letorbi]

I've just released version 0.10.2, which requires at least inotifyrecursive 0.3.2, which requires inotify_simple 1.3.3. This should ensure that everyone uses the fixed version of inotify_simple, so I'm closing this issue.