Unmanic / unmanic

Unmanic - Library Optimiser
GNU General Public License v3.0
1.41k stars 84 forks source link

Update file monitor to only use "closed" event #440

Closed BlakeGardner closed 4 months ago

BlakeGardner commented 4 months ago

Pull request

CLA

Checklist

Description of the pull request

This pull request modifies the event triggers for file checking in Unmanic. It addresses an issue where copying a file to a monitored folder triggers both created and closed events. This redundancy causes the file to be scanned twice, leading to crashes if the file is still in the process of copying. The issue is replicable with slow file transfers to the monitored directory, where files may begin transcoding before the copy is complete.

Examples

  1. Creating an Empty File:

    • Command: touch foo.mp4
    • Logs:
      • Detected 'created' event on '/library/movies/My Movie/foo.mp4'
      • Detected 'closed' event on '/library/movies/My Movie/foo.mp4'
  2. Copying a File to the Monitored Directory:

    • Command: rsync --inplace --progress "/Downloads/My Movie.mp4" "/media/My Movie/"
    • Logs:
      • Initial created event triggers a premature file check, resulting in a subprocess error.
      • Subsequent closed event triggers a successful check, revealing the file is under the minimum size threshold.

Raw log example

INFO:Unmanic.EventProcessor - [FORMATTED] - Detected 'created' event on file path '/library/movies/My Movie/My Movie.mp4'
DEBUG:Unmanic.Plugin.ignore_under_size - Checking file /library/movies/My Movie/My Movie.mp4
DEBUG:Unmanic.Plugin.ignore_under_size - Requested bytes/TimeUnit.SECOND is 687500.0
DEBUG:Unmanic.Plugin.ignore_under_size - Requested (time normalized) bytes/second is 687500.0
ERROR:Unmanic.PluginExecutor - [FORMATTED] - Exception while carrying out 'library_management.file_test' plugin runner 'ignore_video_file_under_size'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/unmanic/libs/unplugins/executor.py", line 256, in execute_plugin_runner
    runner(data)
  File "/config/.unmanic/plugins/ignore_video_file_under_size/plugin.py", line 181, in on_library_management_file_test
    video_data = get_video_data(file_path)
  File "/config/.unmanic/plugins/ignore_video_file_under_size/plugin.py", line 105, in get_video_data
    data = subprocess.check_output(
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ffprobe', '-v', 'error', '-show_entries', 'stream=duration,width,height:format=duration', '-print_format', 'json', '/library/movies/My Movie/My Movie.mp4']' returned non-zero exit status 1.
2023-12-28T15:19:01:DEBUG:Unmanic.Plugin.video_transcoder - File unable to be probed by FFProbe - '/library/movies/My Movie/My Movie.mp4'

# ............ after file finishes copying

INFO:Unmanic.EventProcessor - [FORMATTED] - Detected 'closed' event on file path '/library/movies/My Movie/My Movie.mp4'
DEBUG:Unmanic.Plugin.ignore_under_size - Checking file /library/movies/My Movie/My Movie.mp4
DEBUG:Unmanic.Plugin.ignore_under_size - Requested bytes/TimeUnit.SECOND is 687500.0
DEBUG:Unmanic.Plugin.ignore_under_size - Requested (time normalized) bytes/second is 687500.0
DEBUG:Unmanic.Plugin.ignore_under_size - Video has resolution of 1914x1036 and a duration of 6395.007 seconds
DEBUG:Unmanic.Plugin.ignore_under_size - Video has a size/second of 278853.4734989344
DEBUG:Unmanic.Plugin.ignore_under_size - Scaling threshold by ratio 0.9562615740740741 to compensate for resolution difference
DEBUG:Unmanic.Plugin.ignore_under_size - Requested (resolution normalized) bytes/second is 657429.832175926
INFO:Unmanic.EventMonitorManager - [FORMATTED] - File '/library/movies/My Movie/My Movie.mp4' should be ignored because it is under the configured minimum size/hour '687500B/s'.