search

pymedusa / Medusa

Automatic Video Library Manager for TV Shows. It watches for new episodes of your favorite shows, and when they are posted it does its magic.
https://pymedusa.com
GNU General Public License v3.0
1.78k stars 276 forks source link

Limit filesystem access for subtitle scan and backlog to help RAID spin down #7975

Open Rouzax opened 4 years ago

Rouzax commented 4 years ago

In the past I already created a similar request https://github.com/pymedusa/Medusa/issues/5960 which was resolved.

Today I noticed another file access while maybe not strictly necessary. For a subtitle scan it is reading the episode back again from disk while that data is already available in the database. This episode was completed on 14/04/2020, 16:39:58 and the subtitle info should already be in the DB

2020-04-15 17:43:45 DEBUG    FINDSUBTITLES :: [8f6b666] Starting new thread: FINDSUBTITLES
2020-04-15 17:43:45 INFO     FINDSUBTITLES :: [8f6b666] Checking for missed subtitles
2020-04-15 17:43:46 DEBUG    FINDSUBTITLES :: [8f6b666] Checking subtitle candidates for Saturday Night Live S45E16 (Saturday Night Live - S45E16 - [1080p WEB-DL h264][monkee].mkv)
2020-04-15 17:43:46 DEBUG    FINDSUBTITLES :: [8f6b666] Scanning video C:\DATA\TVSeries\Saturday Night Live\Season 45\Saturday Night Live - S45E16 - [1080p WEB-DL h264][monkee].mkv...
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Starting release refiner [extension=release, release_name=Saturday.Night.Live.S45E16.REPACK.Saturday.Night.Live.at.Home.Tom.Hanks.Chris.Martin.1080p.HULU.WEB-DL.DD+5.1.H.264-monkee, release_file=None]
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Guessing using C:\DATA\TVSeries\Saturday Night Live\Season 45\Saturday.Night.Live.S45E16.REPACK.Saturday.Night.Live.at.Home.Tom.Hanks.Chris.Martin.1080p.HULU.WEB-DL.DD+5.1.H.264-monkee.mkv
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Attribute audio_codec changed from None to 'Dolby Digital Plus'
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Refining using Series information.
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Attribute series_tvdb_id changed from None to 76177
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Attribute series_imdb_id changed from None to 'tt0072562'
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Attribute year changed from None to 1975
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Refining using Episode information.
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Attribute tvdb_id changed from None to 7679364
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Attribute title changed from None to 'SNL At Home: Tom Hanks'
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Video information cached under key medusa.subtitles:video|C:\DATA\TVSeries\Saturday Night Live\Season 45\Saturday Night Live - S45E16 - [1080p WEB-DL h264][monkee].mkv
2020-04-15 17:44:40 DEBUG    FINDSUBTITLES :: [8f6b666] Creating a new ProviderPool instance
2020-04-15 17:44:50 INFO     FINDSUBTITLES :: [8f6b666] No subtitles found for Saturday Night Live - S45E16 - [1080p WEB-DL h264][monkee].mkv
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] 76177: Saving last subtitles search to database
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] 76177: Saving episode to database: Saturday Night Live S45E16
2020-04-15 17:44:50 INFO     FINDSUBTITLES :: [8f6b666] 76177: No subtitles found for Saturday Night Live S45E16
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for The Goldbergs (2013) S07E12 delayed for 6 days
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for Manhunt S02E02 delayed for 13 days, 16 hours, 10 minutes
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for Manhunt S02E04 delayed for 13 days, 16 hours, 10 minutes
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for Manhunt S02E03 delayed for 13 days, 16 hours, 10 minutes
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for Manhunt S02E05 delayed for 13 days, 16 hours, 10 minutes
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for Manhunt S02E06 delayed for 14 days, 15 hours, 33 minutes
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for Manhunt S02E07 delayed for 21 days, 14 hours, 34 minutes
2020-04-15 17:44:50 DEBUG    FINDSUBTITLES :: [8f6b666] Subtitle search for Manhunt S02E08 delayed for 28 days, 16 hours, 39 minutes
2020-04-15 17:44:50 INFO     FINDSUBTITLES :: [8f6b666] Finished checking for missed subtitles

This is what I see in Windows file access

An attempt was made to access an object.

Subject:
    Security ID:        HOME\SA-Medusa
    Account Name:       SA-Medusa
    Account Domain:     HOME
    Logon ID:       0x135DA4BC3

Object:
    Object Server:      Security
    Object Type:        File
    Object Name:        E:\Data\TVSeries\Saturday Night Live\Season 45\Saturday Night Live - S45E16 - [1080p WEB-DL h264][monkee].mkv
    Handle ID:      0x283c
    Resource Attributes:    S:AI

Process Information:
    Process ID:     0x4
    Process Name:       

Access Request Information:
    Accesses:       ReadData (or ListDirectory)

    Access Mask:        0x1
Rouzax commented 3 years ago

I addition I also see that the backlog search initiates a file system access.

An attempt was made to access an object.

Subject:
    Security ID:        HOME\SA-Medusa
    Account Name:       SA-Medusa
    Account Domain:     HOME
    Logon ID:       0x735D95E

Object:
    Object Server:      Security
    Object Type:        File
    Object Name:        E:\Data\TVSeries\The Haunting of Hill House
    Handle ID:      0x2360
    Resource Attributes:    S:AI

Process Information:
    Process ID:     0x4
    Process Name:       

Access Request Information:
    Accesses:       ReadAttributes

    Access Mask:        0x80
2020-10-18 11:03:30 DEBUG    BACKLOG :: [b352bb6] Retrieving the last check time from the DB
2020-10-18 11:03:30 INFO     BACKLOG :: [b352bb6] Running full backlog search on missed episodes for selected shows
2020-10-18 11:03:30 DEBUG    BACKLOG :: [b352bb6] Found needed backlog episodes for: The Haunting S02E02. Reason: Quality is not Preferred. Searching episode
2020-10-18 11:03:30 DEBUG    BACKLOG :: [b352bb6] Found needed backlog episodes for: The Haunting S02E04. Reason: Quality is not Preferred. Searching episode
2020-10-18 11:03:30 DEBUG    BACKLOG :: [b352bb6] Found needed backlog episodes for: The Haunting S02E08. Reason: Quality is not Preferred. Searching episode
2020-10-18 11:03:30 DEBUG    BACKLOG :: [b352bb6] Found needed backlog episodes for: The Haunting S02E09. Reason: Quality is not Preferred. Searching episode
2020-10-18 11:03:30 DEBUG    BACKLOG :: [b352bb6] Setting the last backlog in the DB to 737716
2020-10-18 11:03:30 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Beginning backlog search for: The Haunting
2020-10-18 11:03:30 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Checking for scene exception updates from https://cdn.pymedusa.com/scene_exceptions/scene_exceptions_tvdb.json
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] GET URL: https://cdn.pymedusa.com/scene_exceptions/scene_exceptions_tvdb.json [Status: 200]
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] User-Agent: Medusa/0.4.3 (Windows; 10; 9d561753-1057-11eb-b8a4-00155d02d203)
2020-10-18 11:03:31 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Checking for scene exception updates from https://cdn.pymedusa.com/scene_exceptions/scene_exceptions_tvmaze.json
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] GET URL: https://cdn.pymedusa.com/scene_exceptions/scene_exceptions_tvmaze.json [Status: 200]
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] User-Agent: Medusa/0.4.3 (Windows; 10; 9d561753-1057-11eb-b8a4-00155d02d203)
2020-10-18 11:03:31 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Checking for scene exception updates from https://cdn.pymedusa.com/scene_exceptions/scene_exceptions_tmdb.json
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] GET URL: https://cdn.pymedusa.com/scene_exceptions/scene_exceptions_tmdb.json [Status: 200]
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] User-Agent: Medusa/0.4.3 (Windows; 10; 9d561753-1057-11eb-b8a4-00155d02d203)
2020-10-18 11:03:31 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Checking for XEM scene exceptions updates for TVDBv2
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] GET URL: http://thexem.de/map/allNames?origin=tvdb&seasonNumbers=1 [Status: 200]
2020-10-18 11:03:31 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] User-Agent: Medusa/0.4.3 (Windows; 10; 9d561753-1057-11eb-b8a4-00155d02d203)
2020-10-18 11:03:31 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Checking for scene exceptions updates from AniDB
2020-10-18 11:03:32 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Transaction with 1 queries executed
2020-10-18 11:03:32 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Updated scene exceptions.
2020-10-18 11:03:32 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Updating exception_cache and exception_season_cache
2020-10-18 11:03:32 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Finished processing 3435 scene exceptions.
2020-10-18 11:03:32 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Building internal name cache for The Haunting
2020-10-18 11:03:32 DEBUG    SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Internal name cache for the haunting set to: the haunting of bly manor, the haunting
2020-10-18 11:03:32 INFO     SEARCHQUEUE-BACKLOG-345246 :: [b352bb6] Using backlog search providers
Rouzax commented 3 years ago

@medariox is this something you could look at?

medariox commented 3 years ago

Maybe in the future, but it's not a priority.

Rouzax commented 3 years ago

I understand