Watchdog + File Renamer - Co-operating with the agent

PowerModder commented 4 years ago

Hello. If it is inappropriate to write here for this just move it or delete it. I started to develop a watchdog application to detect and process files to a specific directory and based on their sub-directory (sub-folder) it detects the site (using the PAsearchSites.py), fetches the information back and then it uses them to rename the file. If you didn't understand here is a workflow of the watchdog:

Found a scene from Digital Playground.
Download and placed them to the directory: Watcher\Site's sub-folder (Watcher\Digital Playground)
When download is finished the file is getting processed 3a. Delete the 'quality' information from the filename (all network1services sites downloads now has a name of: a-title-herequality) so it just splits it at ("") if it exists. 3b. Search and Match Site's sub-folder using PAsearchSites array and defined methods (Co-operating) 3c. If match is successful it retrieves siteID, siteName, siteBaseURL, siteSearchURL from the PAsearchSites array 3d. Retrieves the scene's information and creates a new_filename which is being returned to the Watchdog .py
By using the new_filename and os functions it renames the downloaded file and moved them to the Plex directory.
Plex processes the new file at the library and matches it!
You just drink coffee :)

And here it is in action 100

Some notes and 'future' of this Watchdog:

Of course, you can edit the locations of where the watchdog is looking and where the files we move (DIRECTORY_TO_WATCH, DIRECTORY_TO_REST)
You can edit the renamer_network1service.py and chose what information will put at the title. So if you want a SiteName - Title - Date - Actors - Subsite you can do it, or if you want just sceneID you can SiteName - sceneID.
In the future, the above things will be arguments when you start the Watchdog. I am trying to learn more coding as I am newbie. The ideal will be to create a GUI with checkboxs for what you want to write to the title of the filename and if the site is providing to do so, to define the locations, how fast the watchdog will scan and of course a button to run it. Possibly a debug console and a log file. Yeah too much work!
For now it successfully renames the file IF it is downloaded from the official sites. It doesn't process the filename and removes 'trash' or in any other way. It will at the future, but for now the 'subscribers' of network1service sites don't have to do anything. Just simply download the file to the appropriate sub-folder of where the watchdog is active. Dummy tested to all network1services site and works perfectly.
Still needs some debugging before i alpha-alpha-alpha released it.

I wrote it here in case you are interested.

Supported sites (and their subsites)

[x] Brazzers
[x] Reality Kings
[x] Mofos
[x] Babes
[x] Twistys
[x] Digital Playground
[x] SexyHub
[x] FakeHub
[x] Property Sex
[x] FamilyHookups
[x] LookAtHerNow
[x] KinkySpa
[x] LilHumpers
[x] FamilySinners
[x] Bellesafilms
[x] MileHighMedia
[x] Erito
[x] TrueAmateurs
[x] SheWillCheat

PowerModder commented 4 years ago

I just open the repository if anyone wants to give a try to my Watchdog. https://github.com/PowerModder/Watchdog-Adult-Renamer

PowerModder commented 4 years ago

@DirtyRacer1337 If and when you have the time see if you want to add it as postprocessing2 inside here.

PowerModder commented 4 years ago

Added support for Bangbros and subsites too!

PowerModder commented 4 years ago

Added better log function, Support for Logs per searchTitle (video file) After finding all the matching scenes the Watchdog uses the one with the best score like Plex does and determines the name of the scene. However, as Phoenix-Plex does mistakes at matching so this program will do. Now at Log folders you will find a .log file with the searchTitle that that Watchdog used to match the scene. Inside there you can find a Debug log with all the matching scenes and at the end you will find ALL the generated names of all the matching scenes so you can copy-paste it.

DirtyRacer1337 commented 4 years ago

Interesting idea, I will merge if you make PR.

It also push me to idea combine your idea with #236, I already finished it. It's working, but not so good as I expect, also it's can clean title from file metadata. I can post it, if someone interested.

PowerModder commented 4 years ago

Hello @DirtyRacer1337 . I will make a pull request when it will be a little bit more ready. I am adding additional sites (I have added a bunch of them) and also i am trying to make the code more 'robust' if something fails so it will not crush just it will skip the file (yeah i am a noob). So far I tried to rename my already sorted media using my Watchdog and trying to rename some new 'dummy downloads'. The watchdog had a good overall accuracy (right name - matching) for all supported sites around 93%. But if others don't try i can't be sure. The problem for now is networkpornpros as it uses the DirectURL method. But they don't have a good 'download' name as they have the mp4_quality name for all scenes. For example, network1service had 100% accuracy so far. I will embed the googlesearch of PA too as i did to clean the title + searching. I want to fix also some baboones code i have wrote before I PR here. I am also very interested in making a GUI so you can start/stop and enter your preferences from there. Of course, it will be helpful if some of the users have access to some of those sites so they can provide me how they do the download-scheme name. For example: network1service sites now all are doing scenename-quality. pornpros is doing mp4_qt which can never work as you download. How naughtyamerica is doing download naming? How is Kink etc... An example of successful rename (based on my preferences). (Yeah logs are big xD ) File had the name FM - 9359 -> Placed into the folder of F* Machines. Watchdog scanned and see that it is inside this folder (for now it uses first the folderName as i believe this is how must people sort their collection for Plex usage. Folders with siteNames) Use the ID to find the data. Report at the scene-log... ** Current Scene Matching section ** ID: 9359 Title: Back in Black Part 2 of 5 Date: 2010-05-07 Actors: Tori Black Subsite: Site doesn't provide Subsite information Score: 100 ** Scene in Result but Mismatched ** If your scene was in results but the Watchdog mismatched it then use as filename the matched one from below F** Machines - 9359 ~ Tori Black ** Scene in Result but Mismatched ** The watchdog-log (as printed to console too) 2020-08-16 20:03:04,645 : Watchdog : INFO : Received modified event: ...\F*Machines\F*Machines - 9359.mp4 2020-08-16 20:03:04,645 : Watchdog : INFO : Processing filename F** - 9359 which is a type of .mp4 2020-08-16 20:03:04,645 : Watchdog : INFO : The file was placed at folder: F*Machines 2020-08-16 20:03:04,646 : Watchdog : INFO : Filename after initial process: F*Machines 9359 2020-08-16 20:03:04,646 : Watchdog : INFO : **** Process with PAsearchSites follows **** 2020-08-16 20:03:04,668 : Watchdog : INFO : searchTitle (after date processing): 9359 2020-08-16 20:03:04,668 : Watchdog : INFO : File didn't contain Date information. If this is false check the RegEx at PASearchSites for Dates 2020-08-16 20:03:04,668 : Watchdog : INFO : ** PAsearchSites matching ** 2020-08-16 20:03:04,668 : Watchdog : INFO : Use PAsearchSites to match F*Machines folder with a supported PA Site ID 2020-08-16 20:03:04,669 : Watchdog : INFO : PA Site ID: 506 2020-08-16 20:03:04,669 : Watchdog : INFO : PA Site Name: F* Machines 2020-08-16 20:03:04,669 : Watchdog : INFO : PA Site Base URL: http://www.kink.com 2020-08-16 20:03:04,669 : Watchdog : INFO : PA Site Search URL: http://www.kink.com/search?channelIds=f****machines&q= 2020-08-16 20:03:04,672 : Searcher : INFO : **** URL used section ** 2020-08-16 20:03:04,672 : Searcher : INFO : http://www.kink.com/shoot/9359 2020-08-16 20:03:06,379 : Searcher : INFO : Possible matching scenes found in results: 1 2020-08-16 20:03:06,382 : Searcher : INFO : *** Moving to Renamer Function * 2020-08-16 20:03:06,385 : Renamers : INFO : ***** Renamer Function *** 2020-08-16 20:03:06,386 : Renamers : INFO : Calculating new filename based on your preferences and the available information 2020-08-16 20:03:06,386 : Renamers : INFO : * After-Process filename section ***** 2020-08-16 20:03:06,386 : Renamers : INFO : The new filename is: F* Machines - 9359 ~ Tori Black.mp4 2020-08-16 20:03:06,386 : Renamers : INFO : **** Return to Watchdog **** 2020-08-16 20:03:06,387 : Watchdog : INFO : The site sub-folder was detected to ...\FMachines Sorted location. Try to move F Machines - 9359 ~ Tori Black.mp4 there 2020-08-16 20:03:06,390 : Watchdog : INFO : Successfully moved F* Machines - 9359 ~ Tori Black.mp4 to ...\FMachines Sorted location

DirtyRacer1337 commented 4 years ago

@PowerModder I readed your Readme file, I think I know how fix your issue, use from watchdog.observers.polling import PollingObserver as Observer instead just from watchdog.observers import Observer and handler like these.

class WatchFileHandler(FileSystemEventHandler):
    def on_any_event(self, event):
        if not event.is_directory:
            file_path = None
            if event.event_type == 'created' or event.event_type == 'modified':
                file_path = event.src_path
            elif event.event_type == 'moved':
                file_path = event.dest_path

            if file_path:
                file_path = Path(file_path)
                work_with_file(file_path)
        else:
            pass

That's should fix double event.

Also you can use pymediainfo and ffmpeg_python, to remove or change metadata title

def get_title_from_metadata(file_name):
    media_info = MediaInfo.parse(str(file_name))
    for track in media_info.tracks:
        if track.track_type == 'General':
            return track.title

    return None

def cleanup_metadata(file_name, new_file_name):
    stream = ffmpeg.input(str(file_name))
    stream = ffmpeg.output(
        stream,
        str(new_file_name),
        codec='copy',
        metadata='title='
    )
    ffmpeg.run(stream, quiet=True)
    file_name.unlink()

PowerModder commented 4 years ago

Thanks @DirtyRacer1337 . I will do some tests to see this first with double event.

PowerModder commented 4 years ago

Just a note: Double events are only important when you are going to sort already existing files via move to the Watchdog location. When you 'download' a file from browser you always have one created event which is the initialization of the file which is 0 byte and a mp4.part file which is being modified until you download (this is being ignored due to .part). When the download finishes you have ONE modified event to the .mp4 which as i understand is the move from .part to .mp4. So monitor while downloading is working great as is.

For the metadata I have a question-notice. I tried to see if Agent is using the metadata.title if they exist or the file_name. Basically I tried to have a filename which pointed out to one scene and another metadata title which pointed to another scene. Agent worked with the title all cases at my Plex Setup. (Phoenix Adult was top from Local Media Assets) If that is true then the only reason to change the metadata.title is to clean garbages and for better organize the files.

PowerModder commented 4 years ago

Seems the change for double events did the trick. Thanks! I also change the renamer function so date will be first instead of title or ID (for better sorting and to follow the PhoenixAdult suggested title recommended)

PAhelper / PhoenixAdult.bundle

Watchdog + File Renamer - Co-operating with the agent #916