[bug] Not working on windows?

[Mirabis]

Fresh clone and set-up, i'm always getting:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main
    prepare(preparation_data)
  File "C:\Python27\lib\multiprocessing\forking.py", line 510, in prepare
    '__parents_main__', file, path_name, etc
  File "C:\Users\Administrator\AppData\Local\PlexAutoScan\scan.py", line 58, in <module>
    mngr = Manager()
  File "C:\Python27\lib\multiprocessing\__init__.py", line 99, in Manager
    m.start()
  File "C:\Python27\lib\multiprocessing\managers.py", line 524, in start
    self._process.start()
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 258, in __init__
    cmd = get_command_line() + [rhandle]
  File "C:\Python27\lib\multiprocessing\forking.py", line 358, in get_command_line
    is not going to be frozen to produce a Windows executable.''')
RuntimeError:
            Attempt to start a new process before the current process
            has finished its bootstrapping phase.

            This probably means that you are on Windows and you have
            forgotten to use the proper idiom in the main module:

                if __name__ == '__main__':
                    freeze_support()
                    ...

            The "freeze_support()" line can be omitted if the program
            is not going to be frozen to produce a Windows executable.
Traceback (most recent call last):
  File "scan.py", line 58, in <module>
    mngr = Manager()
  File "C:\Python27\lib\multiprocessing\__init__.py", line 99, in Manager
    m.start()
  File "C:\Python27\lib\multiprocessing\managers.py", line 528, in start
    self._address = reader.recv()
EOFError

[Mirabis]

Managed to get it working with some google and re-arranging the code:

#!/usr/bin/env python2.7
import json
import logging
import os
import sys
import time
from logging.handlers import RotatingFileHandler
from multiprocessing import Process, freeze_support, Lock, Manager

from flask import Flask
from flask import abort
from flask import request

# Get config
import config
# local imports
import db
import plex
import utils
############################################################
# INIT
############################################################

# Logging
logFormatter = logging.Formatter('%(asctime)24s - %(levelname)8s - %(name)9s [%(process)5d]: %(message)s')
rootLogger = logging.getLogger()
rootLogger.setLevel(logging.INFO)

# Decrease modules logging
logging.getLogger('requests').setLevel(logging.ERROR)
logging.getLogger('werkzeug').setLevel(logging.ERROR)
logging.getLogger("peewee").setLevel(logging.ERROR)

# Console logger, log to stdout instead of stderr
consoleHandler = logging.StreamHandler(sys.stdout)
consoleHandler.setFormatter(logFormatter)
rootLogger.addHandler(consoleHandler)

# Load initial config
conf = config.Config()

# File logger
fileHandler = RotatingFileHandler(
    conf.settings['logfile'],
    maxBytes=1024 * 1024 * 5,
    backupCount=5
)
fileHandler.setFormatter(logFormatter)
rootLogger.addHandler(fileHandler)

# Set configured log level
rootLogger.setLevel(conf.settings['loglevel'])
# Load config file
conf.load()

# Scan logger
logger = rootLogger.getChild("AUTOSCAN")

############################################################
# QUEUE PROCESSOR
############################################################

def queue_processor():
    logger.info("Starting queue processor in 10 seconds")
    try:
        time.sleep(10)
        logger.info("Queue processor started")
        db_scan_requests = db.get_all_items()
        items = 0
        for db_item in db_scan_requests:
            scan_process = Process(target=plex.scan, args=(
                conf.configs, scan_lock, db_item['scan_path'], db_item['scan_for'], db_item['scan_section'],
                db_item['scan_type'], resleep_paths))
            scan_process.start()
            items += 1
            time.sleep(2)
        logger.info("Restored %d scan requests from database", items)
    except Exception:
        logger.exception("Exception while processing scan requests from database: ")
    return

############################################################
# FUNCS
############################################################

def start_scan(path, scan_for, scan_type):
    section = utils.get_plex_section(conf.configs, path)
    if section <= 0:
        return False
    else:
        logger.debug("Using section id: %d for '%s'", section, path)

    if conf.configs['SERVER_USE_SQLITE']:
        db_exists, db_file = db.exists_file_root_path(path)
        if not db_exists and db.add_item(path, scan_for, section, scan_type):
            logger.info("Added '%s' to database, proceeding with scan", path)
        else:
            logger.info(
                "Already processing '%s' from same folder, aborting adding an extra scan request to the queue", db_file)
            resleep_paths.append(db_file)
            return False
    scan_process = Process(target=plex.scan,
                           args=(conf.configs, scan_lock, path, scan_for, section, scan_type, resleep_paths))
    scan_process.start()
    return True

def start_queue_reloader():
    queue_process = Process(target=queue_processor, args=())
    queue_process.start()
    return True

############################################################
# SERVER
############################################################

app = Flask(__name__)
app.config['JSON_AS_ASCII'] = False

@app.route("/%s" % conf.configs['SERVER_PASS'], methods=['GET'])
def manual_scan():
    if not conf.configs['SERVER_ALLOW_MANUAL_SCAN']:
        return abort(401)
    page = '<html><body>' \
           '<form action="" method="post"> File to be scanned:<br>' \
           '<input type="text" name="filepath" value=""> ' \
           '<input type="hidden" name="eventType" value="Manual"> ' \
           '<br><br><input type="submit" value="Submit"></form> ' \
           '<p>Clicking submit will add this file to the scan backlog.</p></body></html>'
    return page, 200

@app.route("/%s" % conf.configs['SERVER_PASS'], methods=['POST'])
def client_pushed():
    if request.content_type == 'application/json':
        data = request.get_json(silent=True)
    else:
        data = request.form.to_dict()

    if not data:
        logger.error("Invalid scan request from: %r", request.remote_addr)
        abort(400)
    logger.debug("Client %r request dump:\n%s", request.remote_addr, json.dumps(data, indent=4, sort_keys=True))

    if ('eventType' in data and data['eventType'] == 'Test') or ('EventType' in data and data['EventType'] == 'Test'):
        logger.info("Client %r made a test request, event: '%s'", request.remote_addr, 'Test')
    elif 'eventType' in data and data['eventType'] == 'Manual':
        logger.info("Client %r made a manual scan request for: '%s'", request.remote_addr, data['filepath'])
        final_path = utils.map_pushed_path(conf.configs, data['filepath'])
        # ignore this request?
        ignore, ignore_match = utils.should_ignore(final_path, conf.configs)
        if ignore:
            logger.info("Ignored scan request for '%s' because '%s' was matched from SERVER_IGNORE_LIST", final_path,
                        ignore_match)
            return "Ignoring scan request because %s was matched from your SERVER_IGNORE_LIST" % ignore_match
        if start_scan(final_path, 'manual', 'Manual'):
            return "'%s' was added to scan backlog." % final_path
        else:
            return "Error adding '%s' to scan backlog." % data['filepath']

    elif 'Movie' in data:
        logger.info("Client %r scan request for movie: '%s', event: '%s'", request.remote_addr,
                    data['Movie']['FilePath'], data['EventType'])
        final_path = utils.map_pushed_path(conf.configs, data['Movie']['FilePath'])
        start_scan(final_path, 'radarr', data['EventType'])
    elif 'movie' in data:
        # new radarr webhook
        path = os.path.join(data['movie']['folderPath'], data['movieFile']['relativePath'])
        logger.info("Client %r scan request for movie: '%s', event: '%s'", request.remote_addr, path,
                    "Upgrade" if data['isUpgrade'] else data['eventType'])
        final_path = utils.map_pushed_path(conf.configs, path)
        start_scan(final_path, 'radarr', "Upgrade" if data['isUpgrade'] else data['eventType'])
    elif 'Series' in data:
        logger.info("Client %r scan request for series: '%s', event: '%s'", request.remote_addr, data['Series']['Path'],
                    data['EventType'])
        final_path = utils.map_pushed_path(conf.configs, data['Series']['Path'])
        start_scan(final_path, 'sonarr', data['EventType'])
    elif 'series' and 'episodeFile' in data:
        # new sonarr webhook
        path = os.path.join(data['series']['path'], data['episodeFile']['relativePath'])
        logger.info("Client %r scan request for series: '%s', event: '%s'", request.remote_addr, path,
                    "Upgrade" if data['isUpgrade'] else data['eventType'])
        final_path = utils.map_pushed_path(conf.configs, path)
        start_scan(final_path, 'sonarr_dev', "Upgrade" if data['isUpgrade'] else data['eventType'])
    else:
        logger.error("Unknown scan request from: %r", request.remote_addr)
        abort(400)

    return "OK"

############################################################
# MAIN
############################################################

if __name__ == "__main__":
    freeze_support()
    # Multiprocessing
    scan_lock = Lock()
    mngr = Manager()
    resleep_paths = mngr.list()
    if conf.args['cmd'] == 'sections':
        plex.show_sections(conf.configs)

    elif conf.args['cmd'] == 'server':
        if conf.configs['SERVER_USE_SQLITE']:
            start_queue_reloader()

        logger.info("Starting server: http://%s:%d/%s",
                    conf.configs['SERVER_IP'],
                    conf.configs['SERVER_PORT'],
                    conf.configs['SERVER_PASS']
                    )
        app.run(host=conf.configs['SERVER_IP'], port=conf.configs['SERVER_PORT'], debug=False, use_reloader=False)
        logger.info("Server stopped")

    else:
        logger.error("Unknown command...")

[l3uddz]

Sorry for the late response, been a busy couple months. The new version no longer uses the multiprocessing module, it instead uses threads so that I was able to add priority capabilities to queued items.

If you have a free moment to test these changes on Windows, would be greatly appreciated.

[Mirabis]

Updated, will check if it works -- will report back

[Mirabis]

""" 2018-02-03 20:11:40,598 - INFO - PLEX [ 7296]: Scan request is now being processed 2018-02-03 20:11:40,598 - INFO - PLEX [ 7296]: Starting Plex Scanner 2018-02-03 20:11:40,615 - INFO - UTILS [ 7296]: '""C:\Program' is not recognized as an internal or external command, 2018-02-03 20:11:40,615 - INFO - UTILS [ 7296]: operable program or batch file. 2018-02-03 20:11:40,615 - INFO - PLEX [ 7296]: Finished scan! """

Unfortunately still some issues with the path, even though following the manual

[Mirabis]

Changing line 70 from:

   final_cmd = '""%s" --scan --refresh --section %s --directory "%s""' 

to

final_cmd = '"%s" --scan --refresh --section %s --directory ""%s"'  

Fixes it.

[l3uddz]

Ah thanks for testing and finding solution, will make that change now.

[l3uddz]

Is the extra quote mark required for the scan directory?

You mentioned '"%s" --scan --refresh --section %s --directory ""%s"'

Does '"%s" --scan --refresh --section %s --directory "%s"' also work for you? If so will make it the latter.

[Mirabis]

Apologies got notification of your last commit, but your additional question seems to have slipped my mind.

I'm pretty sure I tried your suggestion and had no issues, but just forgot to reply. I will check later today whatever my script currently says.

[l3uddz]

thanks man, much appreciated!

[Mirabis]

My local version is using

final_cmd = '"%s" --scan --refresh --section %s --directory "%s"' \

So suppose single quote is fine :)

[danielloader]

Sorry to dig up an old issue but rather than open a new one, is the latest version of the script windows compatible too? @Mirabis