ArionMiles / Zooqle-qBittorrent-Plugin

Zooqle search plugin for qBittorrent
MIT License
7 stars 3 forks source link

The zooqle plugin, unlike the website behavior, doesn't seem to search in file names. #6

Open Pireo opened 7 years ago

Pireo commented 7 years ago

Try to search "russian" for example. Then order the results by size in both. One of the good things about zooqle is the fact that it does search in file names, and qBT has an option to filter them anyway.

dwilbanks commented 7 years ago

This version is really a messy and untested version, but fixes the search results for Russia. It was actually unrelated to looking inside the torrent. This version has had zero emphasis on version 2.7. I have no idea of it will work or not. It's really messy code now, I've been focused on making it go, there is a lot of artifacts from version that things I worked on that failed and I didn't get around to cleaning up.

RE: Russia. The it does not like having the "all" category specified on the URL. If QB sends category all, I just don't add it to the URL.

Enhancement list: 000.0 If it fails. It tells you why. Duh! It dumps all logging information to the results page with invalid links, but the text is good. I also now have a modified version of nova2 that throws display messages too.

  1. Preface any search with ! like "!Russia" and it will display logging information inside your search results, not important for the end user but way helpful for the developer.

  2. It reads the local web server if it's enabled, and checks which torrents you already have. If you have them it marks them with 'HAVE:'.

  3. It starts to keep an configuration file, and reads the qBittorrent.ini or qBittorrent.conf for information about how it should behave.

  4. Not even started yet, but, using the web api, I can force the C++ portion of qBittorrent to download the torrents instead of allowing the work to be done in python. Python maxes out the threads and times out without any failure messages. I've added a thousand torrents in a in a single throw using the web api and it does not miss a beat. That part's still not done, (or really even started) you'll still have to limit your downloads to 20 or 30 at a time.

  5. The number of pages to read is now a configuration item. Instead of hard coding the arbitrary value of 10.

  6. Future. Max Results, configuration item. Not sure which should take priority, number of pages or number of results, I guess i need some input.

  7. Setting and displaying configuration using search criteria IE: "!!Show Zoogle Config!!" or "!!Set Zoogle Config MaxPages=20!!" something like that. I'm writing the results to a zoogle.ini value in the engines directory. We can go wild with it.

  8. Future: Proxy use, maybe using round robin? I use fiddler whenever I'm doing web or http work. It's so helpful to be able to look at the history of what is actually sent back and forth between the client and server.

#VERSION: 1.14
#AUTHOR: Arion_Miles (https://github.com/ArionMiles/)
#Contributors: affaff (https://github.com/affaff)
#LICENSE: MIT License
# QBWebClient class imported from mookfist/python-qbittorrent
# Oops, I forget where the new one comes from, I'll need to grab a link

import traceback
import os.path
import gzip
import requests
import json

from io import StringIO
from xml.dom import minidom
from novaprinter import prettyPrinter

try:
    from urllib2 import urlopen, Request, URLError
    from urllib import unquote
except ImportError:
    from urllib.request import urlopen, Request, URLError
    from urllib.parse import unquote

user_agent = 'Mozilla/5.0 (X11; Linux i686; rv:38.0) Gecko/20100101 Firefox/38.0'
headers    = {'User-Agent': user_agent}

class LoginRequired(Exception):
    def __str__(self):
        return 'Please login first.'

class v1k45Client(object):
    """class to interact with qBittorrent WEB API"""
    def __init__(self, url):
        if not url.endswith('/'):
            url += '/'
        self.url = url

        session = requests.Session()
        check_prefs = session.get(url+'query/preferences')

        if check_prefs.status_code == 200:
            self._is_authenticated = True
            self.session = session

        elif check_prefs.status_code == 404:
            self._is_authenticated = False
            raise RuntimeError("""
                This wrapper only supports qBittorrent applications
                 with version higher than 3.1.x.
                 Please use the latest qBittorrent release.
                """)

        else:
            self._is_authenticated = False

    def _get(self, endpoint, **kwargs):
        """
        Method to perform GET request on the API.
        :param endpoint: Endpoint of the API.
        :param kwargs: Other keyword arguments for requests.
        :return: Response of the GET request.
        """
        return self._request(endpoint, 'get', **kwargs)

    def _post(self, endpoint, data, **kwargs):
        """
        Method to perform POST request on the API.
        :param endpoint: Endpoint of the API.
        :param data: POST DATA for the request.
        :param kwargs: Other keyword arguments for requests.
        :return: Response of the POST request.
        """
        return self._request(endpoint, 'post', data, **kwargs)

    def _request(self, endpoint, method, data=None, **kwargs):
        """
        Method to hanle both GET and POST requests.
        :param endpoint: Endpoint of the API.
        :param method: Method of HTTP request.
        :param data: POST DATA for the request.
        :param kwargs: Other keyword arguments.
        :return: Response for the request.
        """
        final_url = self.url + endpoint

        if not self._is_authenticated:
            raise LoginRequired

        rq = self.session
        if method == 'get':
            request = rq.get(final_url, **kwargs)
        else:
            request = rq.post(final_url, data, **kwargs)

        request.raise_for_status()
        request.encoding = 'utf_8'

        if len(request.text) == 0:
            data = json.loads('{}')
        else:
            try:
                data = json.loads(request.text)
            except ValueError:
                data = request.text

        return data

    def login(self, username='admin', password='admin'):
        """
        Method to authenticate the qBittorrent Client.
        Declares a class attribute named ``session`` which
        stores the authenticated session if the login is correct.
        Else, shows the login error.
        :param username: Username.
        :param password: Password.
        :return: Response to login request to the API.
        """
        self.session = requests.Session()
        login = self.session.post(self.url+'login',
                                  data={'username': username,
                                        'password': password})
        if login.text == 'Ok.':
            self._is_authenticated = True
        else:
            return login.text

    def logout(self):
        """
        Logout the current session.
        """
        response = self._get('logout')
        self._is_authenticated = False
        return response

    @property
    def qbittorrent_version(self):
        """
        Get qBittorrent version.
        """
        return self._get('version/qbittorrent')

    @property
    def api_version(self):
        """
        Get WEB API version.
        """
        return self._get('version/api')

    @property
    def api_min_version(self):
        """
        Get minimum WEB API version.
        """
        return self._get('version/api_min')

    def shutdown(self):
        """
        Shutdown qBittorrent.
        """
        return self._get('command/shutdown')

    def torrents(self, **filters):
        """
        Returns a list of torrents matching the supplied filters.
        :param filter: Current status of the torrents.
        :param category: Fetch all torrents with the supplied label.
        :param sort: Sort torrents by.
        :param reverse: Enable reverse sorting.
        :param limit: Limit the number of torrents returned.
        :param offset: Set offset (if less than 0, offset from end).
        :return: list() of torrent with matching filter.
        """
        params = {}
        for name, value in filters.items():
            # make sure that old 'status' argument still works
            name = 'filter' if name == 'status' else name
            params[name] = value

        return self._get('query/torrents', params=params)

    def get_torrent(self, infohash):
        """
        Get details of the torrent.
        :param infohash: INFO HASH of the torrent.
        """
        return self._get('query/propertiesGeneral/' + infohash.lower())

    def get_torrent_trackers(self, infohash):
        """
        Get trackers for the torrent.
        :param infohash: INFO HASH of the torrent.
        """
        return self._get('query/propertiesTrackers/' + infohash.lower())

    def get_torrent_webseeds(self, infohash):
        """
        Get webseeds for the torrent.
        :param infohash: INFO HASH of the torrent.
        """
        return self._get('query/propertiesWebSeeds/' + infohash.lower())

    def get_torrent_files(self, infohash):
        """
        Get list of files for the torrent.
        :param infohash: INFO HASH of the torrent.
        """
        return self._get('query/propertiesFiles/' + infohash.lower())

    @property
    def global_transfer_info(self):
        """
        Get JSON data of the global transfer info of qBittorrent.
        """
        return self._get('query/transferInfo')

    @property
    def preferences(self):
        """
        Get the current qBittorrent preferences.
        Can also be used to assign individual preferences.
        For setting multiple preferences at once,
        see ``set_preferences`` method.
        Note: Even if this is a ``property``,
        to fetch the current preferences dict, you are required
        to call it like a bound method.
        Wrong::
            qb.preferences
        Right::
            qb.preferences()
        """
        prefs = self._get('query/preferences')

        class Proxy(v1k45Client):
            """
            Proxy class to to allow assignment of individual preferences.
            this class overrides some methods to ease things.
            Because of this, settings can be assigned like::
                In [5]: prefs = qb.preferences()
                In [6]: prefs['autorun_enabled']
                Out[6]: True
                In [7]: prefs['autorun_enabled'] = False
                In [8]: prefs['autorun_enabled']
                Out[8]: False
            """

            def __init__(self, url, prefs, auth, session):
                super(Proxy, self).__init__(url)
                self.prefs = prefs
                self._is_authenticated = auth
                self.session = session

            def __getitem__(self, key):
                return self.prefs[key]

            def __setitem__(self, key, value):
                kwargs = {key: value}
                return self.set_preferences(**kwargs)

            def __call__(self):
                return self.prefs

        return Proxy(self.url, prefs, self._is_authenticated, self.session)

    def sync(self, rid=0):
        """
        Sync the torrents by supplied LAST RESPONSE ID.
        Read more @ http://git.io/vEgXr
        :param rid: Response ID of last request.
        """
        return self._get('sync/maindata', params={'rid': rid})

    def download_from_link(self, link, **kwargs):
        """
        Download torrent using a link.
        :param link: URL Link or list of.
        :param savepath: Path to download the torrent.
        :param category: Label or Category of the torrent(s).
        :return: Empty JSON data.
        """
        # old:new format
        old_arg_map = {'save_path': 'savepath'}  # , 'label': 'category'}

        # convert old option names to new option names
        options = kwargs.copy()
        for old_arg, new_arg in old_arg_map.items():
            if options.get(old_arg) and not options.get(new_arg):
                options[new_arg] = options[old_arg]

        options['urls'] = link

        # workaround to send multipart/formdata request
        # http://stackoverflow.com/a/23131823/4726598
        dummy_file = {'_dummy': (None, '_dummy')}

        return self._post('command/download', data=options, files=dummy_file)

    def download_from_file(self, file_buffer, **kwargs):
        """
        Download torrent using a file.
        :param file_buffer: Single file() buffer or list of.
        :param save_path: Path to download the torrent.
        :param label: Label of the torrent(s).
        :return: Empty JSON data.
        """
        if isinstance(file_buffer, list):
            torrent_files = {}
            for i, f in enumerate(file_buffer):
                torrent_files.update({'torrents%s' % i: f})
        else:
            torrent_files = {'torrents': file_buffer}

        data = kwargs.copy()

        if data.get('save_path'):
            data.update({'savepath': data['save_path']})
        return self._post('command/upload', data=data, files=torrent_files)

    def add_trackers(self, infohash, trackers):
        """
        Add trackers to a torrent.
        :param infohash: INFO HASH of torrent.
        :param trackers: Trackers.
        """
        data = {'hash': infohash.lower(),
                'urls': trackers}
        return self._post('command/addTrackers', data=data)

    @staticmethod
    def _process_infohash_list(infohash_list):
        """
        Method to convert the infohash_list to qBittorrent API friendly values.
        :param infohash_list: List of infohash.
        """
        if isinstance(infohash_list, list):
            data = {'hashes': '|'.join([h.lower() for h in infohash_list])}
        else:
            data = {'hashes': infohash_list.lower()}
        return data

    def pause(self, infohash):
        """
        Pause a torrent.
        :param infohash: INFO HASH of torrent.
        """
        return self._post('command/pause', data={'hash': infohash.lower()})

    def pause_all(self):
        """
        Pause all torrents.
        """
        return self._get('command/pauseAll')

    def pause_multiple(self, infohash_list):
        """
        Pause multiple torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/pauseAll', data=data)

    def set_label(self, infohash_list, label):
        """
        Set the label on multiple torrents.
        IMPORTANT: OLD API method, kept as it is to avoid breaking stuffs.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        data['label'] = label
        return self._post('command/setLabel', data=data)

    def set_category(self, infohash_list, category):
        """
        Set the category on multiple torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        data['category'] = category
        return self._post('command/setCategory', data=data)

    def resume(self, infohash):
        """
        Resume a paused torrent.
        :param infohash: INFO HASH of torrent.
        """
        return self._post('command/resume', data={'hash': infohash.lower()})

    def resume_all(self):
        """
        Resume all torrents.
        """
        return self._get('command/resumeAll')

    def resume_multiple(self, infohash_list):
        """
        Resume multiple paused torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/resumeAll', data=data)

    def delete(self, infohash_list):
        """
        Delete torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/delete', data=data)

    def delete_permanently(self, infohash_list):
        """
        Permanently delete torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/deletePerm', data=data)

    def recheck(self, infohash_list):
        """
        Recheck torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/recheck', data=data)

    def increase_priority(self, infohash_list):
        """
        Increase priority of torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/increasePrio', data=data)

    def decrease_priority(self, infohash_list):
        """
        Decrease priority of torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/decreasePrio', data=data)

    def set_max_priority(self, infohash_list):
        """
        Set torrents to maximum priority level.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/topPrio', data=data)

    def set_min_priority(self, infohash_list):
        """
        Set torrents to minimum priority level.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/bottomPrio', data=data)

    def set_file_priority(self, infohash, file_id, priority):
        """
        Set file of a torrent to a supplied priority level.
        :param infohash: INFO HASH of torrent.
        :param file_id: ID of the file to set priority.
        :param priority: Priority level of the file.
        """
        if priority not in [0, 1, 2, 7]:
            raise ValueError("Invalid priority, refer WEB-UI docs for info.")
        elif not isinstance(file_id, int):
            raise TypeError("File ID must be an int")

        data = {'hash': infohash.lower(),
                'id': file_id,
                'priority': priority}

        return self._post('command/setFilePrio', data=data)

    # Get-set global download and upload speed limits.

    def get_global_download_limit(self):
        """
        Get global download speed limit.
        """
        return self._get('command/getGlobalDlLimit')

    def set_global_download_limit(self, limit):
        """
        Set global download speed limit.
        :param limit: Speed limit in bytes.
        """
        return self._post('command/setGlobalDlLimit', data={'limit': limit})

    global_download_limit = property(get_global_download_limit,
                                     set_global_download_limit)

    def get_global_upload_limit(self):
        """
        Get global upload speed limit.
        """
        return self._get('command/getGlobalUpLimit')

    def set_global_upload_limit(self, limit):
        """
        Set global upload speed limit.
        :param limit: Speed limit in bytes.
        """
        return self._post('command/setGlobalUpLimit', data={'limit': limit})

    global_upload_limit = property(get_global_upload_limit,
                                   set_global_upload_limit)

    # Get-set download and upload speed limits of the torrents.
    def get_torrent_download_limit(self, infohash_list):
        """
        Get download speed limit of the supplied torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/getTorrentsDlLimit', data=data)

    def set_torrent_download_limit(self, infohash_list, limit):
        """
        Set download speed limit of the supplied torrents.
        :param infohash_list: Single or list() of infohashes.
        :param limit: Speed limit in bytes.
        """
        data = self._process_infohash_list(infohash_list)
        data.update({'limit': limit})
        return self._post('command/setTorrentsDlLimit', data=data)

    def get_torrent_upload_limit(self, infohash_list):
        """
        Get upoload speed limit of the supplied torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/getTorrentsUpLimit', data=data)

    def set_torrent_upload_limit(self, infohash_list, limit):
        """
        Set upload speed limit of the supplied torrents.
        :param infohash_list: Single or list() of infohashes.
        :param limit: Speed limit in bytes.
        """
        data = self._process_infohash_list(infohash_list)
        data.update({'limit': limit})
        return self._post('command/setTorrentsUpLimit', data=data)

    # setting preferences
    def set_preferences(self, **kwargs):
        """
        Set preferences of qBittorrent.
        Read all possible preferences @ http://git.io/vEgDQ
        :param kwargs: set preferences in kwargs form.
        """
        json_data = "json={}".format(json.dumps(kwargs))
        headers = {'content-type': 'application/x-www-form-urlencoded'}
        return self._post('command/setPreferences', data=json_data,
                          headers=headers)

    def get_alternative_speed_status(self):
        """
        Get Alternative speed limits. (1/0)
        """
        return self._get('command/alternativeSpeedLimitsEnabled')

    alternative_speed_status = property(get_alternative_speed_status)

    def toggle_alternative_speed(self):
        """
        Toggle alternative speed limits.
        """
        return self._get('command/toggleAlternativeSpeedLimits')

    def toggle_sequential_download(self, infohash_list):
        """
        Toggle sequential download in supplied torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/toggleSequentialDownload', data=data)

    def toggle_first_last_piece_priority(self, infohash_list):
        """
        Toggle first/last piece priority of supplied torrents.
        :param infohash_list: Single or list() of infohashes.
        """
        data = self._process_infohash_list(infohash_list)
        return self._post('command/toggleFirstLastPiecePrio', data=data)

    def force_start(self, infohash_list, value=True):
        """
        Force start selected torrents.
        :param infohash_list: Single or list() of infohashes.
        :param value: Force start value (bool)
        """
        data = self._process_infohash_list(infohash_list)
        data.update({'value': json.dumps(value)})
        return self._post('command/setForceStart', data=data)    

def retrieve_url_nodecode(url):
    """ Return the content of the url page as a string """
    req = Request(url, headers = headers)
    try:
        response = urlopen(req)
    except URLError as errno:
        print(" ".join(("Connection error:", str(errno.reason))))
        print(" ".join(("URL:", url)))
        return ""
    dat = response.read()
    # Check if it is gzipped
    if dat[:2] == '\037\213':
        # Data is gzip encoded, decode it
        compressedstream = StringIO(dat)
        gzipper = gzip.GzipFile(fileobj=compressedstream)
        extracted_data = gzipper.read()
        dat = extracted_data
        return dat
    return dat

class zooqle(object):
    """ Search engine class """
    url = 'https://zooqle.com'
    name = 'Zooqle'
    cfg = {
        "WebUI.Enabled":False,
        "WebUI.HTTPS.Enabled":False,
        "WebUI.LocalHostAuth":True,
        "WebUI.Username":"admin",
        "WebUI.Password":"admin",
        "WebUI.Port":"8080",
        "Connection.Proxy.IP":"127.0.0.1",
        "Connection.Proxy.Password":"",
        "ignoreHave":False,
        "zeroByteValid":True,
        "LabelHave":True,
        "zeroByteUseMagnet":True,
        "MaxResults":300,
        "MaxPages":10,
    }    
    torrentIHave = {}
    logText = ""
    proxyConfig = {}
    failureMessagesSent = 0
    logmode = 0
    supported_categories = {'all'       : 'all',
                            'movies'    : 'Movies',
                            'tv'        : 'TV',
                            'music'     : 'Music',
                            'games'     : 'Games',
                            'anime'     : 'Anime',
                            'software'  : 'Apps',
                            'books'     : 'Books',
                            'others'  : 'Other'}
    def log(self ,logText):
        if( self.logmode==1):
            self.oneDummyResult(logText)
        self.logText = self.logText + logText + "\n";

    def setProp(self, propName, propValue):
        propName.replace("\\",".")
        if not(propName in self.cfg):
            return False
        if( type(self.cfg[propName]) == bool):
            self.cfg[propName] = self.isTrue(propValue)
        elif( type(self.cfg[propName]) == int):
            self.cfg[propName] = int(propValue)
        else:
            self.cfg[propName] = propValue

    def writeLog(self):
        if( self.logmode==0):
            multiLineArr = self.logText.split("\n")
            for eachLine in multiLineArr:
                self.oneDummyResult(eachLine)

        logName = self.name.lower() + '.log';
        dirpath = os.path.dirname(os.path.realpath(__file__))
        logName = os.path.join(dirpath,logName)
        logFile = open( logName, 'a')
        logFile.write("-------");
        logFile.write(self.logText);
        logFile.write("-------");
        logFile.close()

    def isTrue(self,value):
        if( str(value).lower( )=='true'):
            return True
        if( str(value).lower( )=='yes'):
            return True
        if( str(value).lower( )=='no'):
            return False
        if( str(value).lower( )=='false'):
            return False
        if( str(value)!='0'):
            return True
        return False

    def readQBINI(self):
        appData = os.getenv('APPDATA')
        if( appData == None):
            iniName = "~/.config/qBittorrent/qBittorrent.conf"
        else:
            iniName = os.path.join(os.getenv('APPDATA'),"qBittorrent","qBittorrent.ini")
        if not(os.path.isfile(iniName)):
            self.log( "failed to read qBittorrent configuration")
            return False
        iniFile = open( iniName, 'r')
        content =iniFile.read() 
        linesArr = content.split( "\n")
        for eachLine in linesArr:
            EachLineContent =eachLine.split("=")
            if( len(EachLineContent)==2):
                self.setProp(EachLineContent[0], EachLineContent[1])

    def prop(self,name):
        if (name in self.cfg):
            return self.cfg[name]
        return False

    def readConfig(self):
        self.readQBINI()
        cfgName = self.name.lower() + '.cfg';
        dirpath = os.path.dirname(os.path.realpath(__file__))
        cfgName = os.path.join(dirpath,cfgName)
        self.log('configuration file name is %s' % (cfgName))
        if not(os.path.isfile(cfgName)):
            self.log('configuration file %s does not exist' % (cfgName))
            cfgFile = open( cfgName, 'w')
            for propName in self.cfg:
                cfgFile.write(propName+ "=" + str( self.cfg[propName]) +"\n");
            cfgFile.close()
            return
        cfgFile = open( cfgName, 'r')
        content =cfgFile.read() 
        linesArr = content.split( "\n")
        for eachLine in linesArr:
            EachLineContent =eachLine.split("=")
            if( len(EachLineContent)==2):
                self.setProp(EachLineContent[0], EachLineContent[1])
        cfgFile.close()
        return

    def xmldata(self,item, tagName, attrName = None):
        if attrName == None:
            return item.getElementsByTagName(tagName)[0].childNodes[0].data
        return item.getElementsByTagName(tagName)[0].attributes[attrName].value

    def readOnePage(self, url):
        response = retrieve_url_nodecode(url)
        xmldoc = minidom.parseString(response)
        itemlist = xmldoc.getElementsByTagName('item')
        if( len(itemlist ) ==0):
            return False
        totalResultVal  = int(self.xmldata(xmldoc, 'opensearch:totalResults'))
        startIndex  = int(self.xmldata(xmldoc, 'opensearch:startIndex'))
        itemsPerPage = int(self.xmldata(xmldoc, 'opensearch:itemsPerPage'))
        self.log('reading remote Page %s total resuls %d startindex = %d itemsPerPage = %d' % (url, totalResultVal, startIndex, itemsPerPage))

        for item in itemlist:

            bthash = self.xmldata(item,'torrent:infoHash').lower()
            name = self.xmldata(item,'title')
            size = self.xmldata(item, 'enclosure','length')
            haveIt = ( bthash in self.torrentIHave)
            ignore = False;
            if haveIt and self.prop("ignoreHave") :
                ignore = True
            if size == "0"  and not(self.prop("zeroByteValid")):
                ignore = True
            if not( ignore ):
                if( haveIt):
                    name = "HAVE:"+name
                if( size=='0' and self.prop("zeroByteUseMagnet")):
                    link = self.xmldata(item, 'torrent:magnetURI')
                else:
                    link =self.xmldata(item, 'enclosure','url')
                desc =self.xmldata(item, 'link') 
                leech = self.xmldata(item, 'torrent:peers')
                if not leech.isdigit():
                    leech = ''
                seeds = self.xmldata(item, 'torrent:seeds')
                if not seeds.isdigit():
                    seeds = ''                    
                prettyPrinter({
                    "engine_url" : self.url,
                    'name' : name,
                    'size' : size,
                    'link' : link,                    
                    'desc_link' : desc,                    
                    'size' : size,                    
                    'leech' : leech,
                    'seeds' : seeds,
                })
        if(  startIndex  + itemsPerPage > totalResultVal ):
            return False
        return True
    def oneDummyResult(self, outputText):
        self.failureMessagesSent = self.failureMessagesSent +1
        lineNum = '{:04d}'.format(self.failureMessagesSent) 
        logtext = self.name + " LOG " + lineNum + " "+ outputText

        prettyPrinter({
            "engine_url" : self.url,
            "link" : self.url,
            "desc_link" : self.url,
            "name" : logtext,
            "size" : "",
            "leech" : "",
            "seeds" : ""

                       })

    def readWebList(self):
        if( self.prop("WebUI.HTTPS.Enabled")):
            server = 'https://localhost:' + self.prop("WebUI.Port") +  '/'
        else:
            server = 'http://localhost:' + self.prop("WebUI.Port") +  '/'

        self.log('reading local web server '+server)
        qb = v1k45Client(server)
        if( self.prop("WebUI.LocalHostAuth")):
            qb.login(self.prop("WebUI.Username"), self.prop("WebUI.Password"))
        else:
            self.log('no login is required')
        try:
            torrents = qb.torrents()
        except LoginRequired:
            self.oneDummyResult("Login to your local web server failed")
            return False
        self.log("found " +str(len(torrents) ) + " torrents on your local server")

        for eachTorrent in torrents:
            bthash = eachTorrent['hash']
            size = eachTorrent['size']
            self.torrentIHave[bthash] = size

    def search(self, what, cat="all"):
        try:
            if (what[:3] =="%21"):
                self.logmode =1 
                what = what[3:]
            realwhat = unquote(what);
            self.log("Searching for '%s' in '%s'" % (realwhat, cat) )
            self.readConfig()
            if( self.prop("LabelHave") and self.prop("WebUI.Enabled")):
                self.readWebList()
            """ Performs search """
            page = 1
            while page <= self.prop("MaxPages"):
                realCat = self.supported_categories[cat];
                query = "https://zooqle.com/search?q="+what
                if( realCat !="all"):
                    query = query +"category%3A".realCat
                query = query + "&fmt=rss"
                if( page>1 ):
                    query = query + "&pg=" + str (page)
                if( not (self.readOnePage(query))):
                    return
                page += 1

            return                
        except: # catch *all* exceptions
            tbText = traceback.format_exc()
            self.log("=================")
            self.log("===== FATAL ERROR")
            self.log(tbText)
            self.log("=================")
            self.writeLog()          
Pireo commented 7 years ago

@dwilbanks and torrentproject.se is doable? Also it has RSS, and I'm thinking about proposing it as official. P.s. any updates about the zooqle issue?

dwilbanks commented 7 years ago

torrentproject.se has some anti scraping software on their edge servers. When you go there for the first time, or clear your cookies it does something... I don't know what. I have not looked into it. A quick check from my current IP allowed me to search using rss and download using without any special cookies, but, I know they've got some limits that I've bumped into a lot.

The RSS only gave me 15 links and no ability to get a next page, so it was kinda useless.

I've got an engine that really reduces the effort for searches, that mess that they've been using in all of the plugins is garbage, If I was to do it, I'd just scrape the normal pages.

I'm working on getting all the search engines to work with a common library, fixing broken engines and adding logging to everything.

Pireo commented 7 years ago

ok.