Automatic package download

[scottc]

Can we subscribe to a search query, to have all packages download automatically apon discovery?

So when I get home from work, all my desired tv shows or movies will be automatically downloaded.

Or if i decided to download a previously aired tv show, but needed to download 200 episodes, it would be nice to be able to queue them all matching the search query, with one button click.

[Freedos]

If you know python you can try to write a Plugin for Sickbeard and Couchpotato. It already has proper TV show handling. You just need to use XGs API. I tried to write one but because of of my lag of time and knowledge didn't produced anything usefull.

http://sickbeard.com/ https://couchpota.to/

[scottc]

I don't know python, I know C# and JavaScript, would be easier for me to make a push request.

I would rather this feature to piggyback of the existing search functionality and events, without introducing additional dependencies. I guess it depends on how Lars wants to architect XG, I'll wait and see what Lars recommends. I think there are several ways you can go with this feature.

[lformella]

It is not as easy as it looks, because there are some points you have to keep in mind.

• At the moment searches can contain duplicates. It is hard to detect duplicates, because two packets could be the same even if the size, name or file extension might be different.
• At the moment XG does not remember packets which were already downloaded. Even if these names are saved, there is the duplicate name issue.
• Automatic downloading could fill up your disk with a lot of files if you forget to disable it.

It would be a lot easier to just put an button at the search table, which would download / enqueue all matching packets. So it would be your job to filter out already downloaded files by using optimized search terms. Maybe it would be useful to ad an extended search filter for the last mentioned date or file size.

[scottc]

I had a bit more of a think about it, duplicates would be in issue depending on how you are using XG, for example which bots you are targeting and your search.

We could do an equality comparision between the packet name and size, but you would have things like the same movie from different release groups, which will differ in size and name. I doubt anyone would want to download the same bluray movie by 4 different release groups.

Would be upto the user to have a search filter aggressive enough to target a specific release group etc. But then you could run into duplicates if the file was renammed for example replacing spaces with underscores, or someone modified the file to contain subtitles or downscaled the movie to be 480p or some similar scenario.

So we would need to implement a bit of fussy logic around the file name, but too much effort to get it working most of the time for most of the users etc.

However the duplicates won't be an issue in a scenario where, you are specifically targeting the bots owned by the release group themselves, because they would use a sane namming convention and probably wont release a file containning the same name with different content. Which is how i'm using it.

I guess this a niche feature and will not be used by the majority of users, so i will just write a plugin, by subscribing to the events which are exposed via the abstract APlugin class.

I think download history would be nice to have however, I also noticed in the Files view it does not display the queued files, only the IRC view displays queued files. Another suggestion I have is for a RESTful webserver API. I should probably put all these suggestions in seperate threads. But I feel like I've made too many threads already.

[Freedos]

If you're still interested here's a input and output plugin I've written for XDCCgrabscher in Flexget:

input:

from __future__ import unicode_literals, division, absolute_import
import logging
import sqlite3

from flexget import plugin
from flexget.entry import Entry
from flexget.event import event
from flexget.utils.cached_input import cached

log = logging.getLogger('input_xdcc')

class input_xdcc(object):
    """
    Adds support for XDCCdb created by XDCCgrabscher. 
    Configuration format:

    input_xdcc: <file>
    """

    schema = {
        'type': 'object',
        'properties': {
            'file': {'type': 'string', 'format': 'file'},
        },
        'required': ['file'],
        'additionalProperties': False
    }

    @cached('input_xdcc')
    def on_task_input(self, task, config):
        entries = []
        connection = sqlite3.connect(config['file'])

        with connection:

            cursor = connection.cursor()

            cursor.execute("""
                            SELECT Packet.Name, Packet.Id, Packet.Size, Packet.LastUpdated, Packet.LastMentioned, Bot.Name, Channel.Name, Server.Name
                            FROM Packet, Bot, Channel, Server 
                            WHERE Bot.Guid=Packet.ParentGuid AND
                                  Channel.Guid=Bot.ParentGuid AND
                                  Server.Guid=Channel.ParentGuid""")

            while True:
##                entry = Entry()
                row = cursor.fetchone()
                if row == None:
                    break

                Server = row[7]
                ChannelName = row[6]
                Bot = row[5]
                PacketId = row[1]
                PacketName = row[0]
                Size = row[2]
                LastMentioned = row[4]
                XDCClink = Server + "/" + Server + "/" + ChannelName[1:] + "/" + Bot + "/" + str(PacketId) + "/" + row [0] + "/"

##                entry.name = PacketName
##                entry.title = PacketName
##                entry.url = XDCClink
##                entry.content_size = Size
##                entry.rss_pubdate = LastMentioned

                entries.append(Entry(PacketName, XDCClink))

        return entries

@event('plugin.register')
def register_plugin():
    plugin.register(input_xdcc, 'input_xdcc', api_ver=2)

output:

from __future__ import unicode_literals, division, absolute_import
from logging import getLogger

from flexget import plugin
from flexget.event import event
from flexget.utils.requests import get

log = getLogger('xdccgrabscher')

class OutputXDCCgrabscher(object):
    """
    Example:

    xdccgrabscher:
      url: http://localhost:5556/api/yourapihere/parseXdccLink/

    """
    schema = {
        'type': 'object',
        'properties': {
            'url': {'type': 'string', 'format': 'url'},
        },
        'required': ['url'],
        'additionalProperties': False
    }

    def on_task_output(self, task, config):
        for entry in task.accepted:
            if task.options.test:
                log.info('Would add into XDCCgrabscher: %s' % entry['title'])
                continue
            request_url = config['url'] + entry['url']
            log.debug('request_url: %s' % request_url)
            try:
                response = get(request_url)
            except Exception as e:
                log.critical('Failed to use XDCCgrabscher. Requested %s' % request_url)
                log.critical('Result was: %s' % e)
                entry.fail('XDCCgrabscher unreachable')
                if task.options.debug:
                    log.exception(e)
                continue

@event('plugin.register')
def register_plugin():
    plugin.register(OutputXDCCgrabscher, 'xdccgrabscher', api_ver=2)

here is an example config file for series downloading:

tasks:
  test:
    input_xdcc:
      file: PathToXG\xgobjects.db
    series:
      - a series name
    xdccgrabscher:
      url: http://localhost:5556/api/XXXXXXXXXXXX/parseXdccLink/

You need to specify XDCCgrabscher squlite db for the input plugin and the api adress with key for the output plugin.

more info: flexget.com

[lformella]

You should not use the sqlite file, because it will be removed in the next version. Sqlite ist to slow when loading and saving, so I switched to DB4O which at least doubles the performance.

If you need access to servers, channels, bots, packets or even files and searches, this should be added in the api itself. Please create a new issue and describe which new api calls your are needing.

[scottc]

I also think this feature would add too much bloat and is bad design (unix philosophy: programs should only do one thing and do it well). The responsability should be given to an external app or plugin. As per the sickbeard/flexget suggestion.

Scheduling/automation of packages vs downloading packages.

[Freedos]

First off all.

THANKS FOR XG 3.3 !!!!! I've been looking for something like this for years!!!

With the new API + flexget + filebot + my full automatic media-center is just in reach.

here's my search and output plugin for flexget: http://pastebin.com/kKf5Ln4V http://pastebin.com/XkzAYv0D

NOTICE!!!!

• The output plugin is only working with the input_XG plugin since it's not using XDCC links but XGs guid.
• code is a big mess (what are objects?!)
• but it's working with no issues (at least in the last hour ;-))

You'll need a nice config file here's a quick one for testing: http://pastebin.com/vWBLegxp

When the plugin reaches v1 I will push it to flexget repo Have fun!!! Time for sleep. Good I don't have to work tomorow.