TorrentGalaxy

[Tinuz97]

Hi, it looks that the domain TO is broken, or blocked in the netherlands. I changed it in the scraper (both addresses) to PW and now it is working again (returning links/results)

Could you add PW also (to help other people in the netherlands, because i have it working already) in a future update? Thx

[Tempest0580]

Yes I will fix that in the next update. Thanks

[Tinuz97]

1337x.to changed something also +- 30 minutes ago.

Domain TO not working anymore, TW working. Maybey they changed more, because i am getting no results back from the scraper (just fyi)

[Tempest0580]

Ok I'll try to take a look and see if I can fix them tonight. Thanks

[Tempest0580]

So i took a look at torrentquest and .pw site does not work for me but .com does. And 1337x.to works fine on my end.

I dont know why i thought it was torrentquest. im looking into torrentgalaxy now.

[Tinuz97]

Yeah TO started working yesterday again here also :)

[jewbmx]

If you guys are still looking at 1337x, my scraper is loaded with... self.domains = ['1337x.to', '1337x.st', '1337x.is', 'the1337x.org']

Should all still be valid right now too.

[Tempest0580]

I pushed a update, Still trying to figure out the issue with torrentgalaxy.

[Tinuz97]

At settings, library, I miss the icon (icon like all the others, screwdriver with a wrench), just cosmetic, section working fine (maybey because my kodi is set to Dutch language, no idea.), you miss the icon also, or is it just me?

[Tempest0580]

Thanks for the catch. it was caused by a typo. will be fixed in the next update.

[Tinuz97]

It looks that btdb (torrent scraper) is using CloudFare also now (website changes), so giving no results back (just fyi).

At settings, library,the icon is still missing after a update (also just fyi).

Greetings :)

[Tempest0580]

Ok I'll look into it. Been a little busy but I'll see if I can fix it and push a update sometime this weekend.

I have the icon fixed just been waiting to push it with other fixes.

[Tempest0580]

Pushed a update to fix BTDB scraper and the icon and a few other scrapers. Removed TorrentGalaxy until im able to fix it. Thanks

[Tinuz97]

https://btdb.eu/ changed something again i guess. I can browse the website link with my browser, but not getting results back in Tempest. (just fyi )

And thx for adding the missing icon 👍

[Tempest0580]

Ok I'll try to look at it tonight

[Tinuz97]

Working again with the update, thx 👍

[Tempest0580]

Your welcome.

[Tinuz97]

scenerls changed the way of the search (i think),so not returning sources. I wish i could fix it also,is there a manual anywhere? In the tempest scraper for scenerls the search is: self.search_link = '/search/%s'

if is do a search now on scenerls, the url in the browser is like this: http://scene-rls.net/?s=star+wars&submit=Find

But i have no skills to translate it to code :/ But i see that something has changed?

[Tempest0580]

I'll take a look at it. If I recall they changed the search once before.

[Tempest0580]

Pushed a update to fix it.

[Tinuz97]

Working 100%, thx.

[Tinuz97]

ddlspot looks gone also,it keeps hanging on that one for me. ddlspot.com not loading (website) and the alternative that i have found with google ,has no movies or tv shows (http://s1.ddlspot.com).

Maybey you can replace it for as a example https://www.scnsrc.me/ ?

This is the old one that i find in Artreides, but not working anymore, maybey you could repair and add it (just a idea):

-- coding: utf-8 --

#######################################################################

----------------------------------------------------------------------------

"THE BEER-WARE LICENSE" (Revision 42):

As long as you retain this notice you

can do whatever you want with this stuff. If we meet some day, and you think

this stuff is worth it, you can buy me a beer in return. - Muad'Dib

----------------------------------------------------------------------------

#######################################################################

Addon Name: Atreides

Addon id: plugin.video.atreides

Addon Provider: House Atreides

3/30/2019 - Sent to me from another addon dev, credit on their work (asked to be left anon)

5/28/2019 - Fixed

import re import traceback import urllib import urlparse import requests

from resources.lib.modules import cleantitle, client, control, debrid, dom_parser2, log_utils, source_utils, workers

class source: def init(self): self.priority = 1 self.source = ['www'] self.domains = ['scnsrc.me'] self.base_link = 'https://www.scnsrc.me'

def movie(self, imdb, title, localtitle, aliases, year):
    try:
        url = {'imdb': imdb, 'title': title, 'year': year}
        url = urllib.urlencode(url)
        return url
    except Exception:
        failure = traceback.format_exc()
        log_utils.log('ScnSrc - Exception: \n' + str(failure))
        return

def tvshow(self, imdb, tvdb, tvshowtitle, localtvshowtitle, aliases, year):
    try:
        url = {'imdb': imdb, 'tvdb': tvdb, 'tvshowtitle': tvshowtitle, 'year': year}
        url = urllib.urlencode(url)
        return url
    except Exception:
        failure = traceback.format_exc()
        log_utils.log('ScnSrc - Exception: \n' + str(failure))
        return

def episode(self, url, imdb, tvdb, title, premiered, season, episode):
    try:
        if url is None:
            return

        url = urlparse.parse_qs(url)
        url = dict([(i, url[i][0]) if url[i] else (i, '') for i in url])
        url['title'], url['premiered'], url['season'], url['episode'] = title, premiered, season, episode
        url = urllib.urlencode(url)
        return url
    except Exception:
        failure = traceback.format_exc()
        log_utils.log('ScnSrc - Exception: \n' + str(failure))
        return

def sources(self, url, hostDict, hostprDict, sc_timeout):
    try:
        self._sources = []
        if url is None:
            return self._sources

        data = urlparse.parse_qs(url)
        data = dict([(i, data[i][0]) if data[i] else (i, '') for i in data])

        query = '%s S%02dE%02d' % (
            data['tvshowtitle'],
            int(data['season']),
            int(data['episode'])) if 'tvshowtitle' in data else '%s %s' % (
            data['title'],
            data['year'])
        query = re.sub('(\\\|/| -|:|;|\*|\?|"|\'|<|>|\|)', ' ', query)

        query = cleantitle.geturl(query)
        url = urlparse.urljoin(self.base_link, query)

        self.timer = control.Time(start=True)

        shell = requests.Session()

        headers = {
            'Referer': url,
            'User-Agent':
            'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0'}
        r = shell.get(url, headers=headers)
        r = r.headers['Location']
        r = shell.get(r).content
        posts = dom_parser2.parse_dom(r, 'li', {'class': re.compile('.+?'), 'id': re.compile('comment-.+?')})
        self.hostDict = hostDict + hostprDict
        threads = []

        for i in posts:
            threads.append(workers.Thread(self._get_sources, i.content, sc_timeout))
        [i.start() for i in threads]
        [i.join() for i in threads]

        return self._sources
    except Exception:
        return self._sources

def _get_sources(self, item, sc_timeout):
    try:
        links = dom_parser2.parse_dom(item, 'a', req='href')
        links = [i.attrs['href'] for i in links]
        info = []
        try:
            size = re.findall('((?:\d+\.\d+|\d+\,\d+|\d+)\s*(?:GiB|MiB|GB|MB))', item)[0]
            div = 1 if size.endswith('GB') else 1024
            size = float(re.sub('[^0-9|/.|/,]', '', size)) / div
            size = '%.2f GB' % size
            info.append(size)
        except Exception:
            pass
        info = ' | '.join(info)
        for url in links:
            # Stop searching 8 seconds before the provider timeout, otherwise might continue searching, not complete in time, and therefore not returning any links.
            if self.timer.elapsed() > sc_timeout:
                log_utils.log('Scnsrc - Timeout Reached')
                break

            if 'youtube' in url:
                continue
            if any(x in url for x in ['.rar.', '.zip.', '.iso.']) or any(
                    url.endswith(x) for x in ['.rar', '.zip', '.iso']):
                        raise Exception()
            valid, host = source_utils.is_host_valid(url, self.hostDict)
            if not valid:
                continue
            host = client.replaceHTMLCodes(host)
            host = host.encode('utf-8')
            quality, info2 = source_utils.get_release_quality(url, url)
            if url in str(self._sources):
                continue
            self._sources.append(
                {'source': host, 'quality': quality, 'language': 'en', 'url': url, 'info': info, 'direct': False,
                 'debridonly': debrid.status()})
    except Exception:
        pass

def resolve(self, url):
    return url

[Tempest0580]

Ddlspot works fine on my end. It pull a good amount of sources.

And that scraper for scnsrc was mine that I created. But it is down now because of Craptcha so it cant be fixed at this time.

On Sun, Oct 27, 2019, 9:34 AM Tinuz97 notifications@github.com wrote:

ddlspot looks gone also,it keeps hanging on that one for me. ddlspot.com not loading (website) and the alternative that i have found with google ,has no movies or tv shows (http://s1.ddlspot.com).

Maybey you can replace it for as a example https://www.scnsrc.me/ ?

This is the old one that i find in Artreides, but not working anymore, maybey you could repair and add it (just a idea): -- coding: utf-8 --

#######################################################################

---------------------------------------------------------------------------- "THE BEER-WARE LICENSE" (Revision 42): As long as you retain this notice you can do whatever you want with this stuff. If we meet some day, and you think this stuff is worth it, you can buy me a beer in return. - Muad'Dib

####################################################################### Addon Name: Atreides Addon id: plugin.video.atreides Addon Provider: House Atreides 3/30/2019 - Sent to me from another addon dev, credit on their work (asked to be left anon) 5/28/2019 - Fixed

import re import traceback import urllib import urlparse import requests

from resources.lib.modules import cleantitle, client, control, debrid, dom_parser2, log_utils, source_utils, workers

class source: def init(self): self.priority = 1 self.source = ['www'] self.domains = ['scnsrc.me'] self.base_link = 'https://www.scnsrc.me'

def movie(self, imdb, title, localtitle, aliases, year): try: url = {'imdb': imdb, 'title': title, 'year': year} url = urllib.urlencode(url) return url except Exception: failure = traceback.format_exc() log_utils.log('ScnSrc - Exception: \n' + str(failure)) return

def tvshow(self, imdb, tvdb, tvshowtitle, localtvshowtitle, aliases, year): try: url = {'imdb': imdb, 'tvdb': tvdb, 'tvshowtitle': tvshowtitle, 'year': year} url = urllib.urlencode(url) return url except Exception: failure = traceback.format_exc() log_utils.log('ScnSrc - Exception: \n' + str(failure)) return

def episode(self, url, imdb, tvdb, title, premiered, season, episode): try: if url is None: return

    url = urlparse.parse_qs(url)
    url = dict([(i, url[i][0]) if url[i] else (i, '') for i in url])
    url['title'], url['premiered'], url['season'], url['episode'] = title, premiered, season, episode
    url = urllib.urlencode(url)
    return url
except Exception:
    failure = traceback.format_exc()
    log_utils.log('ScnSrc - Exception: \n' + str(failure))
    return

def sources(self, url, hostDict, hostprDict, sc_timeout): try: self._sources = [] if url is None: return self._sources

    data = urlparse.parse_qs(url)
    data = dict([(i, data[i][0]) if data[i] else (i, '') for i in data])

    query = '%s S%02dE%02d' % (
        data['tvshowtitle'],
        int(data['season']),
        int(data['episode'])) if 'tvshowtitle' in data else '%s %s' % (
        data['title'],
        data['year'])
    query = re.sub('(\\\|/| -|:|;|\*|\?|"|\'|<|>|\|)', ' ', query)

    query = cleantitle.geturl(query)
    url = urlparse.urljoin(self.base_link, query)

    self.timer = control.Time(start=True)

    shell = requests.Session()

    headers = {
        'Referer': url,
        'User-Agent':
        'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0'}
    r = shell.get(url, headers=headers)
    r = r.headers['Location']
    r = shell.get(r).content
    posts = dom_parser2.parse_dom(r, 'li', {'class': re.compile('.+?'), 'id': re.compile('comment-.+?')})
    self.hostDict = hostDict + hostprDict
    threads = []

    for i in posts:
        threads.append(workers.Thread(self._get_sources, i.content, sc_timeout))
    [i.start() for i in threads]
    [i.join() for i in threads]

    return self._sources
except Exception:
    return self._sources

def _get_sources(self, item, sc_timeout): try: links = dom_parser2.parse_dom(item, 'a', req='href') links = [i.attrs['href'] for i in links] info = [] try: size = re.findall('((?:\d+.\d+|\d+\,\d+|\d+)\s*(?:GiB|MiB|GB|MB))', item)[0] div = 1 if size.endswith('GB') else 1024 size = float(re.sub('[^0-9|/.|/,]', '', size)) / div size = '%.2f GB' % size info.append(size) except Exception: pass info = ' | '.join(info) for url in links:

Stop searching 8 seconds before the provider timeout, otherwise might continue searching, not complete in time, and therefore not returning any links.

        if self.timer.elapsed() > sc_timeout:
            log_utils.log('Scnsrc - Timeout Reached')
            break

        if 'youtube' in url:
            continue
        if any(x in url for x in ['.rar.', '.zip.', '.iso.']) or any(
                url.endswith(x) for x in ['.rar', '.zip', '.iso']):
                    raise Exception()
        valid, host = source_utils.is_host_valid(url, self.hostDict)
        if not valid:
            continue
        host = client.replaceHTMLCodes(host)
        host = host.encode('utf-8')
        quality, info2 = source_utils.get_release_quality(url, url)
        if url in str(self._sources):
            continue
        self._sources.append(
            {'source': host, 'quality': quality, 'language': 'en', 'url': url, 'info': info, 'direct': False,
             'debridonly': debrid.status()})
except Exception:
    pass

def resolve(self, url): return url

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/Tempest0580/tempest/issues/12?email_source=notifications&email_token=AHCRXJJIW44YOAPC2OVFS6DQQWDFRA5CNFSM4IL6LEV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECK6QBI#issuecomment-546695173, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHCRXJPW3435NZG5Y2RUU5LQQWDFRANCNFSM4IL6LEVQ .

[Tinuz97]

Weird, maybey blocked in the netherlands (ddlspot). Will try later with maybey a vpn enabled.

[Tinuz97]

Just tested with pia (private internet access vpn). It looks like the netherlands is blocked, because also with vpn (Netherlands server), no acces.

If I turn the vpn on America, I have acces. Is there a way around for the scraper (add a proxy link maybey?as extra), because i don't like using a vpn (and could help other people in the netherlands without vpn), just a idea.

[Tempest0580]

I'll look into it

[Tinuz97]

Torrentquest same, vpn on UK or America, then it works. Weird, even on my phone (4G) in the netherlands, both are blocked, I think Torrentquest and ddlspot are blocking The Netherlands, also Belgium and Germany (tested vpn on those 3).

[Tinuz97]

TorrentQuest and DDLspot started both working again,i think they maybey removed the block for The Netherlands, Germany, Belgium (maybey others, tested those 3 with vpn). Now i can acces the website(s), and getting results back in Tempest 💯 (without vpn)

[Tempest0580]

That's great news. Glad to hear it.