Support for non-OTA listings

[Qiangong2]

I'm trying to download listings for a Cable broadcast, but it seems like I can only download OTA broadcast listings

[daniel-widrick]

Greetings, Have you logged into https://tvlistings.zap2it.com/ and changed the "provider or Timezone" to the provider you are looking for in a web browser?

If that doesn't solve it let me know and we'll dig deeper.

[Qiangong2]

Yes, I have done that. Whatever timezone/postal code I set, it will only download OTA listinhs, not Cable listings

[daniel-widrick]

The following parameters seem to be culprits

&lineupId=USA-NY31651-DEFAULT
&headendId=NY31651

they are currently hard coded around: https://github.com/daniel-widrick/zap2it-GuideScraping/blob/7cde9e8671ecb6c96deb640dbdcefc3aefdc0c82/zap2it-GuideScrape.py#L47

We can turn those into config parameters... but given the circumstances i'm not sure about sane defaults or even if there's a good automatic way to extract them without the user digging through the dev console in chrome/firefox/etc.

Would you be able to locate the those values in the network call to: https://tvlistings.zap2it.com/api/grid? on the page: https://tvlistings.zap2it.com/?aid=gapzap and try running the script with those values plugged into line 45 and 47 of zap2it-GuideScrape.py and let me know if you get sane results?

[Qiangong2]

I tried it with the new values and the script just hung, no ouput at all

[timbenroeck]

I made the following changes to the BuildDataRequest function get non OTA:

 def BuildDataRequest(self,currentTime):
        parameters = {
            'Activity_ID': 1,
            'FromPage': "TV%20Guide",
            'AffiliateId': "gapzap",
            'token': self.zapToken,
            'aid': 'gapzap',
            'lineupId': self.config.get("prefs", "lineupId") ,
            'timespan': 3,
            'headendId': self.config.get("prefs", "headendId"),
            'country': self.config.get("prefs", "country"),
            'device': 'x',
            'postalCode': self.config.get("prefs", "zipCode"),
            'isOverride': "true",
            'time': currentTime,
            'pref': 'm,p',
            'userId': '-'
        }
        data = urllib.parse.urlencode(parameters)
        url = "https://tvlistings.zap2it.com/api/grid?" + data
        req = urllib.request.Request(url)
        return req

I then modified the INI file

Cable:

headendId: WA63873
lineupId: USA-WA63873-DEFAULT

OTA:

headendId: lineupId
lineupId: USA-lineupId-DEFAULT

Inspect the network traffic on the website to see the params for your specific city.

[daniel-widrick]

@timbenroeck Thank you for this. If you want to submit a pull request, i'll look at merging the changes asap. Otherwise, i'll eventually look into implementing your changes.

[Toei79]

this its working? where those data in the ini file are placed?

[creds] Username: xxx@gmail.com Password: xxxxxxxx [prefs] country: USA zipCode: 33023 historicalGuideDays: 7 lang: en headendId: DITV528 lineupId: USA-DITV528-DEFAULT

like this?

i copy the new BuildDataRequest on the script. zap2it-GuideScrape.txt

[daniel-widrick]

I have pushed some changes that should help smooth out some pain points related to supporting non-OTA providers. i have added an -f flag that will parse your config and return a list of providers with the associated headendId/lineupId/device

I have added an optional [lineup] section to the config.ini that will allow you to specify these parameters. d547c0190ec59a9bacb163f92ca639c126ff1670

Please let me know if you folks have any luck with the new version.