Closed Dis90 closed 3 years ago
I tested the version with .com support before you reverted and I got it load the frontpage link by adding
headers = { 'X-disco-params': 'realm=go,siteLookupKey=dplus_us' }
data = json.loads(self.make_request(url, 'get', params=params, headers=headers))
It then lists categories but nothing appears underneath those catagories. Is there any data I can collect for you to try and add .com support? I can use fiddler everywhere to capture data when browsing discoveryplus.com if that would help.
I can get all json listings by using VPN but most difficult part is that I can't see what actual page looks like because I don't have account. Can you take couple screenshots of homepage and other pages where shows and categories is listed?
I used few hours to this and conclusion is that US version does many things little bit different than other versions. Because I don't have account to US it is unfortunately impossible for me add support for discoveryplus.com
Do these screenshots help at all? Can you post an example of the json files from a different region so they can be compared to the US version?
Yes those help a lot. It looks much different than EU versions.
Go to browse while developer tools open and search for file https://us1-prod-direct.discoveryplus.com/cms/routes/shows What params and headers is used? Headers are probably at least "x-disco-params: realm=go,siteLookupKey=dplus_us" is there anything else. And what about params? Those should be at least "include: default"
x-disco-params: realm=go,siteLookupKey=dplus_us x-disco-client: WEB:UNKNOWN:dplus_us:0.0.1 are the only headers that stand out (everything else is standard headers)
include: default decorators: viewingHistory,isFavorite are the params
Can you go to some series page, example Gold Rush and look from developer tools something similar than /cms/routes/shows and save that .txt and paste here. What I'm looking for is file that contains details from shows. For some reason discovery+ US doesn't seem to include those on /shows
https://us1-prod-direct.discoveryplus.com/cms/routes/shows?include=default&decorators=viewingHistory,isFavorite This is when clicking Browse
https://us1-prod-direct.discoveryplus.com/cms/collections/296530408644496528769435570882544597228?include=default&decorators=viewingHistory,isFavorite This is when clicking on the Food Network logo to just show Food Network shows
https://us1-prod-direct.discoveryplus.com/cms/routes/show/diners-drive-ins-and-dives?include=default&decorators=viewingHistory,isFavorite This is when choosing Diners, Drive-ins and Dives show
https://us1-prod-direct.discoveryplus.com/playback/v3/videoPlaybackInfo This is when choosing Episode 1 from Season 1 and the player loads
Hopefully these .txt files help. The URL show for each .txt is the URL that gets loaded for that JSON file
Thanks that helps a lot.
When you go to Food Network is there tabs named Trending, A-Z, Cooking competitions..? Can you click one of those and post url and content.
There are tabs named Trending, A-Z etc but it does not load any new files while clicking on those
Okay that explains why I can't get those to work.
Actually that wasn't problem. Problem is that without login in for example Food Network listing is incomplete. All shows are missing just categories is listed. But I can use your files for testing.
When I go to address https://us1-prod-direct.discoveryplus.com/cms/routes/show/diners-drive-ins-and-dives?include=default&decorators=viewingHistory,isFavorite I got this error. Is there some headers used?
show with id diners-drive-ins-and-dives was filtered by validator, reasonCode=10
x-disco-params: realm=go,siteLookupKey=dplus_us x-disco-client: WEB:UNKNOWN:dplus_us:0.0.1
those are the 2 headers, it looks like x-disco-params is not required to get that page to load but x-disco-client is required from my testing
Yeah some pages needs x-disco-params and some not. After adding x-disco-client to requests I now get full responses. Can you get urls and content of search and my list?
https://us1-prod-direct.discoveryplus.com/cms/routes/my-list?include=default&decorators=viewingHistory,isFavorite https://us1-prod-direct.discoveryplus.com/cms/routes/search?include=default&decorators=viewingHistory,isFavorite https://us1-prod-direct.discoveryplus.com/cms/routes/search/result?include=default&decorators=viewingHistory,isFavorite&contentFilter[query]=mythbusters
I created branch dplus_us https://github.com/Dis90/plugin.video.discoveryplus/tree/dplus_us Could you try does login and video playback work? You know that login didn't work if you find line "Error logging in. Invalid Arkose token." from debug log.
Probably not working features:
Video playback is not working. I tried watching episode 1 of airplane repo and get
2021-01-12 15:28:09.863 T:2724 DEBUG
No stream URL for video exists is the error I get inside Kodi as well.
So v2 videoPlayback doesn't exists .com. Try now.
2021-01-12 15:57:01.979 T:19252 DEBUG
Testing in browser, videoPlaybackInfo is a POST request with this JSON data
{ "deviceInfo": { "adBlocker": true, "hdrCapabilities": [ "SDR" ], "hwDecodingCapabilities": [], "player": { "width": 2630, "height": 1826 }, "screen": { "width": 3840, "height": 2160 }, "soundCapabilities": [ "STEREO" ] }, "videoId": "89315", "wisteriaProperties": { "advertiser": { "adId": "|92223167458192398905978408829301668405||161048906055868be2eed88099f0d06cc1209b623385b", "firstPlay": 0, "fwDid": "", "fwIsLat": 0, "fwNielsenAppId": "P5A0FD4DE-4AE6-4B22-811B-36B9BD091980", "gpaln": "" }, "appBundle": "", "device": { "language": "en", "make": "", "model": "", "name": "Chrome", "os": "Windows", "osVersion": "NT 10.0", "type": "desktop", "id": "2608fb0ce6c5d22839529b5f6019bb94" }, "platform": "desktop", "product": "dplus_us", "sessionId": "e029a1a5-27ed-4856-b1ba-e0281b86ece9", "streamProvider": { "suspendBeaconing": 0, "hlsVersion": 7, "pingConfig": 0 } } }
Made some changes and video playback appears to work.
get_current_episode_info needs to send headers along with request
make_request i changed else: # post req = self.http_session.post(url, params=params, data=payload, headers=headers, json=jsonpayload) so that JSON can be sent with POST
this is the JSON that gets sent with the POST to get the video m3u8 {'deviceInfo': {'adBlocker': 'true'},'videoId': video_id,'wisteriaProperties':{'product':'dplus_us'}}
if self.locale_suffix == 'us': stream['hls_url'] = data_dict['attributes']['streaming'][0]['url'] stream['drm_enabled'] = data_dict['attributes']['streaming'][0]['protection']['drmEnabled']
needed [0] added
sorry this is kind of messy, just wanted to let you see. if you want a pull request instead i can do that
Please do pull request :)
I'm not sure yet how I pull this changes from dplus_us branch to master and matrix after everything US related is done. I'm little bit newbie with Git.
get_current_episode_info needs to send headers along with request
I guess get_next_episode_info also needs headers.
Did update_playback_progress work? It is used when playback is stopped.
Also can you check what url and parameters is used when adding and deleting favorites. In other regions it is {api_url}/users/me/favorites/shows/{show_id}
Sorry for the delay, got the pull request created.
update_playback_progress does work, when I stopped a video it updated on the discovery plus website. Resuming videos doesn't work though. If I go to play a video, it will ask if I want to start at the beginning or the time that was left off, but it always starts at the beginning.
I will check adding/deleting favorites today. Also looking into an why some shows won't work, so far it seems like shows in the magnolia preview give an error when trying to show the episodes. If i can't figure out why I'll post some logs
EDIT: endpoint for favorites is the same and its POST for adding and DELETE for deleting
Thanks for pull request.
update_playback_progress does work, when I stopped a video it updated on the discovery plus website. Resuming videos doesn't work though. If I go to play a video, it will ask if I want to start at the beginning or the time that was left off, but it always starts at the beginning.
Weird it should work if it suggests to resume.
Also looking into an why some shows won't work, so far it seems like shows in the magnolia preview give an error when trying to show the episodes. If i can't figure out why I'll post some logs
Can you post name of those shows?
I tried with Magnolia Table and Super Dad.
Here is snippet of log from Super Dad. It errors when doing Browse->Magnolia Network Preview->Trending->Super Dad-Episodes
2021-01-16 12:50:21.062 T:20088 DEBUG
Testing resume again, I started a show in Kodi, skipped ahead and then stopped. Started again and it resumed properly. Stopped it again and resumed it on discoveryplus.com. Skipped ahead on the site and stopped playback. Loaded the show in Kodi again, it shows the same resume point as the website, but when I resume it resumes at the spot I had stopped last in Kodi. So it seems like there is a resume time in the Kodi database that is overriding the one from the site, even though the addon is showing the proper resume time from the site.
If it works properly for you, are you using Kodi Leia or Matrix?
I wonder if there is a way to update the database resume time before starting playback?
Season listing should now be fixed.
I actually rarely watch anything using discovery+ website. So that can be reason why I haven't found this issue. Have to do tests but reason is probably resume time in Kodi database.
I wonder if there is a way to update the database resume time before starting playback?
Maybe thru Kodi json rpc?
jsonrpc worked. submitted pull request to add that
season listing works now for the episodes that didn't work before. i think everything is working now. can you think of anything else that should be tested?
Thanks. I'm thinking that is maybe bug in Kodi. Something similar was fixed by Netflix plugin author in Kodi Matrix. But that bug caused episodes to start from beginning.
Can you go to url https://us1-prod-direct.discoveryplus.com/users/me/favorites?include=default and check is your favorites listed there? That listing is used in addon to check if show is already added to favorites or not (context menu options).
Dplus_us decorators in requests is "viewingHistory, isFavorite" and that makes me think is US version using different method to check if show is added to favorites.
https://us1-prod-direct.discoveryplus.com/users/me/favorites
I have one show in my favorites, Airplane Repo, and it shows up on that page.
I tested adding a show to favorites through context menu and originally got an error. When I updated add_or_delete_favorite to send self.site_headers, it worked. So that needs to be updated to send headers.
Did find another bug. When listing a category A-Z, it doesn't list all the shows. It looks like the 1st 16 shows are listed, but no more and no option to load page 2 or anything.
There's no pagination in addon. Params 'page[items.number]': 1, 'page[items.size]': 100 worked in Finnish discovery+ to get more items to listings.
Please check what params website uses to load more content. It is possible that US version doesn't support larger listings than 16 items. Parameter page[items.size] is not used by any discovery+ websites but I guessed it since 'page[size]' was used in first version of Dplay website.
Initial loading of /shows (Browse) it loads https://us1-prod-direct.discoveryplus.com/cms/collections/263961364688425944705602256853549055046?include=default&decorators=viewingHistory,isFavorite which had the 1st 16 shows for each different channel.
Scrolling then loads https://us1-prod-direct.discoveryplus.com/cms/collections/92495789997044244789744437364785494204?include=default&decorators=viewingHistory,isFavorite&page[items.number]=2&page[items.size]=16 (92495789997044244789744437364785494204 changes depending on which channel you choose)
I tested and items.size max is 100
I tested and items.size max is 100
Does it display more than 16 shows?
Yes if I use items.size=100 it shows 100 shows
That is weird why addon doesn't get more than 16 shows although page[items.size] is 100? I don't understand.
I noticed in the logs that when going to browse->HGTV->A-Z in kodi, this is what's being accessed
XFILE::CPluginDirectory::StartScript - calling plugin discovery+('plugin://plugin.video.discoveryplus/','6','?action=list_collection_items&collection_id=161374980142915215278619398379069832686&collection_id_main=263961364688425944705602256853549055046','resume:false')
https://us1-prod-direct.discoveryplus.com/cms/collections/263961364688425944705602256853549055046 is then used. This URL only has the first 16 episodes listed, no matter if items.size is passed along. But https://us1-prod-direct.discoveryplus.com/cms/collections/161374980142915215278619398379069832686?include=default&page[items.number]=1&page[items.size]=100 does allow you to change the number of items.size.
What is collection_id_main and why is that used instead of collection_id? If it can be changed to use https://us1-prod-direct.discoveryplus.com/cms/collections/161374980142915215278619398379069832686?include=default&page[items.number]=1&page[items.size]=100 then it would work for more than just the first 16.
Thanks for investigating. Using collection_id instead collection_id_main which is example HGTV's collection_id should work.
collection_id_main is used because otherwise json output doesn't list collections in 'included' section and list_collection_items fails to list anything without that. This was only used in US version of dplus like list_collections()
But I'm now thinking that list_videos should work in this situation because that also lists single collection in 'data' section. Have to think good name for list_videos and add support for listing shows. Maybe list_collection() because it lists single collection? list_collection_items parses pages that contains multiple collections but we only want list content of one. That is why we use collection_id.
Should work now :)
@yeahme49 Can you test that everything works? I'm almost 100% sure login doesn't work. If you want test that, save cookie_file from userdata/addon_data/plugin.video.discoveryplus to somewhere and delete it from addon_data.
As far as I can tell, everything is working properly. I submitted a pull request that adds pagenation.
I removed my cookie_file and login does not work. After trying to log in on the website after removing the cookie_file, i now have to do the captcha, which I never had to do before. So something about doing a new login through the plugin needing a new cookie triggered something on arkose end. Putting my old cookie_file back and it works fine.
Thanks for adding pagenation 😄
Are we ready to merge this changes to master and matrix?
I will add support to use exported cookies.txt after these changes.
Made the couple changes you mentioned for the pull request, should be ready to merge now and add to master and matrix
Support added #10
@yeahme49 can you check what are current headers in d+ US? Previously: x-disco-params: realm=go,siteLookupKey=dplus_us x-disco-client: WEB:UNKNOWN:dplus_us:0.0.1
Also check what is in POST data to videoPlaybackInfo and channelPlaybackInfo (if that is used in USA)
x-disco-client: WEB:UNKNOWN:dplus_us:27.9.0 x-disco-params: realm=go,siteLookupKey=dplus_us
POST videoPlaybackInfo:
{"deviceInfo":{"adBlocker":true,"drmSupported":true,"hdrCapabilities":["SDR"],"hwDecodingCapabilities":[],"player":{"width":2630,"height":1479},"screen":{"width":3840,"height":2160},"soundCapabilities":["STEREO"]},"videoId":"2093355","wisteriaProperties":{"advertiser":{"adId":"|51516834747805739624932105135765523553||163926863916344a7e4ca179724a6edd4902c461a0156","firstPlay":0,"fwDid":"","fwGdprConsent":"","fwIsLat":0,"fwNielsenAppId":"P5A0FD4DE-4AE6-4B22-811B-36B9BD091980","gpaln":"","interactiveCapabilities":["brightline"]},"appBundle":"","device":{"browser":{"name":"chrome","version":"96.0.4664.45"},"language":"en","make":"","model":"","name":"Chrome","os":"Windows","osVersion":"NT 10.0","type":"desktop","id":"846c8d576fcf9844841c961e8da98118","player":{"name":"Discovery Player Web","version":"27.9.0"}},"gdpr":0,"siteId":"dplus_us","platform":"desktop","playbackId":"498263b4-ed01-4c46-b097-edd4af102acd","product":"dplus_us","sessionId":"f1b72d44-8c6a-4844-a589-f5e2222a5c64","streamProvider":{"suspendBeaconing":0,"hlsVersion":7,"pingConfig":0,"version":"1.0.0"}}}
POST channelPlaybackInfo:
{"channelId":"118","deviceInfo":{"adBlocker":true,"drmSupported":true,"hdrCapabilities":["SDR"],"hwDecodingCapabilities":[],"player":{"width":2630,"height":1479},"screen":{"width":3840,"height":2160},"soundCapabilities":["STEREO"]},"wisteriaProperties":{"advertiser":{"adId":"|51516834747805739624932105135765523553||16392688054506041cd55780fa97a64a5409b45fadffe","firstPlay":0,"fwDid":"","fwGdprConsent":"","fwIsLat":0,"fwNielsenAppId":"P5A0FD4DE-4AE6-4B22-811B-36B9BD091980","gpaln":"","interactiveCapabilities":["brightline"]},"appBundle":"","device":{"browser":{"name":"chrome","version":"96.0.4664.45"},"language":"en","make":"","model":"","name":"Chrome","os":"Windows","osVersion":"NT 10.0","type":"desktop","id":"846c8d576fcf9844841c961e8da98118","player":{"name":"Discovery Player Web","version":"27.9.0"}},"gdpr":0,"siteId":"dplus_us","platform":"desktop","playbackId":"f4f20228-e827-4872-ae09-7b49408bdab7","product":"dplus_us","sessionId":"f1b72d44-8c6a-4844-a589-f5e2222a5c64","streamProvider":{"suspendBeaconing":0,"hlsVersion":7,"pingConfig":1,"version":"1.0.0"}}}
Is framerate correct in master.mpd? If it is wrong I probably need to modify proxy.
Edit. please check headers from something else than videoplayback. They seem to be same as in EU.
Headers for tabbed-home: x-disco-client: WEB:UNKNOWN:dplus_us:1.25.0 x-disco-params: realm=go,siteLookupKey=dplus_us,bid=dplus,hn=www.discoveryplus.com,features=ar
Framerate in mpd is 30. Not sure if the video files that are served from mpd are now actually 30fps or if they are 29.970 still like when using the m3u8 file. You could update to use mpd and skip the proxy and if the blank screen issue comes back I can submit a pull request with an updated proxy
Thanks. Please try with current master. Is Ultra HD streams working at all? Is .mpd used in UHD streams?
Trying to play video results in:
2021-12-12 12:03:38.049 T:21924 ERROR
{
"data" : {
"attributes" : {
"markers" : {
"videoAboutToEnd" : 2500895
},
"reportProgressInterval" : 60000,
"ssaiInfo" : {
"forecastTimeline" : [ {
"event" : {
"action" : "streamInitiate",
"schema" : "playback"
},
"time" : {
"contentPosition" : 0.0,
"streamPosition" : 0.0
},
"triggerTime" : 0.0
}, {
"event" : {
"action" : "start",
"schema" : "playback"
},
"time" : {
"contentPosition" : 0.0,
"streamPosition" : 0.0
},
"triggerTime" : 0.001
}, {
"event" : {
"action" : "start",
"duration" : 2530.895,
"position" : 0,
"schema" : "chapter"
},
"time" : {
"contentPosition" : 0.0,
"streamPosition" : 0.0
},
"triggerTime" : 0.002
}, {
"event" : {
"action" : "complete",
"duration" : 2530.895,
"position" : 0,
"schema" : "chapter"
},
"time" : {
"contentPosition" : 2530.895,
"streamPosition" : 2530.895
},
"triggerTime" : 2530.895
}, {
"event" : {
"action" : "complete",
"schema" : "playback"
},
"time" : {
"contentPosition" : 2530.895,
"streamPosition" : 2530.895
},
"triggerTime" : 2530.896
}, {
"event" : {
"action" : "streamComplete",
"schema" : "playback"
},
"time" : {
"contentPosition" : 2530.895,
"streamPosition" : 2530.895
},
"triggerTime" : 2530.897
} ],
"vendorAttributes" : {
"breaks" : [ ],
"interstitialUrl" : "",
"nonLinearAds" : [ ],
"pingUrl" : "",
"sessionId" : "c77e158a-fc3b-4de1-96dc-1871ecd3466e"
}
},
"streaming" : [ {
"cdn" : "Verizon",
"fallback" : false,
"protection" : {
"clearkeyEnabled" : false,
"drmEnabled" : true,
"schemes" : {
"widevine" : {
"certificateUrl" : "",
"licenseUrl" : "https://content-discovery.uplynk.com/wv"
}
}
},
"provider" : "wisteria",
"streamMode" : "VOD",
"type" : "dash",
"url" : "https://content-discovery.uplynk.com/4ef89646fc3b48b3a9da14e73abb2c13.mpd?cid=4ef89646fc3b48b3a9da14e73abb2c13&ct=a&drm_policy_name=Prod_5April2021&exp=1639332609&rays=cdefghiba&rmt=wv&rn=5353259563291162093&tc=1&sig=0aa3d1cfacf3d05f459f0dda5400997912446d2de5814963d0ecd335b0268f86"
} ],
"userInfo" : {
"packages" : [ "Free", "Registered", "MotorTrend", "TheDodo", "DiscoveryLife", "Science", "Tlc", "History", "OprahWinfrey", "DiscoveryFamilyGo", "Travel", "Popsugar", "Bbc", "FoodNetwork", "InvestigationDiscovery", "Cooking", "HomeAndGarden", "AnimalPlanet", "Seeker", "DiscoveryEnEspanol", "DiyNetwork", "DiscoveryFamilia", "NowThis", "DestinationAmerica", "DPlusOriginals", "Lifetime", "Unbranded", "Thrillist", "AmericanHeroes", "Discovery", "MagnoliaNetPreview", "AeNetwork" ]
},
"viewingHistory" : {
"completed" : false,
"lastStartedTimestamp" : "2021-12-12T16:35:11Z",
"position" : 2000,
"viewed" : true
}
},
"id" : "2093355",
"type" : "videoPlaybackInfo"
},
"meta" : {
"site" : {
"attributes" : {
"brand" : "dplus_us",
"brandId" : "dplus",
"mainTerritoryCode" : "us",
"theme" : "dplus_us",
"websiteHostName" : ""
},
"id" : "dplus_us",
"type" : "site"
}
}
}
This is videoPlaybackInfo when I view a video on the website
Ultra HD still doesn't work. When I worked on it last, if I set drmSupported to true, it would give an mpd file. If set to false it was m3u8. But it wouldn't work either way. drmSupported = true the file wouldn't play at all, false it would stop after 4 seconds. Created an issue with ISA and no resolution. One person said Widevine on Windows doesn't support HEVC, so that may be why drmSupported = true fails. Not sure about the issue when drmSupported is set to false. I haven't had much time to try and gather any more info for them to try and troubleshoot more. I've been meaning to try it on my Chromecast with Google TV device to see if it works any better.
Currently broken because of many differences. What I found so far is that unlike other countries US needs headers for requests. Because of geoblock and US version needing account, support for US version is probably not going to happen anytime soon.
api url: https://us1-prod-direct.discoveryplus.com realm: go headers: x-disco-params=realm=go,siteLookupKey=dplus_us