search

firsttris / plugin.video.sendtokodi

:tv: plays various stream sites on kodi using youtube-dl
MIT License
121 stars 27 forks source link

Support original manifest if available #74

Closed nullket closed 1 year ago

nullket commented 1 year ago

Should close https://github.com/firsttris/plugin.video.sendtokodi/issues/72 and increase the quality on a couple of websites. This does not implement custom manifests on websites which hide the manifest url (like youtube).

I had the most of it ready for the development of https://github.com/firsttris/plugin.video.sendtokodi/issues/34

I assume this might fail catastrophic

Therefore, I made a setting for it and disabled the feature by default until a few heroes have tested it in the field.

The changes to the addon.xml requires at least kodi 18.

nullket commented 1 year ago

I made the search for a "valid" manifest a bit less restrictive, this should also close https://github.com/firsttris/plugin.video.sendtokodi/issues/70

GautamMKGarg commented 1 year ago

Issue #72 not resolved. Getting bellow error

2022-09-06 17:00:32.616 T:908     ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'KeyError'>
                                                   Error Contents: 'formats'
                                                   Traceback (most recent call last):
                                                     File "C:\Users\GG\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 250, in <module>
                                                       startingItem = createListItemFromVideo(ydl.extract_info(startingVideoUrl, download=False))
                                                     File "C:\Users\GG\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 120, in createListItemFromVideo
                                                       url = extract_best_all_in_one_stream(result)
                                                     File "C:\Users\GG\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 95, in extract_best_all_in_one_stream
                                                       for entry in result['formats']:
                                                   KeyError: 'formats'
                                                   -->End of Python script error report<--
GautamMKGarg commented 1 year ago

This is the another Error which I am getting on another URL

2022-09-06 17:10:46.985 T:24032   ERROR <general>: CDVDDemuxFFmpeg::Open - error probing input format, https://media-content1.akamaized.net/video/afff35ba48ce69be34bf89ead3f993a9/10/dash/h265_main.mpd
2022-09-06 17:10:46.985 T:24032   ERROR <general>: CVideoPlayer::OpenDemuxStream - Error creating demuxer
nullket commented 1 year ago

Issue https://github.com/firsttris/plugin.video.sendtokodi/issues/72 not resolved. Getting bellow error

Interesting. Please be so kind and

btw the mxplayer link in issue 73 resolves just fine for me (I get the original manifest) but Kodi is not able to play it. VLC on the other hand can. This is due to the fact that not all adapting streaming manifests are currently supported by the input stream adaptive plugin we rely on. Therefore, I am highly interested how yatse gets kodi to play this video (thus requesting a log for that)

GautamMKGarg commented 1 year ago

URL: https://www.mxplayer.in/show/watch-aashram/season-1/duh-swapna-online-d445579792b0135598ba1bc9088a84cb Logs:

2022-09-06 19:56:12.321 T:32580    INFO <general>: JSONRPC Server: New connection added
2022-09-06 19:56:12.560 T:25304    INFO <general>: initializing python engine.
2022-09-06 19:56:16.446 T:25304    INFO <general>: plugin.video.sendtokodi: found original manifest: https://media-content.akamaized.net/video/0feb23ed2f6cb0ddc0839d5317053b14/1/dash/h265_main.mpd
2022-09-06 19:56:16.450 T:25304    INFO <general>: CPythonInvoker(7, C:\Users\GG\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py): script successfully run
2022-09-06 19:56:16.556 T:25304    INFO <general>: Python interpreter stopped
2022-09-06 19:56:16.631 T:6572     INFO <general>: VideoPlayer::OpenFile: plugin://plugin.video.sendtokodi/?https://www.mxplayer.in/show/watch-aashram/season-1/duh-swapna-online-d445579792b0135598ba1bc9088a84cb
2022-09-06 19:56:16.668 T:19644    INFO <general>: Creating InputStream
2022-09-06 19:56:16.838 T:19644    INFO <general>: Creating Demuxer
2022-09-06 19:56:16.855 T:19644   ERROR <general>: CDVDDemuxFFmpeg::Open - error probing input format, https://media-content.akamaized.net/video/0feb23ed2f6cb0ddc0839d5317053b14/1/dash/h265_main.mpd
2022-09-06 19:56:16.855 T:19644   ERROR <general>: CVideoPlayer::OpenDemuxStream - Error creating demuxer
2022-09-06 19:56:16.855 T:19644    INFO <general>: CVideoPlayer::OnExit()
2022-09-06 19:56:16.858 T:25884    INFO <general>: Deleting settings information for files plugin://plugin.video.sendtokodi/?https://www.mxplayer.in/show/watch-aashram/season-1/duh-swapna-online-d445579792b0135598ba1bc9088a84cb
2022-09-06 19:56:16.868 T:6572     INFO <general>: Loading skin file: DialogConfirm.xml, load type: KEEP_IN_MEMORY
2022-09-06 19:56:16.870 T:6572     INFO <general>: CVideoPlayer::CloseFile()
2022-09-06 19:56:16.871 T:6572     INFO <general>: VideoPlayer: waiting for threads to exit
2022-09-06 19:56:16.871 T:6572     INFO <general>: VideoPlayer: finished waiting

Second URL: https://www.mxplayer.in/show/watch-doctor-stranger-hindi-dubbed-series-online-7dd1b7a353d2a1ca62b672248e17865b Logs:

2022-09-06 19:59:56.896 T:32580    INFO <general>: JSONRPC Server: New connection added
2022-09-06 19:59:56.943 T:31636    INFO <general>: initializing python engine.
2022-09-06 19:59:59.133 T:20120    INFO <general>: CActiveAESink::OpenSink - initialize sink
2022-09-06 20:00:01.968 T:31636    INFO <general>: plugin.video.sendtokodi: could not find an original manifest falling back to best all-in-one stream
2022-09-06 20:00:01.969 T:31636   ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'KeyError'>
                                                   Error Contents: 'formats'
                                                   Traceback (most recent call last):
                                                     File "C:\Users\Gautam\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 250, in <module>
                                                       startingItem = createListItemFromVideo(ydl.extract_info(startingVideoUrl, download=False))
                                                     File "C:\Users\Gautam\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 120, in createListItemFromVideo
                                                       url = extract_best_all_in_one_stream(result)
                                                     File "C:\Users\Gautam\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 95, in extract_best_all_in_one_stream
                                                       for entry in result['formats']:
                                                   KeyError: 'formats'
                                                   -->End of Python script error report<--

2022-09-06 20:00:02.134 T:31636    INFO <general>: Python interpreter stopped

I added below code at Line number 136 of service.py file, and it fixed issue with first URL mentioned above. https://github.com/nullket/plugin.video.sendtokodi/blob/use_orignal_manifest/service.py#L136

if 'mpd' in url:
        list_item.setProperty('inputstream', 'inputstream.adaptive')
        list_item.setProperty('inputstream.adaptive.manifest_type', 'mpd')

I have taken reference of above code from yatse https://github.com/Tolriq/repository.yatse.kodi/blob/0e994240717d652c6fe84e0b40c2e418a1051492/script.yatse.kodi/lib/utils.py#L87-L88

Please check if this code has an impact on any other functionality. Also please check if we can improve if 'mpd' in url: condition.

nullket commented 1 year ago

I added below code at Line number 136 of service.py file, and it fixed issue with first URL mentioned above.

Yikes! I forgot that. I will add and test it the next days (might take a bit)

Please check if this code has an impact on any other functionality. Also please check if we can improve if 'mpd' in url: condition.

Yeah, no worries. We just need to check if we are using a manifest and if yes reuse this to automatically choose mpd/hls/ipm , for further reference take a look here.

You seconds URL does not seem to be resolved with yt-dlp at all. It seems to be a playlist (instead of a single video) but the "entries" list which should be there for a playlist is empty. But that might be a geoblock for me. For you it might fail because something with the playlist is on our side broken. To test something like this please always check if the standalone version of yt-dlp is working. Install it from here https://github.com/yt-dlp/yt-dlp and if it does not work, open an issue there.

GautamMKGarg commented 1 year ago

Yes, looks like second URL is not working even for me. There might be some issue with yt-dlp itself. I did not test second URL with Yatse. Will you be able to provide your email address? I will send you an code, which might be useful for you.

GautamMKGarg commented 1 year ago

Found another issue with the code. Below URL was working fine in earlier version. with this code it's not working

https://www.hungama.com/song/tumse-pyaar-karke/82061100/

Logs

2022-09-08 01:17:26.265 T:24148    INFO <general>: initializing python engine.
2022-09-08 01:17:29.760 T:24148    INFO <general>: plugin.video.sendtokodi: could not find an original manifest falling back to best all-in-one stream
2022-09-08 01:17:29.760 T:24148    INFO <general>: plugin.video.sendtokodi: Error: was not able to extract manifest or all-in-one stream. Implement https://github.com/firsttris/plugin.video.sendtokodi/issues/34
2022-09-08 01:17:29.760 T:24148    INFO <general>: CPythonInvoker(12, C:\Users\GG\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py): script aborted
2022-09-08 01:17:29.868 T:24148    INFO <general>: Python interpreter stopped
nullket commented 1 year ago

Found another issue with the code. Below URL was working fine in earlier version. with this code it's not working

Resolving is rather slow, to avoid resolving a second time with ytdlp (like yatse) just the get the "best" stream, I tried to write my own "get the best all in one stream" from the already resolved formats list. I just have not thought about audio only streams. Should be fixed now (the meta fields seems to be optional, thus getting the correct format seems to be really more guessing than knowing - if it remains unstable we should look into detail how ytdlp does it on their end)

Yes, looks like second URL is not working even for me. There might be some issue with yt-dlp itself. I did not test second URL with Yatse.

It is either an issue with yt-dlp itself or simply an unsupported feature.

I added below code at Line number 136 of service.py file, and it fixed issue with first URL mentioned above.

Yikes! I forgot that. I will add and test it the next days (might take a bit)

Should be fixed. Please test it again and report back.

GautamMKGarg commented 1 year ago

Thanks @nullket

All the issues reported earlier are now fixed. I conducted few more tests. here are the other issues I am facing now

  1. URL: https://www.voot.com/shows/asur/1/922885/the-dead-can-talk/922976 
    
2022-09-10 17:28:20.520 T:29120    INFO <general>: initializing python engine.
2022-09-10 17:28:24.351 T:29120   ERROR <general>: WARNING: [Voot] Ignoring subtitle tracks found in the HLS manifest; if any subtitle tracks are missing, please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

2022-09-10 17:28:24.385 T:29120 INFO : plugin.video.sendtokodi: found original manifest: https://cdnapisec.kaltura.com/p/1982551/playManifest/pt/https/f/applehttp/t/web/e/0_8mvarek0 2022-09-10 17:28:24.412 T:29120 ERROR : EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--

2022-09-10 17:28:24.588 T:29120 INFO : Python interpreter stopped 2022-09-10 17:29:06.645 T:1352 INFO : JSONRPC Server: Disconnection detected


2. URL: https://www.zee5.com/movies/details/robot/0-0-movie_466418637

2022-09-10 17:35:50.182 T:22632 INFO : initializing python engine. 2022-09-10 17:35:52.292 T:3388 INFO : CActiveAESink::OpenSink - initialize sink 2022-09-10 17:35:54.393 T:22632 INFO : plugin.video.sendtokodi: found original manifest: https://zee5vodnd.akamaized.net/hls1/elemental/hls/Movies/Drama/ROBOT_Hindi_Movie_Rev1_RTR_hi_bf1ce8b9b0fb4dd1e16587e909bf042e/index.m3u8?hdnea=st=1662811433~exp=1662815153~acl=/hls1/elemental/hls/Movies/Drama/ROBOT_Hindi_Movie_Rev1_RTR_hi_bf1ce8b9b0fb4dd1e16587e909bf042e/index.m3u8*~hmac=15f8d5e277d457b48d9c2bc6b7bffb7225c7e280cab35160432f39da6c315191 2022-09-10 17:35:54.421 T:22632 ERROR : EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--

2022-09-10 17:35:54.585 T:22632 INFO : Python interpreter stopped

nullket commented 1 year ago

All the issues reported earlier are now fixed. I conducted few more tests. here are the other issues I am facing now URL: https://www.voot.com/shows/asur/1/922885/the-dead-can-talk/922976

I don't think there is so much we can do about this. It has a result['manifest_url'] with the value https://cdnapisec.kaltura.com/p/1982551/playManifest/pt/https/f/applehttp/t/web/e/0_8mvarek0 and as far as I understand yt-dlp the manifest url should have a file ending OR deliver at least some readable xml. For the manifest_url is simply a blank 404. I assume this to be, an issue with yt-dlp OR missing info on my side on how to retrieve the actual manifest from that url OR some DRM issue. On the other hand I repurposed the function to get the manifest type to test if the manifest is valid at all. So your url should now fallback to the best all in one stream (so no adapative quality management). Anyway your video seems to be drm protected and thus unsupported. In the future please test you videos with the standalone version of yt-dlp FIRST!

Output of standalone ytdlp ``` yt-dlp --check-formats https://www.voot.com/shows/asur/1/922885/the-dead-can-talk/922976 [Voot] 922976: Downloading JSON metadata [Voot] 922976: Downloading m3u8 information WARNING: [Voot] Ignoring subtitle tracks found in the HLS manifest; if any subtitle tracks are missing, please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U [info] Testing format hls-2723 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-2723. Skipping... [info] Testing format hls-1595 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-1595. Skipping... [info] Testing format hls-1126 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-1126. Skipping... [info] Testing format hls-1011 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-1011. Skipping... [info] Testing format hls-921 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-921. Skipping... [info] Testing format hls-715 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-715. Skipping... [info] Testing format hls-507 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-507. Skipping... [info] Testing format hls-357 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-357. Skipping... [info] Testing format hls-audio-English ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-audio-English. Skipping... [info] Testing format hls-2723 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-2723. Skipping... [info] Testing format hls-1595 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-1595. Skipping... [info] Testing format hls-1126 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-1126. Skipping... [info] Testing format hls-1011 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-1011. Skipping... [info] Testing format hls-921 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-921. Skipping... [info] Testing format hls-715 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-715. Skipping... [info] Testing format hls-507 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-507. Skipping... [info] Testing format hls-357 ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format [info] Unable to download format hls-357. Skipping... ERROR: [Voot] 0_8mvarek0: Requested format is not available. Use --list-formats for a list of available formats ```

URL: https://www.zee5.com/movies/details/robot/0-0-movie_466418637

The manifest url had some additional ending. Fixed that in a more robust way to get the correct manifest type.

Please retest EVERYTHING and if something does not work please make sure it works with the standalone version of ytdlp.

GautamMKGarg commented 1 year ago

After recent change (https://github.com/firsttris/plugin.video.sendtokodi/pull/74/commits/5f7758d96e4185bac92ecc0b8787d739bf776c7c), below URL again stopped working. Before this commit, it was working fine.

URL: https://www.mxplayer.in/show/watch-aashram/season-1/duh-swapna-online-d445579792b0135598ba1bc9088a84cb

2022-09-14 12:06:21.875 T:20732    INFO <general>: initializing python engine.
2022-09-14 12:06:25.677 T:20732   ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'KeyError'>
                                                   Error Contents: 'manifest_url'
                                                   Traceback (most recent call last):
                                                     File "C:\Users\Gautam\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 295, in <module>
                                                       xbmcplugin.setResolvedUrl(__handle__, True, listitem=createListItemFromVideo(result))
                                                     File "C:\Users\Gautam\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 148, in createListItemFromVideo
                                                       url = extract_manifest_url(result)
                                                     File "C:\Users\Gautam\AppData\Roaming\Kodi\addons\plugin.video.sendtokodi\service.py", line 87, in extract_manifest_url
                                                       if 'manifest_url' in entry and 'vcodec' in entry and get_adaptive_type_from_url(result['manifest_url']):
                                                   KeyError: 'manifest_url'
                                                   -->End of Python script error report<--

2022-09-14 12:06:25.800 T:20732    INFO <general>: Python interpreter stopped
nullket commented 1 year ago

That was an unfortunate copy paste error. Please try again.

GautamMKGarg commented 1 year ago

Thanks @nullket I conducted several tests, all of them are working fine. I think code is ready to be merged.

nullket commented 1 year ago

@firsttris it finally works. This will close https://github.com/firsttris/plugin.video.sendtokodi/issues/70, https://github.com/firsttris/plugin.video.sendtokodi/issues/72 and improve quality for some sites. As this is a bigger change I would welcome if you could review this PR - of course as always whenever you have the time for it :)

firsttris commented 1 year ago

Good work!

GautamMKGarg commented 1 year ago

Thanks @nullket and @firsttris

When are you planning to release the next stable version of sendtokodi? It's okay to update the code on Windows/Linux but it's difficult to update code on Smart TV using github.

firsttris commented 1 year ago

why is it difficult? you just install the repo and the addon updates automatically?

on SmartTV you can use Eplorer feature of Kodi to copy the repo

GautamMKGarg commented 1 year ago

Sorry, The version which was downloaded from repo was showing as "0.9.277" and the version in the master was 100.0.0. That's why I got confused. Now I noticed that both are same. Sorry for the confusion.