search

streamlink / streamlink

Streamlink is a CLI utility which pipes video streams from various services into a video player
https://streamlink.github.io/
BSD 2-Clause "Simplified" License
9.9k stars 1.1k forks source link

plugins.dogus: StarTV live TV doesn't work #5298

Open stulluk opened 1 year ago

stulluk commented 1 year ago

Checklist

Streamlink version

Latest stable release

Description

Thank you very much for this plugin. It works fine for NTV and KralTV.

However, it doesn't work for StarTV, log is below.

Debug log

stulluk ~ $  streamlink https://www.startv.com.tr/canli-yayin best -l all
[13:28:04.497582][cli][debug] OS:         Linux-6.2.10-060210-generic-x86_64-with-glibc2.35
[13:28:04.497678][cli][debug] Python:     3.10.6
[13:28:04.497718][cli][debug] Streamlink: 5.4.0
[13:28:04.498169][cli][debug] Dependencies:
[13:28:04.499867][cli][debug]  certifi: 2020.6.20
[13:28:04.500194][cli][debug]  isodate: 0.6.1
[13:28:04.500454][cli][debug]  lxml: 4.8.0
[13:28:04.500846][cli][debug]  pycountry: 20.7.3
[13:28:04.501073][cli][debug]  pycryptodome: 3.17
[13:28:04.501747][cli][debug]  PySocks: 1.7.1
[13:28:04.502014][cli][debug]  requests: 2.28.2
[13:28:04.504453][cli][debug]  urllib3: 1.26.5
[13:28:04.504743][cli][debug]  websocket-client: 1.2.3
[13:28:04.504979][cli][debug]  importlib-metadata: 4.6.4
[13:28:04.505076][cli][debug] Arguments:
[13:28:04.505115][cli][debug]  url=https://www.startv.com.tr/canli-yayin
[13:28:04.505152][cli][debug]  stream=['best']
[13:28:04.505191][cli][debug]  --loglevel=all
[13:28:04.505337][cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
error: No playable streams found on this URL: https://www.startv.com.tr/canli-yayin
stulluk ~ $  streamlink --version
streamlink 5.4.0
stulluk ~ $
bastimeyer commented 1 year ago

The site embeds a YouTube livestream, which the plugin properly finds and hands over to the youtube plugin:

$ streamlink -l debug 'https://www.startv.com.tr/canli-yayin'
[cli][debug] OS:         Linux-6.2.11-1-git-x86_64-with-glibc2.37
[cli][debug] Python:     3.11.3
[cli][debug] Streamlink: 5.4.0+5.g5846abd6
[cli][debug] Dependencies:
[cli][debug]  certifi: 2022.12.7
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.9.2
[cli][debug]  pycountry: 22.3.5
[cli][debug]  pycryptodome: 3.17
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.28.2
[cli][debug]  urllib3: 1.26.15
[cli][debug]  websocket-client: 1.5.1
[cli][debug] Arguments:
[cli][debug]  url=https://www.startv.com.tr/canli-yayin
[cli][debug]  --loglevel=debug
[cli][debug]  --player=mpv
[cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
[plugins.youtube][debug] Using video ID: _F3fvsRAFF0
[plugins.youtube][debug] This video is live.
[utils.l10n][debug] Language code: en_US
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)
stulluk commented 1 year ago

@bastimeyer thank you for looking into this. However, why my debug log is much different than yours ( Due to geographic location ? )

stulluk ~ $ streamlink -l debug 'https://www.startv.com.tr/canli-yayin' [cli][debug] OS: Linux-6.2.10-060210-generic-x86_64-with-glibc2.35 [cli][debug] Python: 3.10.6 [cli][debug] Streamlink: 5.4.0 [cli][debug] Dependencies: [cli][debug] certifi: 2020.6.20 [cli][debug] isodate: 0.6.1 [cli][debug] lxml: 4.8.0 [cli][debug] pycountry: 20.7.3 [cli][debug] pycryptodome: 3.17 [cli][debug] PySocks: 1.7.1 [cli][debug] requests: 2.28.2 [cli][debug] urllib3: 1.26.5 [cli][debug] websocket-client: 1.2.3 [cli][debug] importlib-metadata: 4.6.4 [cli][debug] Arguments: [cli][debug] url=https://www.startv.com.tr/canli-yayin [cli][debug] --loglevel=debug [cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin error: No playable streams found on this URL: https://www.startv.com.tr/canli-yayin stulluk ~ $

Could you please let me know, what is needed for you to be able to reproduce my above situation ?

mkbloke commented 1 year ago

I get the same working YT live feed from a UK IP address. @stulluk are you accessing from an IP in Turkey? Maybe the site does something different for local users.

stulluk commented 1 year ago

I get the same working YT live feed from a UK IP address. @stulluk are you accessing from an IP in Turkey? Maybe the site does something different for local users.

Exactly. Thats why we are observing different behavior.

To confirm, I also tried from a server located in germany:

root@xxx ~ # streamlink -l debug 'https://www.startv.com.tr/canli-yayin'
[cli][info] streamlink is running as root! Be careful!
[cli][debug] OS:         Linux-5.4.0-139-generic-x86_64-with-glibc2.29
[cli][debug] Python:     3.8.10
[cli][debug] Streamlink: 5.4.0
[cli][debug] Dependencies:
[cli][debug]  certifi: 2019.11.28
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.9.2
[cli][debug]  pycountry: 22.3.5
[cli][debug]  pycryptodome: 3.17
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.28.2
[cli][debug]  urllib3: 1.26.15
[cli][debug]  websocket-client: 1.5.1
[cli][debug] Arguments:
[cli][debug]  url=https://www.startv.com.tr/canli-yayin
[cli][debug]  --loglevel=debug
[cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
[plugins.youtube][debug] Using video ID: _F3fvsRAFF0
[plugins.youtube][debug] This video is live.
[utils.l10n][debug] Language code: en_US
error: Unable to open URL: https://manifest.googlevideo.com/api/manifest/hls_variant/expire/1681842392/ei/eIw-ZIjUNpSU-gbZ-r6QCA/ip/2a01%3A4f8%3A221%3A2f16%3A0%3A0%3A0%3A2/id/_F3fvsRAFF0.4/source/yt_live_broadcast/requiressl/yes/hfr/1/playlist_duration/30/manifest_duration/30/maudio/1/gcr/de/vprv/1/go/1/pacing/0/nvgoi/1/keepalive/yes/fexp/24007246/dover/11/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cplaylist_duration%2Cmanifest_duration%2Cmaudio%2Cgcr%2Cvprv%2Cgo%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIgSyLIWjLcE3bIZk9Y1_wv-g4sOCl-N-_G7vnI75akFSUCIQCMus-EGwygnshYaj_3YQ_D94xwYtHd05Dpcwq5nDO03A%3D%3D/file/index.m3u8 (429 Client Error: Too Many Requests for url: https://manifest.googlevideo.com/api/manifest/hls_variant/expire/1681842392/ei/eIw-ZIjUNpSU-gbZ-r6QCA/ip/2a01%3A4f8%3A221%3A2f16%3A0%3A0%3A0%3A2/id/_F3fvsRAFF0.4/source/yt_live_broadcast/requiressl/yes/hfr/1/playlist_duration/30/manifest_duration/30/maudio/1/gcr/de/vprv/1/go/1/pacing/0/nvgoi/1/keepalive/yes/fexp/24007246/dover/11/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cplaylist_duration%2Cmanifest_duration%2Cmaudio%2Cgcr%2Cvprv%2Cgo%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIgSyLIWjLcE3bIZk9Y1_wv-g4sOCl-N-_G7vnI75akFSUCIQCMus-EGwygnshYaj_3YQ_D94xwYtHd05Dpcwq5nDO03A%3D%3D/file/index.m3u8)
root@xxx ~ #

Could you please suggest me, how to fix this issue ?

bastimeyer commented 1 year ago

Unless someone else already has, I'll have a look in a couple of hours with a VPN.

As you can see, the plugin is really simple and whatever this specific site does in your region won't be much different from what's already implemented, so basically just an additional xpath query or so. https://github.com/streamlink/streamlink/blob/5.4.0/src/streamlink/plugins/dogus.py

mkbloke commented 1 year ago

In the case of your access via German IP, @stulluk, you're getting a different issue:

429 Client Error: Too Many Requests for url:

stulluk commented 1 year ago

Unless someone else already has, I'll have a look in a couple of hours with a VPN.

As you can see, the plugin is really simple and whatever this specific site does in your region won't be much different from what's already implemented, so basically just an additional xpath query or so. https://github.com/streamlink/streamlink/blob/5.4.0/src/streamlink/plugins/dogus.py

@bastimeyer thank you. I am not a web developer, so I don't have experience on what is an xpath query, but if I understand correctly, this is to find HLS links in the webpage code ?

Additionally, would it be possible for you to check with a VPN ?

In the case of your access via German IP, @stulluk, you're getting a different issue:

429 Client Error: Too Many Requests for url:

@mkbloke I made a quick test on my Ubuntu Desktop via Proton VPN to connect to a Netherlands VPN:

stulluk ~ $  streamlink -l debug 'https://www.startv.com.tr/canli-yayin' 480p
[cli][debug] OS:         Linux-6.2.10-060210-generic-x86_64-with-glibc2.35
[cli][debug] Python:     3.10.6
[cli][debug] Streamlink: 5.4.0
[cli][debug] Dependencies:
[cli][debug]  certifi: 2020.6.20
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.8.0
[cli][debug]  pycountry: 20.7.3
[cli][debug]  pycryptodome: 3.17
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.28.2
[cli][debug]  urllib3: 1.26.5
[cli][debug]  websocket-client: 1.2.3
[cli][debug]  importlib-metadata: 4.6.4
[cli][debug] Arguments:
[cli][debug]  url=https://www.startv.com.tr/canli-yayin
[cli][debug]  stream=['480p']
[cli][debug]  --loglevel=debug
[cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
[plugins.youtube][debug] Using video ID: _F3fvsRAFF0
[plugins.youtube][debug] This video is live.
[utils.l10n][debug] Language code: en_US
[cli][info] Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)
[cli][info] Opening stream: 480p (hls)
[cli][info] Starting player: /usr/bin/vlc
[stream.hls][debug] Reloading playlist
[cli][debug] Pre-buffering 8192 bytes
[stream.hls][debug] First Sequence: 1651641; Last Sequence: 1651646
[stream.hls][debug] Start offset: 0; Duration: None; Start Sequence: 1651644; End Sequence: None
[stream.hls][debug] Adding segment 1651644 to queue
[stream.hls][debug] Adding segment 1651645 to queue
[stream.hls][debug] Adding segment 1651646 to queue
[stream.hls][debug] Writing segment 1651644 to output
[stream.hls][debug] Segment 1651644 complete
[cli.output][debug] Opening subprocess: /usr/bin/vlc --input-title-format https://www.startv.com.tr/canli-yayin -
[stream.hls][debug] Writing segment 1651645 to output
[stream.hls][debug] Segment 1651645 complete
[stream.hls][debug] Writing segment 1651646 to output
[stream.hls][debug] Segment 1651646 complete
[cli][debug] Writing stream to output
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 1651647 to queue
[stream.hls][debug] Writing segment 1651647 to output
[stream.hls][debug] Segment 1651647 complete
[cli][info] Player closed
[stream.segmented][debug] Closing worker thread
[stream.segmented][debug] Closing writer thread
[cli][info] Stream ended
[cli][info] Closing currently open stream...
stulluk ~ $  streamlink -l debug 'https://www.startv.com.tr/canli-yayin' 480p --stream-url
https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1681904784/ei/MIA_ZNToGI3RgQeEk4aICQ/ip/149.34.244.155/id/_F3fvsRAFF0.4/itag/94/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/sgoap/gir%3Dyes%3Bitag%3D140/sgovp/gir%3Dyes%3Bitag%3D135/hls_chunk_host/rr1---sn-5hne6nz6.googlevideo.com/playlist_duration/30/manifest_duration/30/gcr/nl/vprv/1/playlist_type/DVR/initcwndbps/538750/mh/t5/mm/44/mn/sn-5hne6nz6/ms/lva/mv/m/mvi/1/pl/24/dover/11/pacing/0/keepalive/yes/fexp/24007246/mt/1681882827/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,live,sgoap,sgovp,playlist_duration,manifest_duration,gcr,vprv,playlist_type/sig/AOq0QJ8wRgIhAM21e-7DsM53s8eW5PirpA28b2c4llvQBpD4IS3LcnGGAiEA7gctXoMBsEN3iQqlji4CYMOz5dr9cowUTcNpMoU_9Gk%3D/lsparams/hls_chunk_host,initcwndbps,mh,mm,mn,ms,mv,mvi,pl/lsig/AG3C_xAwRQIhAPMcWq5aVeW0aTvu7vQy_BHOfPDdTzsVehkYFh_N-dEUAiBJmgDP5rnZom_3EdxE7uHc6mK-oTqgpJbWFvjHZGdJnQ%3D%3D/playlist/index.m3u8
stulluk ~ $

So the backed behaves differently depending on geographic location :(

So, if I understand correctly, we need to find a way to fix this plugin for Turkey locations, am I wrong ?

bastimeyer commented 1 year ago

I had a look yesterday and I got the different stream implementation when accessing the site via a French proxy. It's a bit more difficult though because the stream URL is embedded in their player JS file in that case. This can be extracted rather easily, but there's an additional base64 encoded query string added by their player code. When accessing the stream URL without that data, it still seemed to work, but I'm not 100% sure if there will be issues when not including that data. I didn't manage to find out where the data was coming from and where it was calculated, because the site is terribly bloated and loads a ton of additional JS scripts asynchronously while initializing the player and the stream URL.

meatab78 commented 10 months ago

Checklist

Streamlink version

Latest stable release

Description

Thank you very much for this plugin. It works fine for NTV and KralTV.

However, it doesn't work for StarTV, log is below.

Debug log

stulluk ~ $  streamlink https://www.startv.com.tr/canli-yayin best -l all
[13:28:04.497582][cli][debug] OS:         Linux-6.2.10-060210-generic-x86_64-with-glibc2.35
[13:28:04.497678][cli][debug] Python:     3.10.6
[13:28:04.497718][cli][debug] Streamlink: 5.4.0
[13:28:04.498169][cli][debug] Dependencies:
[13:28:04.499867][cli][debug]  certifi: 2020.6.20
[13:28:04.500194][cli][debug]  isodate: 0.6.1
[13:28:04.500454][cli][debug]  lxml: 4.8.0
[13:28:04.500846][cli][debug]  pycountry: 20.7.3
[13:28:04.501073][cli][debug]  pycryptodome: 3.17
[13:28:04.501747][cli][debug]  PySocks: 1.7.1
[13:28:04.502014][cli][debug]  requests: 2.28.2
[13:28:04.504453][cli][debug]  urllib3: 1.26.5
[13:28:04.504743][cli][debug]  websocket-client: 1.2.3
[13:28:04.504979][cli][debug]  importlib-metadata: 4.6.4
[13:28:04.505076][cli][debug] Arguments:
[13:28:04.505115][cli][debug]  url=https://www.startv.com.tr/canli-yayin
[13:28:04.505152][cli][debug]  stream=['best']
[13:28:04.505191][cli][debug]  --loglevel=all
[13:28:04.505337][cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
error: No playable streams found on this URL: https://www.startv.com.tr/canli-yayin
stulluk ~ $  streamlink --version
streamlink 5.4.0
stulluk ~ $
Benedek012 commented 3 months ago

Startv NOT working,please help streamlink.