fe80Grau / ytdlp2STRM

A little script to serve Youtube / Twitch / Crunchyroll videos without storage it. Uses yt-dlp HTTP data throught Flask and dynamic URLs. We can use this dynamic URLs to set STRM files.
https://github.com/fe80Grau/ytdlp2STRM
MIT License
173 stars 15 forks source link

Live Streams #6

Closed CrazyWolf13 closed 1 year ago

CrazyWolf13 commented 1 year ago

Theoretically yt-dlp works with many different video hosting platforms, even with twitch.tv or youtube live streams, does ytdlp2strm work with live streams? Has this ever been tested?

Traace commented 1 year ago

I like to know that, too.

I want to convert 24/7 Live Streams into STRM for jellyfin 👯

CrazyWolf13 commented 1 year ago

@fe80Grau Any idea?

fe80Grau commented 1 year ago

Hi, check the last commit, you can add a new line in plugins/youtube/channel_list.json with /streams after all, example: [ "@todonoticias/streams" ]

This only gets the videos from the /streams channel tab, and strm file is forced with "direct" mode . But carefully in the last commit, I changed the folder structure of the project, now the channel list and the configuration files are in different folders.

Configurations (divided into 2 files):

I'm thinking of developing other integrations with yt-dlp like Crunchyroll, so I need to work this as a "plugin". Hence the change of the folder structure

CrazyWolf13 commented 1 year ago

Wow, this is amazing! Can't wait to test it:)

fe80Grau commented 1 year ago

You can test Twitch live stream with the last commit. I added more instruction in readme but i put here a litte instructions.

restart service after git pull Copy channel_list.json (older version) yo ./plugins/youtube/channel_list.json Is not necessary edit the older cronjob (i added compatibility with last verions), but i recommend it to keep an order mkdir for strm Twitch (example; mkdir /media/Twitch) Edit config and channel_list in plugins/twitch You can test it with cd /opt/ytdlp2STRM && /usr/bin/python3 /opt/ytdlp2STRM/cli.py --m plugins.twitch.to_strm --p twitch,direct If you want to run this every 10 minutes:

cd /etc/cron.d && sudo echo "*/10 * * * * root cd /opt/ytdlp2STRM && /usr/bin/python3 /opt/ytdlp2STRM/cli.py --m plugins.twitch.to_strm --p twitch,direct" > ytdlp2STRM_twitch_direct
CrazyWolf13 commented 1 year ago

Wow, amazing! Been a little bit busy in last time, but can't wait to test it:)

Traace commented 1 year ago

Streams are not working. I think Youtube is blocking my Server WAN IP.

06/06/2023 09:33:25
Running plugins.youtube.to_strm with ['youtube', 'direct'] params
Preparing channel @uproxx/streams
No channel ID Found, Research with no video tab
yt-dlp --compat-options no-youtube-channel-redirect --compat-options no-youtube-unavailable-videos --restrict-filenames --ignore-errors --no-warnings --playlist-start 1 --playlist-end 1 --print channel_url https://www.youtube.com/@uproxx/streams
Cleaning /media/Youtube/@uproxx [False] folder...
Inflating nfo file..
No poster detected
No landscape detected
Command 
 yt-dlp https://www.youtube.com/channel/False --write-description --playlist-items 0 --output "/media/Youtube/ERROR: [youtube:tab] False: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: 'Bad Request'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U.description" > /dev/null 2>&1 && cat "/media/Youtube/ERROR: [youtube:tab] False: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: 'Bad Request'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U.description"
Output 

Processing videos in channel
{'video_id': 'ERROR: [youtube] 05689ErDUdM: No video formats found!', 'video_name': ' please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U [ERROR: [youtube] 05689ErDUdM: No video formats found!]'}

Any plan for adding proxy support?

fe80Grau commented 1 year ago

hi @Traace , I'm getting properly the streams tab:

Preparing channel @uproxx/streams
Cleaning D:/Youtube/@uproxx [UCI2HI_aQ_S3v4vfDQgNtK1g] folder...
Inflating  nfo file..
Descriptions only works in Linux system at the moment
Processing videos in channel
{'video_id': '05689ErDUdM', 'video_name': 'Rap Live Radio 24/7 | Hip-Hop, Popular Rap Music & UPROXX Sessions Live Performances! 2023-06-06 13:22
5689ErDUdM]'}

Can you check your yt-dlp version with yt-dlp -U command? You need last stable version

Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)

Anyway, proxy support is a good feature i will work about it

Traace commented 1 year ago

Yes, sorry maybe there was a misunderstanding.

I know that yt-dlp returns 'No video formats found' when the IP was blocked by youtube. This is no issue with your code at all :) (btw: many thanks for the work)

Proxy support would be awesome, indeed. Meanwhile as a workaround I can send that traffic through a VPN tunnel.

fe80Grau commented 1 year ago

Yep @Traace , I added proxy feature for youtube plugin. You can check it in last commit. Add to your plugins/youtube/config.json proxy and proxy_url params like

{
    "strm_output_folder" : "D:/Youtube",
    "channels_list_file" : "./plugins/youtube/channel_list.json",
    "days_dateafter" : "0", 
    "videos_limit" : "10",
    "sponsorblock" : false,
    "sponsorblock_cats" : "sponsor",
    "proxy" : true,
    "proxy_url" : "socks5://user:pass@127.0.0.1:1080/"
}

Let me know if you try this.

Traace commented 1 year ago

Hey @fe80Grau, thanks again. The Proxy works :)

I've noticed that since i added the proxy param, while creating the .strm file the following gets logged:

Inflating  nfo file..
Output 
 cat: unrecognized option '--proxy'
Try 'cat --help' for more information.

That warning also gets written into the tvshow.nfo plot section and is visible in jellyfin:

 <plot>cat: unrecognized option '--proxy'
Try 'cat --help' for more information.</plot>
fe80Grau commented 1 year ago

I think that is fixed with last commit.

Traace commented 1 year ago

Yes, nfo and strm files are fine now. But the stream can't be played in jellyfin because the IP is blocked. Google is known for blocking many server networks around the world.

I could route jellyfin and ffmpeg outgoing traffic via proxy, too. But that would also route more than 3000 LiveTV Channels through the proxy... I dont want that to happen :)

Routing just the ipv4 from manifest.googlevideo.com through a VPN does fix nothing. I guess they are using a CDN Backend with many different IP's.

Can't find another solution... Maybe my infrastucture is to big for this :)

fe80Grau commented 1 year ago

Okaaay, direct mode is not good for you because ytdlp2strm will do a redirect to video url and Jellyfin process this stream trying to connect to Youtube servers, i think that you can wok with bridge mode with streams, the proxy param is setted to yt-dlp command to extract bytes from stream (jumping google restrictions), this bytes will be storage in temp chunk files (managed by stream_with_context Flask function), then http://127.0.0.1:5000/youtube/bridge/HPiv1pzIPRc is served by Jellyfin as local file and never request anything from youtube

Pull last comit i fixed a little param in yt-dlp for bridge mode.

Is simple to test, edit your strm file and change /direct/ to /bridge/ , http://127.0.0.1:5000/youtube/direct/HPiv1pzIPRc -> http://127.0.0.1:5000/youtube/bridge/HPiv1pzIPRc Can you try and tell me if it works?

fe80Grau commented 1 year ago

@Traace i close this because the Live stream feature is implemmented now. If you need more help with your case you can open a new issue

Thank you for all tests 😄