Streaming audio from Youtube

[mrhh69]

Right now I'm trying to implement streaming audio from youtube and came across this discussion and your crate. It's a really great project, but youtube has some idiosyncrasies that I've had to work around to get it to work:

They are described in this really helpful blog post, but essentially to avoid being rate limited a streaming client needs to send various range requests that are all under ~10MB in size. For now this change is working for me with short audio, but I was wondering if adding support for youtube's weird streaming system would be something for this project in the long run?

Happy to help any way I can. Thanks!

[aschey]

Hey @mrhh69, thanks for submitting an issue. I'm not surprised that YouTube is a bit picky about this. Based on your code, it looks like there are two issues:

1. The GET response doesn't return the content length, but the HEAD request does. It should be simple enough to add a fallback to check this.
2. The built-in reqwest client doesn't provide a way to customize the stream behavior.

For the second issue, we probably need a new implementation of the stream method in the Response trait that continuously makes range requests until the stream is complete rather than using the bytes_stream method. The pseudocode would look something like this, possibly utilizing the async_stream crate:

stream! {
   for current_chunk in (chunk_start..content_length).step_by(chunk_size) {
       yield client.get_range(current_chunk, current_chunk + chunk_size).await;
   }
}

Are there any other issues you're seeing so far? I probably don't want to add anything too Youtube-specific to this crate, but I'd be happy to extend the core API to make it more flexible if necessary.

[cdrani]

@aschey: Sorry, I'm not 100% on how to implement 1. I'm new to rust and not even sure what's meant my this. I'm trying to implement a similar audio streamer for live videos, such as one of those 24/7 lofi videos. Any guidance here would be greatly appreciated.

[aschey]

@cdrani thanks for bringing this up again. I think this will take a bit more thought to plan out than I had initially given.

tl;dr on the long response below - Just use yt-dlp or some other dedicated Youtube download tool if you can. If someone wants to integrate this directly into stream-download, it will take a bit of research and planning.

--

Thinking about this some more, I think trying to support YouTube directly will be challenging and require a lot of effort to maintain. YouTube doesn't want external programs using their content so they're not gonna make it easy. Taking a look at yt-dlp, there's over 200k lines of code in that repo. Recreating even a small portion of that will be a large effort.

I think we have a few options here.

1. Just use yt-dlp in your project directly. There is a Rust wrapper.
2. Add some logic in your program that will use either stream-download or yt-dlp depending on whether it's a Youtube URL or not.
3. We modify stream-download to support getting info from yt-dlp or some other Youtube downloader library or external binary.
4. We add the Youtube logic directly into stream-download.

I'm not in favor of #​4 as it will add too much extra maintenance burden for a specific use case. I think implementing #​3 would be best, but without getting into the weeds, I can't say for sure what a proper design for that would be.

If it's possible to implement the SourceStream trait for a yt-dlp source, that should be all that's required to make it work. I'm not sure if that's possible though. If someone wants to take it on, I'd be happy to discuss the design and/or review a proof of concept. I'm also happy to make any necessary changes to the core traits in order to make them flexible enough.

I will look at implementing this myself eventually if no one else does, but I can't say when. In the meantime, I would recommend yt-dlp as mentioned above.