Receiving 403 errors - all videos - main branch

[nick42d]

Hi there, thanks for the excellent library. Reporting in with a possible breaking change from youtube.

Repro steps:

1. Clone main
2. cd cli
3. cargo r -- download -vvv --id 'https://www.youtube.com/watch?v=FZ8BxMU3BYc (example video from homepage).

Error below.

Night Lovell- I Know Your Ways (Official Lyric Video) is downloading...

⠋ [00:00:00] [█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 0 B/0 B (0.0s)thread 'tokio-runtime-worker' panicked at cli/src/main.rs:149:58:
called `Result::unwrap()` on an `Err` value: Reqwest(reqwest::Error { kind: Status(403), url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("rr1---sn-fpqxc5oq-2xgs.googlevideo.com")), port: None, path: "/videoplayback", query: Some("expire=1722629213&ei=_eesZoK9IrS-juMP1bXqaA&ip=180.150.80.137&id=o-ABzySAV2WX2Tkf6qajNJPbSzN4Dt9hrjgvHU2yMkgYcR&itag=18&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=0_&mm=31%2C29&mn=sn-fpqxc5oq-2xgs%2Csn-hxa7zn7z&ms=au%2Crdu&mv=m&mvi=1&pl=22&initcwndbps=1795000&bui=AXc671In62MWmYZHEd3DFPA9b4VHET2vnxkJHIz5DMNxbYyseByLHSWYsiwxYCjsA4glmwDBvMx5COoT&spc=NO7bAR_XBV4ci4_lTo9vks5U8aW-MxeByVNMV7BCdtJHXY99W5_zTrEzZQiSKo4&vprv=1&svpuc=1&mime=video%2Fmp4&ns=qPFtopvCwJdVP4MeZyhORXkQ&rqh=1&cnr=14&ratebypass=yes&dur=192.075&lmt=1705918895201156&mt=1722607067&fvip=1&c=WEB&sefc=1&txp=4538434&n=0kwSgrYgO4dtMnnx&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Ccnr%2Cratebypass%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AGtxev0wRQIgEFtFgyBODdDrGkAEjCN5wliKi4uYddrZ-yZyV6JuxEECIQCIJVZv6W4aOCNM24rKeMuc3bRMfrlDBtSKSfJIJx0zcQ%3D%3D&sig=AJfQdSswRAIgJCfPQFQ7cc2UTf7TfuKaKV2DQktaQ1qii7oPSob2fkECIDLXvb-WmM6sTszsIj60oxVZgr2iXn6Sr4XV6DTT3Ul5"), fragment: None } })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
ERROR [rusty_ytdl:30]: task 21 panicked

Also tried a bulk video download using my crate ytmapi_rs, receieved an error from each song.

use ytmapi_rs::common::YoutubeID;

#[tokio::main]
async fn main() {
    let yt = ytmapi_rs::YtMusic::from_cookie_file("/home/nickd/.config/youtui/cookie.txt").await.unwrap();
    let song_ids = yt.search_songs("Beatles").await.unwrap().into_iter().map(|s| s.video_id);
    for id in song_ids {
        let video = rusty_ytdl::Video::new(id.get_raw()).unwrap();
        let path = std::path::Path::new(r"test.mp3");
        let res = video.download(path).await;
        println!("{:?}", res.is_err());
    }
}

[nick42d]

I'm not sure I know enough about the topic to contribute code, but I did do some digging and the following issues may be related: https://github.com/DzenanJupic/rustube/issues/73, https://github.com/ytdl-org/youtube-dl/issues/32815

[Mithronn]

Thanks for your feedback. I was aware of the error and I published https://github.com/Mithronn/rusty_ytdl/commit/7606f78958f8dc97af74e702382eaed769f09d1f it seems fixed but sometimes some video formats give 403 error. I'm investigating now

[nick42d]

Thanks for your feedback. I was aware of the error and I published 7606f78 it seems fixed but sometimes some video formats give 403 error. I'm investigating now

Nice one, will see if I can run some stress testing on the new version.

[nick42d]

From a little testing, doesn't seem to be a great deal of consistency to it. Videos will seemingly fail at random, and succeed after retrying. Tried changing filtering options to no impact either. Here's a gist of what I was trying if you find it at all useful. https://gist.github.com/nick42d/f1ff0d1c29dc85256b0a50900c7f39e7

Example output:

video: "Eleanor Rigby (Remastered 2015)" retries: 0
 download succeeded: true
Task finished
video: "The Long And Winding Road (Remastered 2009)" retries: 0
 download succeeded: true
Task finished
video: "Here Comes The Sun (Remastered 2009)" retries: 0
 download succeeded: true
Task finished
video: "When I'm Sixty Four (Remastered 2009)" retries: 0
 download succeeded: true
Task finished
video: "Blackbird (Remastered 2009)" retries: 0
 download succeeded: true
Task finished
video: "Do You Want To Know A Secret (Remastered 2009)" retries: 0
 download succeeded: true
Task finished
video: "Oh! Darling (Remastered 2009)" retries: 1
 download succeeded: true
Task finished
video: "Helter Skelter (Remastered 2009)" retries: 1
 download succeeded: true
Task finished
video: "The Honeymoon Song (Live At The BBC For \"Pop Go The Beatles\" / 6th August, 1963)" retries: 0
 download succeeded: true
Task finished
video: "Something (Remastered 2015)" retries: 0
 download succeeded: true
Task finished
video: "A Day In The Life" retries: 1
 download succeeded: true
Task finished
video: "Oh! Darling (Anthology 3 Version)" retries: 1
 download succeeded: true
Task finished
video: "In My Life (Remastered 2009)" retries: 2
 download succeeded: true
Task finished
video: "Come Together (Remastered 2015)" retries: 1
 download succeeded: true
Task finished
video: "While My Guitar Gently Weeps (Anthology 3 Version)" retries: 2
 download succeeded: true
Task finished
video: "Don’t Let Me Down (First Rooftop Performance)" retries: 4
 download succeeded: true
Task finished
video: "While My Guitar Gently Weeps" retries: 3
 download succeeded: true
Task finished
video: "Help! (Live From The ABC Theatre, Blackpool, UK/1965 / Anthology 2 Version)" retries: 3
 download succeeded: true
Task finished
video: "Norwegian Wood (This Bird Has Flown) (Remastered 2009)" retries: 4
 download succeeded: true
Task finished
video: "Let It Be (Remastered 2009)" retries: 5
 download succeeded: false
Task finished

[Mithronn]

https://github.com/Mithronn/rusty_ytdl/commit/d4798aaab5d8079d8e0fd2a354325d65685a712f seems to have solved the problem.