search

Tyrrrz / YoutubeExplode

Abstraction layer over YouTube's internal API
MIT License
2.86k stars 482 forks source link

403 error when running on mobile #519

Closed trungnt2910 closed 2 years ago

trungnt2910 commented 3 years ago

YoutubeExplode doesn't work on the Uno platform for WASM when the User-Agent string is set to a mobile device. The link fetcher works fine, but trying to access it gives a 403 error.

It works fine on Desktop devices, however, so I think the problem comes from either YoutubeExplode or Youtube itself.

Tyrrrz commented 3 years ago

Can you reproduce it consistently?

trungnt2910 commented 3 years ago

Yes. I've tested on my own computer and several public ones. The web app works fine.

However, on my android emulator (Bluestacks 4) on Chrome, the web app fails to download the video. Also, whenever I enable the mobile emulation mode to change the user agent of my Edge browser to a mobile one, the same 403 error occurs.

Tyrrrz commented 3 years ago

Hm. Can you identify what's different between requests sent from Edge with and without mobile emulation?

trungnt2910 commented 3 years ago

They're basically the same: Without mobile emulation: image With mobile emulation: image

Both have a few failed ajax requests (because I'm using the latest version on NuGet, which is a really old one, those requests don't appear on the latest build from master).

They all query the manifests, and generate download links.

The only difference is the user-agent: Without mobile emulation:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36 Edg/89.0.774.63

With mobile emulation:

Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Mobile Safari/537.36 Edg/89.0.774.63

And also, out of all the 403 requests, there are a few mysterious successful 200 requests, on the mobile version. They, however, all return a zero-length json response.

Request URL: https://cors.bridged.cc/https://r3---sn-n4v7sn7y.googlevideo.com/videoplayback?expire=1617136405&ei=tTZjYOyPHtvPkgbzxJzoCw&ip=54.219.171.152&id=o-AGy--_khHvfkjcs-yJY2y2bg9dfTi_pLVbo157iOzDE5&itag=18&source=youtube&requiressl=yes&mh=Bd&mm=31%2C26&mn=sn-n4v7sn7y%2Csn-a5meknle&ms=au%2Conr&mv=m&mvi=3&pl=23&initcwndbps=453750&vprv=1&mime=video%2Fmp4&ns=DLc9e8QXWo0ZSQ8pE3XiAY4F&gir=yes&clen=12826583&ratebypass=yes&dur=245.249&lmt=1577632939612645&mt=1617114540&fvip=3&fexp=24001373%2C24007246&c=MWEB&txp=5531432&n=OOynJHVKuVgi82Oilz_gZ&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAIlQDVhCjwcaM6zst1mOQ0wPnY5_y7U4cmlstwAOXDmGAiEAidnckquBxw4kVKDuOEBV7wOxw3rCWHEPIQwqT1h_pCw%3D&sig=syhypXkBvtu2RcPukrtcb9vfD5FwhQYyc02Lfy-fYZCSGICgt16hF3ld6bVWnlmYa18QVSexHw1AO6Uv74qcM_CZ-dAIARw8JQ0qOgqOg
Request Method: OPTIONS
Status Code: 200 
Remote Address: 54.183.156.92:443
Referrer Policy: strict-origin-when-cross-origin
access-control-allow-headers: range
access-control-allow-methods: GET
access-control-allow-origin: *
access-control-expose-headers: access-control-allow-origin,access-control-allow-methods,access-control-allow-headers
content-length: 0
content-type: application/json
date: Tue, 30 Mar 2021 14:33:35 GMT
x-amz-apigw-id: dAV97FTlSK4FugQ=
x-amzn-remapped-connection: close
x-amzn-remapped-date: Tue, 30 Mar 2021 14:33:35 GMT
x-amzn-requestid: 1609bca5-1270-47c9-83ee-bed084a3aa87
x-amzn-trace-id: Root=1-606336bf-69a461e07bee10556be4b259;Sampled=0
x-powered-by: Express
:authority: cors.bridged.cc
:method: OPTIONS
:path: /https://r3---sn-n4v7sn7y.googlevideo.com/videoplayback?expire=1617136405&ei=tTZjYOyPHtvPkgbzxJzoCw&ip=54.219.171.152&id=o-AGy--_khHvfkjcs-yJY2y2bg9dfTi_pLVbo157iOzDE5&itag=18&source=youtube&requiressl=yes&mh=Bd&mm=31%2C26&mn=sn-n4v7sn7y%2Csn-a5meknle&ms=au%2Conr&mv=m&mvi=3&pl=23&initcwndbps=453750&vprv=1&mime=video%2Fmp4&ns=DLc9e8QXWo0ZSQ8pE3XiAY4F&gir=yes&clen=12826583&ratebypass=yes&dur=245.249&lmt=1577632939612645&mt=1617114540&fvip=3&fexp=24001373%2C24007246&c=MWEB&txp=5531432&n=OOynJHVKuVgi82Oilz_gZ&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAIlQDVhCjwcaM6zst1mOQ0wPnY5_y7U4cmlstwAOXDmGAiEAidnckquBxw4kVKDuOEBV7wOxw3rCWHEPIQwqT1h_pCw%3D&sig=syhypXkBvtu2RcPukrtcb9vfD5FwhQYyc02Lfy-fYZCSGICgt16hF3ld6bVWnlmYa18QVSexHw1AO6Uv74qcM_CZ-dAIARw8JQ0qOgqOg
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9,vi;q=0.8
access-control-request-headers: range
access-control-request-method: GET
origin: https://trungnt2910.github.io
referer: https://trungnt2910.github.io/
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Mobile Safari/537.36 Edg/89.0.774.63
expire: 1617136405
ei: tTZjYOyPHtvPkgbzxJzoCw
ip: 54.219.171.152
id: o-AGy--_khHvfkjcs-yJY2y2bg9dfTi_pLVbo157iOzDE5
itag: 18
source: youtube
requiressl: yes
mh: Bd
mm: 31,26
mn: sn-n4v7sn7y,sn-a5meknle
ms: au,onr
mv: m
mvi: 3
pl: 23
initcwndbps: 453750
vprv: 1
mime: video/mp4
ns: DLc9e8QXWo0ZSQ8pE3XiAY4F
gir: yes
clen: 12826583
ratebypass: yes
dur: 245.249
lmt: 1577632939612645
mt: 1617114540
fvip: 3
fexp: 24001373,24007246
c: MWEB
txp: 5531432
n: OOynJHVKuVgi82Oilz_gZ
sparams: expire,ei,ip,id,itag,source,requiressl,vprv,mime,ns,gir,clen,ratebypass,dur,lmt
lsparams: mh,mm,mn,ms,mv,mvi,pl,initcwndbps
lsig: AG3C_xAwRgIhAIlQDVhCjwcaM6zst1mOQ0wPnY5_y7U4cmlstwAOXDmGAiEAidnckquBxw4kVKDuOEBV7wOxw3rCWHEPIQwqT1h_pCw=
sig: syhypXkBvtu2RcPukrtcb9vfD5FwhQYyc02Lfy-fYZCSGICgt16hF3ld6bVWnlmYa18QVSexHw1AO6Uv74qcM_CZ-dAIARw8JQ0qOgqOg

And one more thing, links generated by the Desktop version can be used on other devices and on mobile browsers, before they expire. However, links generated by the Mobile version returns 403 everywhere.

Tyrrrz commented 3 years ago

I wonder if it could be because we're setting user-agent header like this?

https://github.com/Tyrrrz/YoutubeExplode/blob/cb115a9319e19cdbfc6cde610df146524f6e44be/YoutubeExplode/YoutubeClient.cs#L75-L79

trungnt2910 commented 3 years ago

I don't think so. That line of code actually explains why it works on Xamarin Forms and stuff but fails on Mobile WebAssembly. Chromium has a bug that disallows the User-Agent to be changed. But why are the links provided to the mobile platform different? Can you reverse engineer those links?

Tyrrrz commented 3 years ago

Do you mean the cors.bridged.cc part? It looks like it's coming from this CORS proxy: https://app.cors.bridged.cc/

trungnt2910 commented 3 years ago

Do you mean the cors.bridged.cc part? It looks like it's coming from this CORS proxy: https://app.cors.bridged.cc/

This should have nothing to do with the CORS proxy, as it simply forwards the requests.