LuanRT / YouTube.js

A wrapper around YouTube's internal API — reverse engineering InnerTube
https://www.npmjs.com/package/youtubei.js
MIT License
3.43k stars 218 forks source link

[v10.1.0] - Initial Requests Taking Excessive Time #690

Closed sgebr01 closed 3 hours ago

sgebr01 commented 1 month ago

Steps to reproduce

Issue is not reproducible 100% of the time, but I have experienced it by just doing

Innertube.create().then((yt) => {
  yt.getHomeFeed().then((feed) => {
    console.log(feed);
  });
});

Failure Logs

Timing logs for each request are below.
524819.49ms - https://www.youtube.com/iframe_api
848.54ms - https://www.youtube.com/s/player/8d9f6215/player_ias.vflset/en_US/base.js
773.26ms - https://www.youtube.com/youtubei/v1/account/accounts/list?prettyPrint=false&alt=json.

And Debug Logs are below

[YOUTUBEJS][Session]: Found session data in cache.
[YOUTUBEJS][Session]: Session data: {"api_key": "**************", "api_version": "v1", "context": {"client": {"browserName": "Chrome", "browserVersion": "109.0.0.0", "clientFormFactor": "UNKNOWN_FORM_FACTOR", "clientName": "WEB", "clientVersion": "2.20240620.05.00", "deviceMake": "Apple", "deviceModel": "", "gl": "US", "hl": "en", "mainAppWebInfo": [Object], "memoryTotalKbytes": "8000000", "originalUrl": "https://www.youtube.com", "osName": "Macintosh", "osVersion": "10_15_7", "platform": "DESKTOP", "remoteHost": "********", "screenDensityFloat": 1, "screenHeightPoints": 1440, "screenPixelDensity": 1, "screenWidthPoints": 2560, "timeZone": "America/New_York", "userInterfaceTheme": "USER_INTERFACE_THEME_LIGHT", "utcOffsetMinutes": -240, "visitorData": "**********"}, "request": {"internalExperimentFlags": [Array], "useSsl": true}, "user": {"enableSafetyMode": false, "lockedSafetyMode": false}}}
[YOUTUBEJS][Player]: Got player id (8d9f6215). Checking for cached players..
[YOUTUBEJS][Player]: Could not find any cached player. Will download a new player from https://www.youtube.com/s/player/8d9f6215/player_ias.vflset/en_US/base.js.
[YOUTUBEJS][Player]: Got signature timestamp (19914) and algorithms needed to decipher signatures.

Expected behavior

Typically doesn't take much time for the request to go through usually taking 2 to 3 seconds at most.

Current behavior

Now can take up to 8 minutes for it to go through and get the needed data.

Version

Default

Anything else?

Isn't a network issue since YouTube works fine when I open it. I'm making the request from an Android Device, so maybe inconsistencies with the client info listed here may be the issue (session cache says that its a Mac and is on Desktop, when it's a mobile Android instead). I'm currently on NodeJS, but have also experienced this issue with React Native.

Checklist

sgebr01 commented 1 month ago

Trying generate_session_locally: true and retrieve_player: false doesn't help, with the first request being https://www.youtube.com/youtube/v1/account/aacounts_list?prettyPrint=false&alt=json which ends up taking just as much time (so it's not something specific to those first few initialization requests).

LuanRT commented 1 month ago

Did this start happening recently or has it always been that way? Personally, even after trying multiple times, I haven't been able to reproduce the issue on my side.

sgebr01 commented 1 month ago

Do you have access to an Android device (this issue seems to be exclusive to Android devices)? If so, I can send you an MRE of it that you can try to test it out.

LuanRT commented 1 month ago

Yes I do. And sure, that would help.

sgebr01 commented 1 month ago

I've made an MRE repo here and the readme should guide you through everything you need to get it to work. Make sure your phone and computer are on the same network. Like I said, it doesn't happen all of the time, you may need to refresh and make the request a couple times to try and get it to occur.

Let me know if you have any issues setting up.

sgebr01 commented 4 weeks ago

Some of my users have reported never encountering the issue (in Europe), but I'm in the US. I'm not sure if location could be an influence.

LuanRT commented 3 weeks ago

FYI I'll be looking into this today. I'm from Brazil, but I'll use a VPN if needed.

sgebr01 commented 3 weeks ago

Great, let me know if you have any questions.

If it helps, you can also shake the device and click open js debugger while you're on the app to inspect network requests.

sgebr01 commented 1 week ago

Have you made any progress on replicating the issue?

sgebr01 commented 6 days ago

I think I've found the solution, but I need some more data on what types of requests the library makes. Is it all it does just basic fetch requests? Are there any special types of requests needed for streaming live chats as well as the auth listeners?

sgebr01 commented 3 hours ago

I have great news that this isn't an issue with the library! In case someone else runs into this problem in the future, the issue was the current version of OKHttp (the native request library that powers react native's fetch on android), v4.12.0, doesn't support the Happy Eyeballs algorithm, which is used to speed up requests to servers using IPv6 and IPv4 (by making a request to both at once and killing the connection to whichever takes longer to start sending data back). What has fixed this issue for me is upgrading to v5.0.0, which adds support for the algorithm by adding these dependencies to thebuild.gradle of my app.

implementation 'com.squareup.okhttp3:logging-interceptor:5.0.0-alpha.10'
implementation 'com.squareup.okhttp3:okhttp:5.0.0-alpha.10'
implementation 'com.squareup.okhttp3:okhttp-urlconnection:5.0.0-alpha.10'

And then creating a custom fetch native module that uses that version of the library. Then you can pass that into Innertube's fetch method to make it use your fetch native module using the latest version of OKHttp rather than v4.12 provided through the default fetch function.