freetube 0.7.1 subscriptions loading erratically [bug]

[mcmurphy8097234789]

Expected Behaviour: Freetube appimage boots, loads subscriptions, and refreshing of subscriptions is allowed to occur

Actual behaviour: When Freetube Appimage is loaded initially, or by clicking 'refresh,' channels load erratically. Some will load, others will not. Which ones load seems to be somewhat random. Never are all of them loaded.

Parameters:

This is the same regardless of whether proxying through invidio.us is used or not or whether videos are grabbed locally. A Tor/proxy is NOT used for API calls. Redownloading appimage does not change outcome.

Bug is reproducible on different computers on different internet networks with different IP addresses. This leads me to believe this is a problem with a function of the program, and not the current hardware or network being used.

If requested, will try to provide subscription text and debug log

Edit with OS information: OS: Solus 4.0 fortitude Kernel: x86_64 Linux 5.3.10-134.current

[ghost]

This issue has also been affecting me now for a few days, have not tried on other systems, but I have a couple of other machines I can potentially try this on during the weekend.

I have been on 0.7.1 since release and this has only started happening this week. I haven't looked at the codebase, but could it possibly be related to a youtube API change of some sort?

Here is my OS info just in case: Windows 10 Version 1809 Build 17763.379

EDIT: Should also note that going to any of the channels listed when trying to refresh subscription feed functions as expected and I can see videos that don't show up in the feed.

[LWFlouisa]

Same thing seems to be happening on this end.

OS: Linux Mint 19.1 I'm unsure of what specific build. It's an Intel Core 3.0 I think.

[linuxgirl22]

Same here (arch zen installer/mate), and just started over the last week ... 3rd time needing to change invidiou.us instance, but keeps breaking. Otherwise, can click on individual channels from the list, and sometimes an error message, but the channel runs. Too difficult a way to use, with the left column's text being so small, so am also using GTK Youtube Viewer manually (not logging in, but generally not sure how private it is). Worrying that no longer any dev responses in Riot general chat.

[LWFlouisa]

My issue seems to be caused by Invidio.us instance being blocked, going to test out another instance of invidio and see if that resolves the issue.

Yep the issue has to do with Invidio.us being blocked, I do a FreeTube search using another Invidio instance and works fine.

This is a good reminder of why we need decentralized architexture.

[ghost]

This seems to be caused by blocked invidious instances (see this reddit thread). It looks like freetube proxies everything through invidous even if you haven't enabled the option. I was unaware of this and thought it was accessing youtube directly (a la newPipe). Changing to a not-blocked instance should solve this (https://invidiou.sh was recommended in the thread and works for me).

EDIT: Hahah yeah, now I see that the it always uses invidous instances because of the setting at the bottom of the settings list... Kind of makes me wonder what's happening with the proxy videos through invidous toggle though. does it load the video urls through youtube by default? Chances are this was all obvious from the get go and I just never read about it (as per usual hahah)...

EDIT 2: @LWFlousia and @linuxgirl22 totally pointed out the invidous thing being the actual issue before I did, me not reading everything strikes again, my bad!

[linuxgirl22]

No problem @Omelettechu and, yes, I just discovered the settings make no difference, after discovering I didn't even have the invidio.us option turned on. Have tried various instances and various settings, but Freetube seems too broken at this point.

[linuxgirl22]

Today, invidio.us itself is finding and playing videos great (some channels blocked/may work with different IP?), but, whether with 'proxy through invidio.us' turned on or off, Freetube wasn't loading anything ... until I've found mention of the instance https://invidious.snopyta.org/ (I don't have 'proxy through invidio.us' turned on). When clicking through, there's a message saying 'Google currently rate limits IP addresses much faster, it could be that this instance wont work for some time' but it's working today; thanks to whoever is running this.

[ghost]

Nice find, it's probably safe to say in this case that the issue is being cause by the fact that freetube does everything through an invidous instance, so the problem actually lies within the fact that invidous instances are being struck down and/or limited access by google. AFAIK this seems to happen from time to time with invidous instances and they usually end up recovering from it, but I think a better, more long term solution, would be to have the option to access youtube APIs directly (like other youtube frontends eg: newPipe) instead of proxying through invidous for when things like this happen.

Of course the permanent solution would be getting everyone on to a decentralized video platform, such as PeerTube, but I don't see that happening for a few years (would be nice though hahah).

[linuxgirl22]

Yes, good to find that instance (Trisquel forum, I think) ... at the moment, lol, working well in the browser, but not Freetube (loads subs, but error when click through to a video), but, as you say, it can be very up and down, or some channels work on one instance, some on another.

Sometimes needing to fall back on GTK youtube-viewer at the moment (YT api, but not logging in), if neither browser instance works, but it's good to hear that things end up fixed okay.

A button to switch to YT api could be cool as backup, and decentralised would be awesome!

[LWFlouisa]

I wonder if the router has somewhat to do with it. I ask as moving to a different network I didn't have the same problems.

But I can find my profile again, and listen to music playlists.

[ghost]

@LWFlouisa It's probably not the router as everything (minus loading the actual video) is through the invidious API as stated at freetubeapp.io:

FreeTube is a YouTube client for Windows, Mac, and Linux built around using YouTube more privately. You can enjoy your favorite content and creators without your habits being tracked. All of your user data is stored locally and never sent or published to the internet. Being powered by the Invidious API, FreeTube has become one of the best methods to watch YouTube privately on desktop.

If the issue has gone away then that most likely means whichever invidious instance you were using (invidio.us by default) has solved its' youtube connectivity issues.

Also FYI for anyone following me trying to make sense of how connections to youtube are made, I actually read things this time, the proxy videos toggle actually does serve a purpose and is part of the invidious API.

Basically with the invidious API everything except watching a video is proxied through an invidious instance by default. So this means once you start loading a video, your accessing youtube directly via googlevideo.com. When you enable video proxying through invidious, instead of accessing googlevideo.com to fetch the video, you make to the request to the invidious instance which then makes the request to googlevideo.com.

Also I spelled invidious wrong in my previous posts, too lazy to go edit all that hahah.

[linuxgirl22]

Thanks for explaining exactly how Freetube works ... everything but vids was playing, and now, via switching on 'proxy through invidious', they're working.

[LWFlouisa]

Still happening. Is there not a way to regulate this instance blocking thing? This out of control instance blocking is one the reasons I no longer take Mastodon seriously as a fediverse platform.

Or at least, some way to quickly work around it.

This kind of thing never happens on Diaspora. Not sure why Invidio is any different.

It shouldn't matter if an instance is in Denmark or Japan, it should operate on the same general guidelines if it's going to enforce them, otherwise you having people blocked for largely arbitrary reasons: in practice extremely authoritarian. And not consistently so.

( It's one reason I don't even fool with Peertube anymore. )

[ghost]

TL;DR: Invidious instances aren't at fault, it's scumbag Google doing the blocking.

Just to be clear, any invidious instance does not have control on whether it gets blocked or not. Invidious scrapes the youtube API and is an unofficial front end for youtube.

Google does not endorse and is probably very against these unofficial apps scraping their web APIs. In the case of newPipe on android, all requests to youtube servers are made via your device, thus exposing your IP address to Google. Since Google is still able to mine some sort of information from you (your IP address), they would disavow such an app, but not go out of their way to shut it down.

In the case of Invidious instances, the only requests that could be linked back to your IP address via the google video server URL to fetch the video data. Now it could also be the case that users on an instance enable the "Proxy Videos" option, which would then eliminate any tracking of your IP address by Google (in terms of youtube usage).

Knowing how Invidious instances actually provide content from youtube to the end user is important to understand why the instances get blocked.

Since Google has started limiting requests from any single IP address to their youtube API, this means that Invidious instances, which would have all requests except for possibly the actual fetching of video data, are potentially sending out enough requests to be seen as a bad client by Google and subsequently blocked for spamming their API. Couple that with fact that many users on invidious most likely proxy the videos through the instance too, and it becomes more certain that an instance will be blocked.

A normal individual user would probably get nowhere near the amount of requests required to be put on the blocklist, so since apps like newPipe make requests via your own IP address for everything, such apps would not be affected by this.

So the invidious instances are not at fault here, it's all Google trying to have a monopoly on everything due to their insane fetish of data mining everyone.