[Bug]: Network download and buffering is significantly slower compared to browser

[RevAngel7]

Guidelines

• [X] I have encountered this bug in the latest release of FreeTube.
• [X] I have searched the issue tracker for open and closed issues that are similar to the bug report I want to file, without success.
• [X] I have searched the documentation for information that matches the description of the bug I want to file, without success.
• [X] This issue contains only one bug.

Describe the bug

Watching videos on FreeTube on 1080p often runs into buffer-underruns. Watching the same videos on browser there is no such problem.

This can confirmed with any video that buffer-underruns on FreeTube - youtube on browser is not running into the same issue.

Tested on local API. Ubuntu .deb version v0.18.0-nightly-2844 Beta on ubuntu 22.04, kernel 6.2.12 mainline, mesa 23.1, all common codecs present. 100+mbit/s download speed. 12/16GB free RAM, fast SSD with enough free space.

Expected Behavior

A "as fast as" download speed as viewing a video on my browser on youtube - with the local API of FreeTube. No buffer-underruns on 1080p (or higher).

Issue Labels

content not loading, usability issue

FreeTube Version

v0.18.0-nightly-2844 Beta

Operating System Version

ubuntu 22.04 (LTS)

Installation Method

.deb

Primary API used

Local API

Last Known Working FreeTube Version (If Any)

None, issue was with previous versions as well.

Additional Information

As I understood, the local API and FreeTube itself uses no special API key to access content from youtube. So why is it considerably slower than viewing content from youtube itself to a point where it buffer-underruns on 1080p on most videos in an environment where 100mbit/s and other hard- and software issues should not throttle in any way?

Nightly Build

• [X] I have encountered this bug in the latest nightly build.

[RevAngel7]

Definition to clarify: "1080p" means in my case 1080p60, 1080p30, 1080p23. Buffer underruns occur in all these 3 cases (even tho the contents bitrate varies).

[absidue]

Have you configured a proxy in FreeTube? #3405 says that having a proxy configured in FreeTube slows stuff down a lot.

[Joerg123]

I can confirm this behavior of the nightly builds, at least beginning with 2844, when i first tried out working with them 3 weeks ago because of an issue with the main 0.18.0 beta version because of a change Youtube did then. I did not put a comment because i'm just an user, not understanding much from the stuff, as you probable already guessed by my writing. But perhapps i can give you further information, e.g. i'm using Windows 10 and i'm not aware of having any proxy set, i'm using local api. Because for whatever reason, a few days after having problems with the 0.18.0 beta main version everything worked fine again, so i could fallback to 0.18.0 beta. Actually i always had sometimes problems with buffering 1080p, but in this cases i had to pause for 30 seconds to get enough buffer for not having buffer underuns again during the playback of the video. But with the nightly build it takes a minute to buffer 5 seconds, thus totally unusable.

I think You have to ask me specific questions for getting the right informations, hope i can help...

[tytqnxt898]

The same issue happens to me on the latest build (not nightly). The way I fixed it was using a socks5 proxy which made the bandwidth on par with browsers.

To test this issue what I did was use 2 browser (chrome and firefox) and freetube + im using the same vpn ip for the tests. While checking the nerd stats of the same video on chrome and firefox I get 200k kpbs on th bandwidth. On freetube with the same ip and video I get 8k kbps which is drastically slower.

The only fix I found was to use a socks5 proxy (which has the same ip as the vpn I'm using) and somehow it fixes it by going 200k kpbs on the bandwidth.

On the nightly build sadly I cannot use this fix because the new method to use a proxy makes it worst than not having a proxy (100-200 kpbs) 50 times slower.

@RevAngel7 are you using a vpn if so which one, could help find the issue

[RevAngel7]

Hello, and thank you all for your contributions.

To answer the questions: No, I am not using a proxy (nor TOR). Neither over FreeTube nor the system. No, I am not using my VPN with FreeTube. (Tested: using it does not change the situation.)

Thank you again for you trying to help!

[Growebis]

I though i was going crazy when some youtube videos would load perfectly fine in 1080p but others not.

[absidue]

Do you have any specific videos that this happens with? Every video I've tried so far has worked fine on my end in the nightly builds.

[Joerg123]

I tested it yesterday to comment, it didn't work: exremely slow buffering with the nightlies, but 0.18.0 beta worked. Today, it just works with every version, thus also nightlies. I don't konw...

[PikachuEXE]

Default Video Format and Allow AV1? When I have VPN (home) I use legacy format but Dash one works fine for me when no VPN enabled (work).

[RevAngel7]

Do you have any specific videos that this happens with? Every video I've tried so far has worked fine on my end in the nightly builds.

Nearly every video, but some work, some work for initial buffering and underrun some time in. On the non-nightly versions it worked sometimes to press CTRL-R to refresh the video page, sometimes this had to be repeated. I guessed at that time that servers were chosen at each loading of the video, and probably one was faster - but that was just my guess.

Default Video Format and Allow AV1? When I have VPN (home) I use legacy format but Dash one works fine for me when no VPN enabled (work).

DASH video formats (default). As per default settings AV1 was not checked (on). But since I have the necessary codecs (libaom3, libdav1d5) installed I will test it again with AV1 checked (on) and report the results later.

By the way, I had these issues with the non-nighty versions too - maybe not as significant as on the nighy version I am using now. But I had to change the version since the non-nighty stopped working for me entirely.

[RevAngel7]

Turning AV1 on did not improve the issue.

[Joerg123]

Hmm, today , again, it's not working with the nightlies, but v0.18.0 Beta is working. For a moment i thought, turning of AV1 Dash fixes the problem, but this was just for two times checking the same video and switching vise versa. After the third time swithichg AV1 off it not worked anymore.

[RevAngel7]

Changing to #2878 (freetube_0.18.0-nightly-2878_amd64.deb) and testing if that improves the issue.

[RevAngel7]

Pre-Buffering seems to work a bit better, but after that buffer-underruns occur as frequent as with the previous versions.

[absidue]

Going back and forth between old stuff and new stuff won't make a difference, it's clearly a YouTube change but as we haven't been able to reproduce the issue on our end, we can't investigate and try to solve it.

[RevAngel7]

Okay, I get that. Strangely this issue now seems sporadic and unpredictable as well. I watched 10 videos now from various world news channels, and 5 of them went fine, 5 of them underrun.

Should we keep this issue open to have an eye on user experiences? And would that help wit the investigation of the issue?

[RevAngel7]

Or: Can I provide you with more information that would help with the investigation?

Maybe I can stay updated with the nighties and write a test update from time to time. And if some users find the same issue, they maybe can contribute information about it until the puzzle turns into a readable picture?

[absidue]

Would be very useful if someone found a way to consistently reproduce this e.g. specific videos or that it happens more frequently on specific channels. Definitely keeping it open, because it's clearly an issue that multiple people are having so it's not just an issue with someone's setup, we just can't do anything about it until we have a way to reproduce the problem.

[RevAngel7]

Would be very useful if someone found a way to consistently reproduce this e.g. specific videos or that it happens more frequently on specific channels. Definitely keeping it open, because it's clearly an issue that multiple people are having so it's not just an issue with someone's setup, we just can't do anything about it until we have a way to reproduce the problem.

Okay, lets work with that. Reproducing an issue that is as sporadic as this one is a pain, especially if it works fine on the dev's side (I worked on communications equipment and software in a lab, testing and fixing issues that were sporadic as well, with multiple carrier changes included to complicate puzzles). Let's gather more information from users to this issues until we maybe can get a clearer picture of the causes.

Thank you for your effort and clear response, @absidue

[Growebis]

I find that this channel seems to have buffering underruns on most of the videos, is this the same for anyone else? https://www.youtube.com/channel/UCXoKg7Uvy4E7G3oW-Id5k1A

[shaicoleman]

At higher playback speeds (e.g. 3x), it can be consistently reproduced

[absidue]

@Growebis @shaicoleman Are you both using the nightly builds? Tried both playing at 3x speed and videos from that channel on the nightlies, but unfortunately I still can't replicate it.

[shaicoleman]

@absidue , seems fixed on the nightly build (v0.18.0-nightly-2885 Beta)

[RevAngel7]

It got better, but is still underrunning buffering on 50% of 10 videos I tested. After some underruns the recovery went fine, then catching up better than on the initial buffering.

One video in particular went really bad. Buffer underruns nearly every (visual) cache segmentation (the red/gray bar under the video ;-) on 1x speed, 1080p from start to end of the video. Tested multiple times with a restart of the video from pos. 0:00 to 1:01. At least 6 countable underruns each try, 5 times video [audio still working, video stopped until buffer catches up] and one complete underrun [both audio and video stopped, buffer circle showing]. DASH format, AV1 activated). Watching this video on browser, the whole video was buffered in aprox. 1.5 seconds, no buffer underruns therefore.

https://youtu.be/dm8wcJROo4c (Reuters - One dead, 23 wounded in Russian missile strike - 2023-04-27 - video length 1:01 minutes, 1080p file size: webm 19.1 MB, VP9 29.97 fps, opus 48KHz /// MP4 H.264, 22.9 MB, 29.97 fps, 2909 kbps, MPEG-4 AAC stereo, 44.1KHz, 127 kbps)

This was tested with freetube_0.18.0-nightly-2885_amd64.deb

[scampsi]

I'm intermittently reproducing the reported behaviour on nightly #2844 and #2885. The videos in this playlist https://youtube.com/playlist?list=PL37EkmqQJzsjVlYSpzqLVAhATrDdGxmsZ mostly. Other channels / playlists don't reproduce it. It was so bad in https://www.youtube.com/watch?v=i3DwBAa1boY that I checked the issues and updated to #2885 which doesn't seem any better.

Edit: repros on freetube-0.18.0-nightly-2885-setup-x64.exe

[RevAngel7]

I can confirm the issue from @scampsi on my setup with the video https://www.youtube.com/watch?v=i3DwBAa1boY . All 4 seconds only audio with a still picture from the video, until 7 seconds the audio buffer runs out, circling loading animation, next segment, after 4 seconds only audio, after 7 seconds full underrun. And so on and on.

This was tested with freetube_0.18.0-nightly-2885_amd64.deb

Compared to youtube in a firefox browser, the video buffers nearly 90 seconds instantly, no underruns occur there.

[PikachuEXE]

Yup I can confirm the loading is slow for https://www.youtube.com/watch?v=i3DwBAa1boY DASH only, legacy format works fine

[Joerg123]

So for me, no video is buffering normally with the nightlies, there is hardly any loading of the videos (perhapps 5sec/min), thus testing stuff is easy for me. And Yes @PikachuEXE, i actually can confirm, changing to legacy formats works perfect, so at least, this could be a go around for some people and perhapps it helps to find the actual cause.

[Growebis]

@Growebis @shaicoleman Are you both using the nightly builds? Tried both playing at 3x speed and videos from that channel on the nightlies, but unfortunately I still can't replicate it.

Yes i was using the nightly builds, the build was v0.18.0-nightly-2772 Beta. It seems to depend on the videos as some of the videos i watched were buffering a lot

[allusive-dev]

I have just installed freetube-git from the paru AUR on arch linux and haven't changed any settings. I am also experiencing constant buffering on even 720p60 quality. On normal browser youtube i can play 4k60 with no buffering.

[RevAngel7]

@PikachuEXE @absidue

Would that window with the information helpful, if users encounter slow buffering/loading videos? (ONLY) If yes is the answer, can one of you explain how to get that window?

Thank you!

[PikachuEXE]

I don't think it's needed from all affected users That screenshot is showing that the requests to something (properly video segments) is slow causing the issue However finding out the cause is another thing as we have no idea if it's VPN (I got VPN on always), YT server issue, specific video format issue, request generation (in FT and/or underlying libraries), specific video source issue, etc

I will still need to reproduce with VPN off but I might or might not be able to find a solution

[RevAngel7]

Thank you!

[Gorrrg]

I am too experiencing this issue from time to time. Comparing the gray segments that indicate buffered video, on the same video at the same time, YouTube's browser player is quicker to buffer, whereas it clearly stalls in FreeTube.

One thing to consider would be YT detecting and deliberately slowing down access via something other than the official web player. I remember youtube-dl/yt-dlp having similiar issues with YT that regularly needs to be fixed.

This means there might be ways around it if it's a bug on YouTube's side or even deliberate.

[Growebis]

One thing to consider would be YT detecting and deliberately slowing down access via something other than the official web player. I remember youtube-dl/yt-dlp having similiar issues with YT that regularly needs to be fixed.

How do we know if its a deliberate change by youtube or youtube is just messing around?

[RevAngel7]

One thing to consider would be YT detecting and deliberately slowing down access via something other than the official web player. I remember youtube-dl/yt-dlp having similiar issues with YT that regularly needs to be fixed.

How do we know if its a deliberate change by youtube or youtube is just messing around?

That depends how FreeTube identifies, I think. Google decided on several occations, that other apps than browsers should get limited bandwidth. That is why it is so complicated to have youtube on Kodi, with the creation of an API key etc. Maybe, and I guess it is, that is why FreeTube gets less bandwidth than a browser, because FreeTube is considered (from a google point of view) as bad for business, because it prevents ads from showing up.

How does FreeTube identify to the google backend?

[RevAngel7]

Or to be blunt: can FreeTube identify as (firefox) browser? I think one of the devs said, it uses the same API key that any browser uses. But maybe it is the identification of FreeTube that does not match with any browser identification that makes google throttle the connection speed.

[RevAngel7]

I just realized, that might be more complicated that it sounds. Checking the information that the firefox addon "Trace" obfuscates, there are a lot of identifying factors a browser normally sends. But on the other hand, YouTube is way faster that FreeTube on my end, and I am using the addon Trace for Firefox - and it caches videos maybe 10 times as fast as FreeTube.

[RevAngel7]

I would upload some screenshots of the configuration windows from "Trace" up here, but I think for someone to be interested in these, it is easier to install the addon to your browser yourself and check out the settings, to understand what a browser identifies by and what information can be changed regularly without youtube realizing it is getting random information and therefore throttling the connection speed.

[PikachuEXE]

Just tested #3474 and loading seems faster (If you want to test it you need to setup dev environment, might be difficult/time consuming for non dev) You can subscribe to that PR to know when it's merged (and then download nightly version to test)

[RevAngel7]

Great, anticipating a download / build we "normies" can get!

[PikachuEXE]

Get one here: https://github.com/FreeTubeApp/FreeTube/actions/workflows/build.yml

[RevAngel7]

Thank you @PikachuEXE !

Downloaded and installed it. Will test it later and report back the results here.

Thank you again, at all the contributors here, and have a nice day!

[RevAngel7]

It got considerably better!

On the tested 10 videos, 8 went without any buffering problems at 1080p.

Yeah, there is a but. 2 Videos still had buffering problems, where the audio still played, but the picture/video information stopped until a better buffer state was re-accomplished.

No complete audio and video buffer underruns (circle in the middle of the video) were there, tho. So that is a big win for 1080p video watchers of FreeTube!

Great work on this update!

As I understand, the resolution is a maximum of 1080p, right? At least that is on my client and hardware combination. Can FreeTube supply/satisfy 4K or even 8K display users at that regard, or is 1080p the maximum choice of resolution to be had? And if 4K/8K is possible, how is the new patch faring on these resolutions?

Thank you for your thoughts here! And again, thanks for the 80% fix of this issue that can be easily go to a 100% fix, if you are willing to wait a couple of seconds after loading a video, before playing it.

[RevAngel7]

I have tested the video buffering a bit more in depth, with time now. Sadly, the pre-buffering seems to run out on 50% of most videos I tested. Not 20% as in the first test.

That might be a random event, considering I used other videos to test from, but it also could mean that the video start should be delayed until at least 10 seconds are buffered.

Or it could mean that google has already adjusted to your programming changes to throttle connections.

I will keep you updated if that unwanted behaviour is constant or even getting worse over time.

Any comment on my last post @PikachuEXE and @absidue ?

[PikachuEXE]

I have no idea on video resolution I mostly work on FT features (non video playing part)

https://github.com/FreeTubeApp/FreeTube/pull/3474 is made by another member And it depends on https://github.com/LuanRT/YouTube.js So I have no idea if there is anything can be done/related to video delivery speed itself

[absidue]

I'll comment regarding the 1440p and higher qualities. YouTube provides the DASH streams in multiple formats, MP4 h264, MP4 AV1 and WebM VP9 (audio in M4A and WebM opus). YouTube only provides h264 streams up to 1080p, they provide AV1 and VP9 streams for all resolutions. The video player we use video.js only supports MP4 DASH, so we can't support playback of the WebM VP9 DASH in FreeTube, that leaves us with the h264 and AV1 streams. While the h264 ones are available for all videos all the time, YouTube doesn't have AV1 formats for all videos and even on the few videos that they do have them, they only occasionally return them in the API response, so we can only use them when they are provided.

What does that mean exactly? By default you are limited to 1080p, if you enable the "Allow AV1" setting in FreeTube it will use the AV1 formats when they are available, so if you are lucky with that setting enabled, you'll sometimes be able to get the higher resolution formats.

If you wondered why we don't just simply switch to a different player, it is something we want to do eventually but it will be a lot of work to do. It's also not one of those changes that you could only half do and then forget about the rest of it.

[RevAngel7]

Thank you for that extended explanation (and for ignoring my name mixup, sorry for that) @absidue ! That explains a lot.

[Joerg123]

I now have tested the v0.18.0-nightly-2892 Beta with dash formats turned on and tested both, with av1 and without. Everything is working, video playback is possible, buffer loads as usual with the v.0.18.0 version (which was never as fast as with firefox but enough for stable playback of 1080p 60FPS). Thus, for me, the buffering problem is acually fixed. Nice, thanks for all your support.

[Joerg123]

Hmm, i have to correct myselft, today it looks different. I side to side compared (today) with this video: https://youtu.be/m-LjEJS9UnM v0.18.0: about 15 seconds of stable buffer during the whole video no buffer underruns v0.18.0-nightly-2873: 5 seconds playback, 5 seconds paused due to buffering, 5 sec playback, 5 sec paused ..... v0.18.0-nightly-2892: only a few seconds buffer, occasional (every 30 seconds) short buffer underrun.

Thus, there is still a huge differents between the versions. What is going on? Settigs were Dash Formats, Dash AV1 off. The video was 1080p 60 FPS.