ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.41k stars 10.04k forks source link

Low download speed of youtube videos (another) #32295

Closed DmytroUsenko closed 1 year ago

DmytroUsenko commented 1 year ago
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): www.youtube.com:443
DEBUG:urllib3.connectionpool:https://www.youtube.com:443 "GET /watch?v=m3MgAOe2zSs HTTP/1.1" 200 None
[youtube] m3MgAOe2zSs: Downloading webpage
[youtube] m3MgAOe2zSs: Downloading player 8c7583ff
WARNING: [youtube] Unable to decode n-parameter: download likely to be throttled (Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.10/site-packages/youtube_dl/extractor/youtube.py", line 1674, in _n_descramble
    raise ExtractorError('Unhandled exception in decode')
youtube_dl.utils.ExtractorError: Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
)
[dashsegments] Total fragments: 4
[download] Destination: Temp/5cf7061a-c3c5-40e0-b7d7-3ccd7903c74f/How To Cook Meals Kids Will Love.f135.mp4
[download]  57.5% of ~40.00MiB at 66.87KiB/s ETA 04:20

Description

Hey there, I got stuck today with a low-speed download again. used python -m pip install git+https://github.com/ytdl-org/youtube-dl.git in order to fix Usually, updates help, but not today (

command options

options = { "format": "bestvideo[vcodec=h264][height<=?480][ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]", "outtmpl": f"{temp_subfolder}/%(title)s.%(ext)s", 'nocheckcertificate': True, 'ignoreerrors': True, "ffmpeg_location": "/usr/bin", 'postprocessors': [{ 'key': 'FFmpegVideoConvertor', 'preferedformat': 'mp4', 'preferedcodec': 'h264', }], }

RobustCoder commented 1 year ago

I was about to post this exact same issue. Thanks for reporting it first.

I started seeing very slow download speeds today, but couldn't figure out why.

Then I noticed that same log WARNING __main__.report_warning #617 - [youtube] Unable to decode n-parameter: download likely to be throttled

I tried reinstalling the latest nightly build of youtube-dl, which I do via pip install --upgrade --force-reinstall "git+https://github.com/ytdl-org/youtube-dl.git" but that did not fix it for me either.

I am not sure if this is a red herring or not, but...

After downloading a youtube video, I am trying to convert it into an mp3 file using these options: { "verbose": True, "format": "bestaudio/best", # https://stackoverflow.com/questions/60328066/youtube-dl-format-options-api "outtmpl": outtmpl, # this is a variable that is the path to the mp3 file "logger": logger, # variable that is a Python Logger object "noplaylist": True, # +++ NOT SURE YET IF I WANT THIS OPTION.... see also: https://superuser.com/questions/1341684/how-do-i-download-only-the-youtube-playlist-not-the-files-therein "postprocessors": [ { "key": "FFmpegExtractAudio", "preferredcodec": "mp3", "preferredquality": "320", } ],

"progress_hooks": [progress_hook], # on 2023-05-05 I decided to comment this out, since logging at DEBUG level gives better info

}

During the mp3 conversion, I noticed that this log as well: [ffmpeg] Post-process file E:\temp\test_music_file.mp3 exists, skipping

Is that related to the decode error above?

Furthermore, I am also then trying to use mutagen to add ID3 tags to the mp3 file.

But I am unable to open it in mutagen, I get this error: mutagen.mp3.HeaderNotFoundError: can't sync to MPEG frame

dirkf commented 1 year ago

Duplicate of #32292 for yt-dl, but the log mentions urllib3, not directly used by yt-dl, leading me to wonder if the module installed as youtube_dl is in fact it.

dirkf commented 1 year ago

@RobustCoder, please try to reproduce your other problems with the yt-dl command-line program and raise a new issue/new issues (subject to #30839) if that doesn't resolve them.

DmytroUsenko commented 1 year ago

@dirkf, what do you mean??? just for you, I put a part of the code:

import youtube_dl
logging.basicConfig(level=logging.DEBUG)

......

    options = {
        "format": "bestvideo[vcodec=h264][height<=?480][ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
        "outtmpl": f"{temp_subfolder}/%(title)s.%(ext)s",
        'nocheckcertificate': True,
        'ignoreerrors': True,
        "ffmpeg_location": "/usr/bin",
        'postprocessors': [{
            'key': 'FFmpegVideoConvertor',
            'preferedformat': 'mp4',
            'preferedcodec': 'h264',
        }],
    }

    url = youtube_link
    try:
        with youtube_dl.YoutubeDL(options) as ydl:
              ydl.download([url])
....
_the rest of the code_
dirkf commented 1 year ago

Read my comment again. I think it was clear enough (also study #30839).

Your command is basically yt-dl --f "bestvideo[vcodec=h264][height<=?480][ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]"-o "wherever/%(title)s.%(ext)s" --no-check-certificate -i --ffmpeg-location /usr/bin --recode-video mp4 whichever_URL

preferedcodec isn't a known parameter of the yt-dl FFMpegPostProcessor.

If, as usual, there are no paths in the $PATH ahead of /usr/bin that might have ffmpeg, that option is redundant.

DmytroUsenko commented 1 year ago

Sorry. yes. postprocess was from my test repo. But the server has options without it, and logs above were from the server. Options on server are:

options = {
        #"format": "bestvideo[height<=360][ext=webm]+bestaudio[ext=webm]/best[height<=360][ext=webm]/best[ext=webm]",
        "format": "bestvideo[height <=? 480][ext = mp4]+bestaudio[ext = m4a] / best[ext = mp4]",
        "outtmpl": f"{temp_subfolder}/%(title)s.%(ext)s",
        'nocheckcertificate': True,
        'ignoreerrors': True,
        "ffmpeg_location": "/usr/bin",
    }

and it doesn't work now but it worked one week ago.

Vangelis66 commented 1 year ago

@ProvisionlabTeam

Respectfully, why didn't you follow and adhere to the Broken site report "New issue template" ? Your log contains:

[youtube] m3MgAOe2zSs: Downloading webpage
[youtube] m3MgAOe2zSs: Downloading player 8c7583ff
WARNING: [youtube] Unable to decode n-parameter: ...

and a search on recently submitted new open/closed issues would have pointed to #32292, which I submitted 2 1/2 days ago...

I was about to post this exact same issue. Thanks for reporting it first.

Same comment applies here, I guess 😜 ; OP wasn't "first", in any case...

Usually, updates help, but not today

I tried reinstalling the latest nightly build of youtube-dl, ... but that did not fix it for me either.

To both: If one stumbles upon a new breakage, it'd be worthwhile to first check the youtube-dl master branch,

https://github.com/ytdl-org/youtube-dl/commits/master

to inquire whether such a breakage has been addressed in code - if not, attempting to update via git+https://github.com/ytdl-org/youtube-dl.git would be moot as far as that specific breakage is concerned (but, OTOH, might prove advantageous with respect to other bugs fixed since the last time you updated 😉 ) ...

To avoid any possible misunderstanding, I'm not being overly argumentative here, just trying to be helpful (as a simple yt-dl user) in any eventual, future, yt-dl predicament you find yourselves in 😉 ...

Best regards ...

DmytroUsenko commented 1 year ago

@dirkf, your last commit in master was two weeks ago. Indeed I see that one guy provided a clear fast fix two days ago. But I missed it in the thread because it had not explained my issue explicitly. About search- agree. My bad. I could check it, lol. Will be appreciated if you make a fast patch, if any. and thanks for the fast turnaround

dirkf commented 1 year ago

There's a lot to do, both here and IRL. But the equivalent fix is almost ready to be uploaded.

DmytroUsenko commented 1 year ago

@dirkf, in any case, many thanks for what you are doing!! Appreciate it!

RobustCoder commented 1 year ago

@RobustCoder, please try to reproduce your other problems with the yt-dl command-line program and raise a new issue/new issues (subject to #30839) if that doesn't resolve them.

@dirkf: I think that my issue is likely the same as what has already been reported here and in 32292, so I will wait for your patch for this issue before I investigate further to see if it also clears up my mp3 tag issue or not.

UPDATE: I just installed the master version of youtube-dl as per my comment below.

Slow download speeds resolved! Thanks much for your quick work!

But my mp3 tag issue remains... Will investigate further, and either post here or on mutagen. I may try using eyeD3, since the Windows programs mp3tag and foobar2000 all read the ID3 tags in my mp3 files downloaded by youtube-dl just fine.

Vangelis66 commented 1 year ago

so I will wait for your patch for this issue before I investigate further

... No need to wait any longer though 😉 , the fix for original issue #32292 has already been committed in master :

a2534f7b888416e872d5afd1862eb3e30fc69fc7

, so you only need to update your yt-dl instance...

DmytroUsenko commented 1 year ago

@dirkf , I am very sorry, but I still have the same issue. Maybe I missed something?

  Cloning https://github.com/ytdl-org/youtube-dl.git to /tmp/pip-req-build-91r7iz9j
  Running command git clone --filter=blob:none --quiet https://github.com/ytdl-org/youtube-dl.git /tmp/pip-req-build-91r7iz9j
  Resolved https://github.com/ytdl-org/youtube-dl.git to commit a2534f7b888416e872d5afd1862eb3e30fc69fc7
  Preparing metadata (setup.py) ... done

made clone of your fresh version and still see the same

[youtube] 03OZsmDV56k: Downloading webpage
WARNING: [youtube] Unable to decode n-parameter: download likely to be throttled (Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.10/site-packages/youtube_dl/extractor/youtube.py", line 1674, in _n_descramble
    raise ExtractorError('Unhandled exception in decode')
youtube_dl.utils.ExtractorError: Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
)
[dashsegments] Total fragments: 2
[download] Destination: Temp/7ac53038-6b71-44bc-9e8e-fe7aa4469784/iPad Killer or Clone.f397.mp4
[download]   8.1% of ~20.00MiB at 43.99KiB/s ETA 07:07
Vangelis66 commented 1 year ago

I still have the same issue. Maybe I missed something?

Purge the cache first via the --rm-cache-dir commandline flag?

DmytroUsenko commented 1 year ago

Yeah. You are funny.

On Mon, Jun 12, 2023, 01:19 Vangelis66 @.***> wrote:

I still have the same issue. Maybe I missed something?

Purge the cache first via the --rm-cache-dir commandline flag?

— Reply to this email directly, view it on GitHub https://github.com/ytdl-org/youtube-dl/issues/32295#issuecomment-1586358905, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABY467PSH62MCSDUH2WRPHDXKY76BANCNFSM6AAAAAAZBVEKFY . You are receiving this because you were mentioned.Message ID: @.***>

RobustCoder commented 1 year ago

@Vangelis66

Same comment applies here, I guess 😜 ; OP wasn't "first", in any case...

Sorry for missing 32292. I found this thread first, and wrongly assumed it was the first report. In my defense, I was searching for slow download speeds!

To both: If one stumbles upon a new breakage, it'd be worthwhile to first check the youtube-dl master branch,

https://github.com/ytdl-org/youtube-dl/commits/master

to inquire whether such a breakage has been addressed in code - if not, attempting to update via git+https://github.com/ytdl-org/youtube-dl.git would be moot as far as that specific breakage is concerned (but, OTOH, might prove advantageous with respect to other bugs fixed since the last time you updated 😉 )

I am very new to youtube-dl. And git is a complex beast of vast mysteries to me. So I would appreciate it if you could clear up a few things for me.

I know that the last official release is 2021-12-17, I guess what you get from executing pip install youtube-dl

I read here to use: pip install --upgrade --force-reinstall "git+https://github.com/ytdl-org/youtube-dl.git" But now that I read that comment more carefully, I have no idea what the above gives you. Do you?

Based on your suggestion to use the master branch, I found this post by dirkf that recommends either of these commands to install the master branch:

python3 -m pip install --force-reinstall 'git+https://github.com/ytdl-org/youtube-dl.git@master#egg=youtube_dl'
python3 -m pip install --force-reinstall 'https://github.com/ytdl-org/youtube-dl/archive/refs/heads/master.tar.gz'

(I never bother with the python3 -m prefix myself, because I am always installing in my Anaconda's base environment, so I know exactly what python is being used.)

I post the above here for the benefit of any other newbies.

Vangelis66 commented 1 year ago

In my defense, I was searching for slow download speeds

For starters, I'm not a prosecutor, so I did NOT accuse you of anything for that matter 😜 😄 ; I simply emphasised the correct procedure when about to submit a new issue here in this tracker, which is always visit first pinned issue #30839, read it in its entirety and with close attention paid to its content, then, if you indeed decide to go through with filing an issue, use the available new issue templates...

You are right, though, that a "proper" search term in the current issue list is somewhat "subjective"; one has to first "study", so to speak, and understand what the given errors are about, and then proceed to search for relevant info in the issues tracker; doesn't help if, as you said, one's new to the app; Unable to decode n-parameter is a very common/frequent (at least in the past) error associated with youtube.com, when Google, in their constant strive to thwart downloaders, push out specialised player code to induce throttling (and make downloading unappealing/impractical...).

What I do myself is visit in my browser successively

https://github.com/ytdl-org/youtube-dl/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc

https://github.com/ytdl-org/youtube-dl/issues?q=is%3Aissue+sort%3Aupdated-desc+is%3Aclosed

and then manually search those two lists for anything related 😜 ; with youtube.com itself being a popular service the app supports, any eventual breakage report is bound to appear therein, sooner or later...

I guess what you get from executing

pip install youtube-dl

In this scenario, pip behaves as a package manager and will fetch+install the latest version of the youtube-dl python module that is available on PyPI

pip install --upgrade --force-reinstall "git+https://github.com/ytdl-org/youtube-dl.git"

But now that I read that comment more carefully, I have no idea what the above gives you. Do you?

In this scenario, pip will fetch+install the source code of youtube-dl NOT from PyPI, but from its GitHub repository, more specifically it will install the very latest code snapshot in the master branch; the "git+" part in the command implies you have a working version of git in your system (not the default under Windows, though...); to learn more about the posted commands, you should visit the relevant pip documentation:

https://adamj.eu/tech/2019/03/11/pip-install-from-a-git-repository/

https://pip.pypa.io/en/stable/cli/pip_install/#pip-install-examples

Greetings 😄

Later addition: Do note that relying on youtube-dl --version after installing from git will give false clues, because source file version.py still contains 2021.12.17 as the app's version...