Control frequency of progress updates

MymsMan commented 6 years ago

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2018.10.05. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x ] I've verified and I assure that I'm running youtube-dl 2018.10.05

Before submitting an issue make sure you have:

[x ] At least skimmed through the README, most notably the FAQ and BUGS sections
[x ] Searched the bugtracker for similar issues including closed ones
[ x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[x ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:


[debug] System config: []
[debug] User config: []
[debug] Custom config: [u'--prefer-ffmpeg', u'-f', u'best[height=540][fps=25][ext=mp4]', u'--newline']
[debug] Command-line args: [u'--config-location', u'/mod/etc/youtube-dl.conf', u'-o', u'/mnt/hd2/My Video/%(title)s.%(ext)s', u'-v']
[debug] Encodings: locale ASCII, fs ASCII, out ASCII, pref ASCII
[debug] youtube-dl version 2018.10.05
[debug] Python version 2.7.1 (CPython) - Linux-2.6.18-7.1-7405b0-smp-with-libc0
[debug] exe versions: ffmpeg 2.8, ffprobe 2.8
[debug] Proxy map: {}

15/10/2018 15:49:31 - [bbc.co.uk] b0bg9rdk: Downloading video page
15/10/2018 15:49:34 - [bbc.co.uk] b0bg9rdk: Downloading playlist JSON
15/10/2018 15:49:36 - [bbc.co.uk] b0bg9rd4: Downloading media selection XML
15/10/2018 15:49:37 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:37 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:38 - [bbc.co.uk] b0bg9rd4: Downloading MPD manifest
15/10/2018 15:49:39 - [bbc.co.uk] b0bg9rd4: Downloading MPD manifest
15/10/2018 15:49:39 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:40 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:40 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:41 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:41 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:41 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:42 - [bbc.co.uk] b0bg9rd4: Downloading MPD manifest
15/10/2018 15:49:42 - [bbc.co.uk] b0bg9rd4: Downloading MPD manifest
15/10/2018 15:49:43 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:44 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:45 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:45 - [bbc.co.uk] b0bg9rd4: Downloading m3u8 information
15/10/2018 15:49:56 - [debug] Invoking downloader on u'https://bbcfmt-ic-9393c500-1464b0-vodhlsuklive.s.loris.llnwd.net/usp/auth/vod/piff_abr_full_hd/5ce6b3-b0bg9rd4/vf_b0bg9rd4_0220853d-f324-4edb-90c8-61ea8370eb35.ism/vf_b0bg9rd4_0220853d-f324-4edb-90c8-61ea8370eb35-audio_eng=320000-video=1604000.m3u8'
15/10/2018 15:49:56 - [hlsnative] Downloading m3u8 manifest
15/10/2018 15:49:56 - [hlsnative] Total fragments: 32
15/10/2018 15:49:56 - [download] Destination: /mnt/hd2/My Video/Bitz & Bob, You Can Do It Too! - Series 2, Woolly Wonder Girl.mp4
15/10/2018 15:49:57 - [download]   0.0% of ~53.90MiB at 159.59KiB/s ETA 08:45:12
15/10/2018 15:49:57 - [download]   0.0% of ~53.90MiB at 207.15KiB/s ETA 02:57:48
15/10/2018 15:49:57 - [download]   0.0% of ~53.90MiB at 212.46KiB/s ETA 01:18:26
15/10/2018 15:49:57 - [download]   0.0% of ~53.90MiB at 340.14KiB/s ETA 37:17
15/10/2018 15:49:57 - [download]   0.1% of ~53.90MiB at 453.42KiB/s ETA 18:45
15/10/2018 15:49:57 - [download]   0.1% of ~53.90MiB at 485.63KiB/s ETA 10:08
15/10/2018 15:49:57 - [download]   0.2% of ~53.90MiB at 601.18KiB/s ETA 05:36
15/10/2018 15:49:57 - [download]   0.5% of ~53.90MiB at 554.11KiB/s ETA 03:40
15/10/2018 15:49:58 - [download]   0.9% of ~53.90MiB at 603.39KiB/s ETA 02:31
15/10/2018 15:49:59 - [download]   1.9% of ~53.90MiB at 603.87KiB/s ETA 01:59
15/10/2018 15:50:00 - [download]   2.9% of ~53.90MiB at 597.70KiB/s ETA 01:48
15/10/2018 15:50:00 - [download]   3.1% of ~53.90MiB at 608.33KiB/s ETA 01:45
15/10/2018 15:50:00 - [download]   3.1% of ~53.90MiB at 608.33KiB/s ETA 01:45
15/10/2018 15:50:01 - [download]   3.0% of ~55.36MiB at 67.43KiB/s ETA 02:19
15/10/2018 15:50:01 - [download]   3.0% of ~55.36MiB at 116.51KiB/s ETA 02:19
15/10/2018 15:50:01 - [download]   3.1% of ~55.36MiB at 196.01KiB/s ETA 02:19
15/10/2018 15:50:01 - [download]   3.1% of ~55.36MiB at 331.96KiB/s ETA 02:19
.
. About 400 lines removed
.
15/10/2018 15:51:57 - [download]  99.6% of ~56.37MiB at 391.81KiB/s ETA 00:00
15/10/2018 15:51:57 - [download]  99.7% of ~56.37MiB at 516.97KiB/s ETA 00:00
15/10/2018 15:51:57 - [download]  99.7% of ~56.37MiB at 591.62KiB/s ETA 00:00
15/10/2018 15:51:57 - [download]  99.8% of ~56.37MiB at 605.33KiB/s ETA 00:00
15/10/2018 15:51:57 - [download] 100.0% of ~56.37MiB at 577.04KiB/s ETA 00:00
15/10/2018 15:51:57 - [download] 100.0% of ~56.37MiB at 577.04KiB/s ETA 00:00
15/10/2018 15:51:57 - [download] 100% of 56.37MiB in 02:00
15/10/2018 15:51:58 - [debug] ffmpeg command line: ffprobe -show_streams 'file:/mnt/hd2/My Video/Bitz & Bob, You Can Do It Too! - Series 2, Woolly Wonder Girl.mp4'
15/10/2018 15:51:59 - [ffmpeg] Fixing malformed AAC bitstream in "/mnt/hd2/My Video/Bitz & Bob, You Can Do It Too! - Series 2, Woolly Wonder Girl.mp4"
15/10/2018 15:51:59 - [debug] ffmpeg command line: ffmpeg -y -i 'file:/mnt/hd2/My Video/Bitz & Bob, You Can Do It Too! - Series 2, Woolly Wonder Girl.mp4' -c copy -f mp4 '-bsf:a' aac_adtstoasc 'file:/mnt/hd2/My Video/Bitz & Bob, You Can Do It Too! - Series 2, Woolly Wonder Girl.temp.mp4'

Description of your issue, suggested solution and other information

When piping the ouput of youtube-dl to a file with the --newline option there can be many hundreds of download progress messages - several per second which can be excessive. I would like an option to control how frequently the down load status is updated. eg --progress 30 to update the status every 30 seconds

I know I can use the --no-pogresss opion to suppress the download progress messages - I would like something in-between,

BTW It would also be nice if all ouput messages had a time stamp so it was not necessary to use awk or other tool to add time stamps

indianakernick commented 5 years ago

In my case, progress updates from the progress hook aren't frequent enough. When I use the progress hook to create a progress bar, it stutters and looks kind of unpleasant. I think it would be really nice to update progress as fast or as slow as required.

Running ffmpeg with a big file on a slow computer can sometimes take a couple of minutes. It would be nice to get progress updates for that as well. My tool just freezes while it's waiting for ffmpeg so it's not great for UX. Doing this would involve reading the stdout of the ffmpeg subprocess one line at a time instead of using p.communicate and then extracting the frame number with regex or something. Then you probably want a separate progress hook status for "postprocessing". I read somewhere that you have to pass universal_newlines=True to Popen to make it work. Not really sure what that means.

Mamadou442 commented 2 years ago

hi, is this still an issue ? if so i would want to know how to contribute to this

MymsMan commented 2 years ago

I am not aware on any change being made in this area so yes it is still an issue.

ytdl-org / youtube-dl