Download the same file a second time

[trimechee]

Hello, sometimes I download several YouTube videos at once and I happen to add the same video twice, before the new version, M-D could detect that the file had already been downloaded and did not download the same file again but recently, M-D downloads the same downloaded file a second time, this bug happens in particular when I download a Facebook video then I receive notifications then I download the same video a second time without realizing it because I activated the copy automatically option in the clipboard and the name of the file changes because I received a Facebook or YouTube notification.....

[mhogomchungu]

I can not reproduce this.

There is a code in MD that prevents the same url to be added more than once and yt-dlp has functionality that prevents the same file from getting downloaded a second time.

The only way i can think of that will make it possible for yt-dlp to download the same file more than once is if the second download starts before the first one has a chance to finish and create the final file name.

Do you have more than one instance of MD running?

[trimechee]

Ohh we are reassured that the new version of M-D has kept the functionality that prevents the same file from getting downloaded a second time! I feared this option had disappeared!

my download folder is full of files of all types, I will try to make a new clean download folder and try with the same facebook videos and try different configurations....thank you for this reassuring explanation and I close the issue for the moment and I would reopen it if this bug happens again....

I have already seen in my donwload folder 2 versions of the same file downloaded with the same name but have different sizes, I think I downloaded the facebook video early from the live facebook finished then facebook puts post processing on the video and changes the quality and size of the video and I am wrong and I re-download this same video a second time and and yt-dlp believes that it is a new file because different size....

I will do other tests in sha Allah....another request please, sometimes my favorite YouTube channel of documentaries which apparently has been deleted or banned, this channel posts several episodes of a season of a show talking about crime or paranormal and the title of the videos posted are the same even if they are different episodes and at the time, I was using another donwloader which blocked my download by telling me the name of the video already exists and so I had to search and rename manually my downloaded videos.....

if for example we add to M-D several different youTube videos but which have the same name, for example all the episodes have the title "crims in the storm", M-D will detect that they are different videos despite the same name of the videos and automatically rename ou add a number to the files or block the download because the name of the file already exists ?

the youtube channel which posted the videos with the same name has been deleted so I cannot try to do the tests

[mhogomchungu]

By default, MD tells yt-dlp to create files with a name that has a format of title-id.extension. The "id" part is there to work around a problem of multiple videos having the same title.

Each youtube video has a unique id so putting it as part of a file name guarantees no name collision will happen when downloading different videos.

Closing this one and feel free to reopen when you can reproduce the problem.

[trimechee]

Ok, thank you :) excuse me i forget to close the issue :(

[trimechee]

@mhogomchungu Hello, I tried with the default download folder and another clean folder, M-D downloads the same file again even if the file has already been downloaded, I use this preset: "-f worstaudio -x" and the engine "yt-dlp-aria2c", perhaps it is a bug of aria2c, for example with this video:

https://www.youtube.com/watch?v=OtHtR9U-QKU

we must open the download folder and observe the behavior of M-D, M-D downloads the audio then when the download is complete, if we add the same url link of the downloaded file to M-D, Media-Downloader downloads the same file again then when the download is completed, M-D deletes the downloaded file and only one file remains which is a waste of data and hard drive, I am using the latest version of windows 10 and I have activated the "show files" option hidden" from the Windows 10 file explorer, I think we need to reopen this issue so that Windows 10 users can confirm a significant bug in order to fix this bug, thank you!

[mhogomchungu]

I tested with a different smaller in size youtube video and output is below

[media-downloader] cmd: "/home/ink/.local/share/media-downloader/bin/yt-dlp" "--match-filter" "!playlist" "--break-on-reject" "--newline" "--ignore-config" "--no-playlist" "-o" "%(title).200s-%(id)s.%(ext)s" "--output-na-placeholder" "NA" "-f" "worstaudio" "-x" "--compat-options" "2022" "--progress-template" "download:[download] {"filename":"%(progress.filename)s","downloaded_bytes":"%(progress.downloaded_bytes)s","ETA":"%(progress.eta)s","total_bytes_estimate":"%(progress.total_bytes_estimate)s","total_bytes":"%(progress.total_bytes)s","speed":"%(progress.speed)s","fragment_index":"%(progress.fragment_index)s","fragment_count":"%(progress.fragment_count)s"}" "https://www.youtube.com/watch?v=hoXWcq6AHXM"
[youtube] Extracting URL: https://www.youtube.com/watch?v=hoXWcq6AHXM
[youtube] hoXWcq6AHXM: Downloading webpage
[youtube] hoXWcq6AHXM: Downloading ios player API JSON
[youtube] hoXWcq6AHXM: Downloading android player API JSON
[youtube] hoXWcq6AHXM: Downloading m3u8 information
[info] hoXWcq6AHXM: Downloading 1 format(s): 233
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 5
[download] Destination: Taylor Swift is ECSTATIC after Mahomes finds Kelce for Chiefs' TD 🤩 ｜ NFL on ESPN-hoXWcq6AHXM.mp4
[download] 154.79 KiB / 154.79 KiB (100%) at 26.91 KiB/s, ETA NA
[ExtractAudio] Destination: Taylor Swift is ECSTATIC after Mahomes finds Kelce for Chiefs' TD 🤩 ｜ NFL on ESPN-hoXWcq6AHXM.m4a
[media-downloader] Download Completed Successfully

In this output, we see that the downloaded file is named "ESPN-hoXWcq6AHXM.mp4" and the "-x" option you added converts this already audio only file to a different audio only file named "ESPN-hoXWcq6AHXM.m4a" causing yt-dlp not to notice this file came from an already downloaded file since the original file gets deleted.

It does not make sense to extract audio from already audio only file and you should remove the "-x" option. This option should be use only when wanting audio only from file that has both audio and video.

A bullet proof solution to this problem is to use --download-archive yt-dlp option since it will notice a media is already downloaded regardless of what name the final file will end up having.

Documentation for the option says the following

--download-archive FILE         Download only videos not listed in the
                                archive file. Record the IDs of all
                                downloaded videos in it

[mhogomchungu]

Just remembered in MD, i use "-x --embed-thumbnail" because i want a thumbnail to be embedded in the audio file and "-x" is there to convert the downloaded file even if it is already audio only to a format that supports embedding thumbnails.

[trimechee]

Ohh very very important and precious infirmations and advices ! 💯 🥇 Thank you so much :)

[trimechee]

Hello, I tried with -f worstaudio , and indeed, I can download an audio file with the mp4 extension, but I cannot playback this audio file with potplayer but I can play this file with foobar2000 but there is no progress bar ....

I remember now, it was Mr Mhogomchungu who kindly advised me to add "-x" to solve this problem and indeed it solved my problems :)

I followed your advice and I added --download-archive to the preset and I think I will keep the preset -f worstaudio -x and I will try to be careful not to download the same files anymore, and even if I do it, it doesn't matter, I use the worstaudio preset so the file size is very minimal and the download is so fast with aria2c that I don't even realize which file is downloading, I barely add several files to our beloved M-D that they are downloaded at rocket speed, well done, bravoo ! 🥇