bezzad / Downloader

Fast, cross-platform and reliable multipart downloader with asynchronous progress events for .NET applications.
MIT License
1.25k stars 193 forks source link

Whether the download can be resumed from where the last download was not completed #148

Closed bigbosschenyibo closed 5 months ago

bigbosschenyibo commented 11 months ago

Considering that the file may be interrupted in downloading, I will download the file xx to xx.temp first, and if the download is successfully completed, I will convert xx.temp to xx. But if the download is interrupted, I want to be able to continue starting from the left off the next time the program starts, and I want to know if you have provided the relevant features to support my scenario? Thank you in advance for your help. Below is the code I tried for this purpose:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.IO;
using System.Net;
using System.Reflection;
using System.Threading.Tasks;
using Downloader;

namespace FileDownLoader
{
    public class TestDownLoaderFromGit
    {
        private static string _downloadUrl = "https://mirrors.tuna.tsinghua.edu.cn/centos/7/isos/x86_64/CentOS-7-x86_64-NetInstall-2009.iso";

        private static string _downloadfile = "testdownload.iso";

        private List<Downloader.IDownloadService> _downloaders;

        public TestDownLoaderFromGit()
        {
            _downloaders = new List<Downloader.IDownloadService>();
        }

        public static async Task RunTest()
        {
            TestDownLoaderFromGit testDownLoaderFromGit = new TestDownLoaderFromGit();
            await testDownLoaderFromGit.StartNew();
        }

        /// <summary>
        /// file name magic
        /// </summary>
        private static int _fileNum;

        private async Task StartNew()
        {
            var downLoadFileName = $"test_{_fileNum++}_{_downloadfile}";
            var downLoadFilePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, downLoadFileName);
            var downLoadingFilePath = $"{downLoadFilePath}.temp";
            var downloadConfig = GetDownloadConfiguration();

            if (File.Exists(downLoadFilePath))
            {
                return;
            }

            if (System.IO.File.Exists(downLoadingFilePath))
            {
                //TODO: I want the downloader can  Resume  with prev breakpoint
                using (System.IO.FileStream fileStream = File.OpenWrite(downLoadingFilePath))
                {
                    var startPos = fileStream.Length;
                    downloadConfig.RangeDownload = true;
                    downloadConfig.RangeLow = startPos;
                    downloadConfig.RangeHigh = long.MaxValue;
                }
            }

            var downloadService = CreateDownloadService(downloadConfig);
            await downloadService.DownloadFileTaskAsync(_downloadUrl, downLoadingFilePath);
            Console.WriteLine("downloader end ---------->");
        }

        private DownloadService CreateDownloadService(DownloadConfiguration config)
        {
            var downloadService = new DownloadService(config);

            // Provide `FileName` and `TotalBytesToReceive` at the start of each downloads
            downloadService.DownloadStarted += OnDownloadStarted;

            // Download completed event that can include occurred errors or 
            // cancelled or download completed successfully.
            downloadService.DownloadFileCompleted += OnDownloadFileCompleted;

            return downloadService;
        }

        private void OnDownloadFileCompleted(object sender, AsyncCompletedEventArgs e)
        {
            if (e.Cancelled == false && e.Error == null)
            {
                var ds = sender as DownloadService;
                if (ds == null)
                {
                    return;
                }

                var targetFilePath = ds.Package.FileName.Substring(0, ds.Package.FileName.LastIndexOf('.'));
                File.Move(ds.Package.FileName, targetFilePath);
            }
        }

        private void OnDownloadStarted(object sender, DownloadStartedEventArgs e)
        {
        }

        private DownloadConfiguration GetDownloadConfiguration()
        {
            var cookies = new CookieContainer();
            cookies.Add(new Cookie("download-type", "test") {Domain = "domain.com"});

            return new DownloadConfiguration
            {
                BufferBlockSize = 10240, // usually, hosts support max to 8000 bytes, default values is 8000
                ChunkCount = 16, // file parts to download, default value is 1
                MaximumBytesPerSecond =
                    1024 * 1024 * 10, // download speed limited to 10MB/s, default values is zero or unlimited
                MaxTryAgainOnFailover = 5, // the maximum number of times to fail
                MaximumMemoryBufferBytes = 1024 * 1024 * 50, // release memory buffer after each 50 MB
                ParallelDownload = true, // download parts of file as parallel or not. Default value is false
                // ParallelCount = 4, // number of parallel downloads. The default value is the same as the chunk count
                Timeout = 3000, // timeout (millisecond) per stream block reader, default value is 1000
                RangeDownload =
                    false, // set true if you want to download just a specific range of bytes of a large file
                RangeLow = 0, // floor offset of download range of a large file
                RangeHigh = 0, // ceiling offset of download range of a large file
                ClearPackageOnCompletionWithFailure =
                    true, // Clear package and downloaded data when download completed with failure, default value is false
                MinimumSizeOfChunking =
                    1024, // minimum size of chunking to download a file in multiple parts, default value is 512                                              
                ReserveStorageSpaceBeforeStartingDownload =
                    false, // Before starting the download, reserve the storage space of the file as file size, default value is false //TODO:set ReserveStorageSpaceBeforeStartingDownload false
                RequestConfiguration =
                {
                    // config and customize request headers
                    Accept = "*/*",
                    CookieContainer = cookies,
                    Headers = new WebHeaderCollection(), // { your custom headers }
                    KeepAlive = false, // default value is false
                    ProtocolVersion = HttpVersion.Version11, // default value is HTTP 1.1
                    UseDefaultCredentials = false,
                    // your custom user agent or your_app_name/app_version.
                    UserAgent = $"DownloaderSample/{Assembly.GetExecutingAssembly().GetName().Version?.ToString(3)}"
                    // Proxy = new WebProxy() {
                    //    Address = new Uri("http://YourProxyServer/proxy.pac"),
                    //    UseDefaultCredentials = false,
                    //    Credentials = System.Net.CredentialCache.DefaultNetworkCredentials,
                    //    BypassProxyOnLocal = true
                    // }
                }
            };
        }
    }
}

I was confused that the file download did not work as expected.

akprogram commented 8 months ago

In the name of GOD. I have same as this problem. I checked DownloadService.cs codes and i knew, this class delete uncompleted download file, before start (resume) download, then continue, but i got errors for delete, so I close my project without resume. if everybody find solution please help me. tank you viewers and bezzad. ممنونیم بهزاد. کاملش کل. عالیه

bezzad commented 8 months ago

Hi everybody, First, make sure you set false to the ClearPackageOnCompletionWithFailure option. Because, this option tells the Downloader to clear the package and downloaded data when the download is completed with failure, but the default value is false.

Second, before downloading, keep the downloadService.Package in a variable;

// At first, keep and store the Package file to resume 
// your download from the last download position:
DownloadPackage pack = downloader.Package;

after downloading, you can stop or pause a download or the download may be completed with an error. Therefore, when you want to resume downloading at last point, just pass the package to the downloader like this:

await downloader.DownloadFileTaskAsync(pack);

Note: Sometimes a server does not support downloading in a specific range. That time, we can't resume downloads after canceling. So, the downloader starts from the beginning.

Phantom-KNA commented 4 months ago

Additional info for saved your Packed, you can use json file for this:

public void SavedPacked(DownloadPackage packet, string rutaArchivo) { string json = JsonConvert.SerializeObject(packet); File.WriteAllText(rutaArchivo, json); }

public DownloadPackage ReadPacked(string rutaArchivo) { string json = File.ReadAllText(rutaArchivo); return JsonConvert.DeserializeObject<DownloadPackage>(json); }

if (File.Exists("your path .json file")) { var packed = LeerPacked("your path .json file"); Console.WriteLine("!---- OnResume ----!"); downloader.DownloadFileTaskAsync(packed); } else { Console.WriteLine("!---- OnStart ----!"); downloader.DownloadFileTaskAsync("url file to download", " your file name"); }

Use SavePacked in dDownloadProgressChanged