bezzad / Downloader

Fast, cross-platform and reliable multipart downloader with asynchronous progress events for .NET applications.
MIT License
1.25k stars 193 forks source link

Large file download blocks other download tasks. #152

Closed incheon-kim closed 5 months ago

incheon-kim commented 8 months ago
  var downloadTask = DownloadBuilder.New()
      .WithUrl(url)
      .WithFileLocation(targetPath)
      .WithConfiguration(new DownloadConfiguration()
      {
          ParallelDownload = false,
          Timeout = 600000, // 10 min
          MaximumMemoryBufferBytes = 1024 * 1024 * 50,
          MaxTryAgainOnFailover = 5,
          BufferBlockSize = 10240,
          RangeDownload = false,
          CheckDiskSizeBeforeDownload = false,
          ClearPackageOnCompletionWithFailure = true,
          RequestConfiguration = new RequestConfiguration()
          {
              CachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore),
              KeepAlive = true,
              UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
              Headers = new WebHeaderCollection()
              {
                    // Some custom headers
              }
          }
      })
      .Build();
     await downloadTask.StartAsync();

I am trying download multiple files to network drive(UNC path) simultaneously (2~3) with code above. The problem is a download task sometimes blocks others until it is finished. And it causes timeout (or The downloader cannot continue downloading because the network or server failed to download in range.) for other tasks.

I know it might be server-side or network drive issues but I want to clarify if there is anything i missed when I write configuration.

bezzad commented 8 months ago

Hi @incheon-kim, When your server doesn't support downloading in range, any failure in the network can stop downloading because the Downloader can't continue from the last download position and raises an error to get the situation.

incheon-kim commented 8 months ago

I thought RangeDownload = false prevents that kind of situation. dotnet's System.Net.WebClient had no problem downloading files that I tried though.

bezzad commented 8 months ago

Thank you for providing more information. I haven't tested the Downloader package for network files other than the HTTP and HTTPS protocols. To further investigate and simulate the issue, it would be helpful to have additional information about the server configuration.

The Downloader package utilizes the HttpWebRequest library, which is also used in the base of WebClient. Therefore, it is designed primarily for downloading files from HTTP and HTTPS addresses. If you are attempting to download files from non-HTTP or non-HTTPS addresses, such as internal network addresses, it is recommended to thoroughly test the package and verify its compatibility.

Thank you for bringing this to our attention, and we will consider your feedback as a feature request for supporting download requests on internal networks. If feasible, we will work towards adding this capability in a future release.

Please let us know if you have any further questions or require additional assistance.

Best regards