Download a zip package that exceeds 1 gigabyte Waiting time is too long how to solve？

[FirstBloodR]

_completionEvent.Wait(); The waiting time here is too long

[bezzad]

Please explain your issue in more detail. What is your URL and config for download, and how did you call the downloader service?

[FirstBloodR]

Please explain your issue in more detail. What is your URL and config for download, and how did you call the downloader service? Url It's my local server. downloadConfiguration = new DownloadConfiguration() { BufferBlockSize = 10240, // 通常，主机最大支持8000字节，默认值为8000。 ChunkCount = 64, // 要下载的文件分片数量，默认值为1 MaximumBytesPerSecond = 50(10241024), // 下载速度限制为1MB/s，默认值为零或无限制 MaxTryAgainOnFailover = 10, // 失败的最大次数 //OnTheFlyDownload = false, // 是否在内存中进行缓存？ 默认值是true ParallelDownload = true, // 下载文件是否为并行的。默认值为false //TempDirectory = TempDir, // 设置用于缓冲大块文件的临时路径，默认路径为Path.GetTempPath()。 Timeout = 30000, // 每个 stream reader 的超时（毫秒），默认值是1000 RequestConfiguration = // 定制请求头文件 { Accept = "/", AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate, CookieContainer = new CookieContainer(), Headers = new WebHeaderCollection(), KeepAlive = false, ProtocolVersion = HttpVersion.Version11, UseDefaultCredentials = false, UserAgent = $"DownloaderSample/{Assembly.GetExecutingAssembly().GetName().Version.ToString(3)}" } }; call the downloader service is :DirectoryInfo path = new DirectoryInfo(tempUpdatePath); string url = CdnUrl + DownLoadFileName; await downloadService.DownloadFileTaskAsync(url, path);

[bezzad]

Which version of the Downloader do you use? If your downloader lib is not the latest version so please update it to the last version and test it again.

[FirstBloodR]

Which version of the Downloader do you use? If your downloader lib is not the latest version so please update it to the last version and test it again.

version is 3.0.1

[diegoadan]

Hello, thanks for the excellent work in this project. I have the same issue reported by FirstBloodR, and I think that it is related to the way the class CouncurrentStream works when you have a slow storage and fast speed internet connection. In my case I have a fiber 1Gbps internet connection and my media storage disk is a SATA slow 6 TB disk, if I download 1GB at high speed it is downloaded fully in just seconds to RAM and takes a lot to writte down all data to my SATA disk, my temporal solution is manually throttle down the download speed usign maximumBytesPerSecond, would be nice to have a setting to set the max amount of RAM to be used for buffering. Thanks!

[bezzad]

@diegoadan Yes, this is the hardware limitation problem. We need a balance between internet and storage speed. When the internet speed exceeds the storage, it's normal to wait to complete storing operation.

Your suggestion to have the option to set the maximum allocation size of RAM is good, but I have a question when the Download exceeds the RAM limitation what can I do to control it? Slow down the download speed? or wait to release RAM and continue after that?

[FirstBloodR]

Hello, thanks for the excellent work in this project. I have the same issue reported by FirstBloodR, and I think that it is related to the way the class CouncurrentStream works when you have a slow storage and fast speed internet connection. In my case I have a fiber 1Gbps internet connection and my media storage disk is a SATA slow 6 TB disk, if I download 1GB at high speed it is downloaded fully in just seconds to RAM and takes a lot to writte down all data to my SATA disk, my temporal solution is manually throttle down the download speed usign maximumBytesPerSecond, would be nice to have a setting to set the max amount of RAM to be used for buffering. Thanks!

Hello I set maxmaximumBytesPerSecond to 2M, but the question still remains. Is it really useful to set maxmaximumBytesPerSecond? Especially in win11 the problem is more serious.

[FirstBloodR]

Hello, thanks for the excellent work in this project. I have the same issue reported by FirstBloodR, and I think that it is related to the way the class CouncurrentStream works when you have a slow storage and fast speed internet connection. In my case I have a fiber 1Gbps internet connection and my media storage disk is a SATA slow 6 TB disk, if I download 1GB at high speed it is downloaded fully in just seconds to RAM and takes a lot to writte down all data to my SATA disk, my temporal solution is manually throttle down the download speed usign maximumBytesPerSecond, would be nice to have a setting to set the max amount of RAM to be used for buffering. Thanks!

Did you solve it, brother？

[FirstBloodR]

@diegoadan

[bezzad]

I am working on this feature. It will be completed in a week at most.

[bezzad]

I released a new version (v3.0.5) to control memory buffering capacity. If you want to limit memory usage with the Downloader add the below config to your DownloadConfiguration:

 MaximumMemoryBufferBytes = 1024 * 1024 * 50,  // release memory buffer after each 50 MB