Limit simultaneous downloads to 15

[7shi]

When a large number of files are downloaded in a batch download, the download cannot keep up and a large number of files accumulate in the queue, causing an abnormal load. In my environment, the mouse pointer did not even move properly. The script stopped working when the number exceeded 12,000.

Therefore, the number of simultaneous downloads is limited to 5 10 15. This can be adjusted by changing the value of the variable maxConnections.

Since the download status is monitored, the timing of the done notification is also at the same time as all the downloads are completed.

[7shi]

The number of simultaneous connections was raised from 5 to 10 after confirming stable operation.

[7shi]

In very rare cases, download detection may fail. In that case, the download is performed, but the Promise is pending, and the number of running Promises is reduced. I am investigating and would like to put this pull request on hold. My apologies.

[7shi]

The cause has been identified. Sometimes the Promise returned by chrome.downloads.download() does not issue resolve nor reject when an error occurs. (not always an error)

This should happen not only in my code. I will try to implement a timeout as a workaround.

[7shi]

I am testing this implementation. The timeout works and no longer stalls after about 3000 consecutive downloads of files.

I will test with a much larger volume, tens of thousands of files.

[7shi]

I have modified it so that it is not fatally slow even with tens of thousands of downloads and confirmed that it works fine.

The main points are.

• Wrapped chrome.downloads.download so it can await to completion like the fetch API.
  • Set timeout for rare cases where id is not returned.
  • After the timeout, search the download history and pick up the id.
• Implemented split sending for the list which contains a large number of files, because it is trapped by the message limit.
• Clear successfully completed downloads from the history because the browser becomes unusually slow when the download history grows too large.
• Control the number of simultaneous downloads to prevent the queue from overflowing.
  • The number of simultaneous downloads was originally set to 5, but was increased to 15 in consideration of the overhead of history deletion.
  • The number can be adjusted by rewriting the maxConnections variable.

The interface is backward compatible, so it works just by replacing it. Use of split transmission is optional.

Points of behavior change.

• The number of simultaneous downloads is kept constant.
• Files that have been successfully downloaded are deleted from the download history.

These are to avoid the browser becoming so heavy that it affects system operation.

Note: To download hundreds of courses in batch mode, coursedump2022.js also needs to be modified to limit the number of simultaneous connections, but this is out of the requirements for this pull request, so it is excluded here.

[Eltaurus-Lt]

Thank you for all the work, that is quite a lot of changes. I wasn't feeling well last week, so I couldn't get to reviewing and merging it at the time, but I will try to do this in the coming days.