Mangadex-Archive-Devs / mangadex-archive

GNU General Public License v3.0
3 stars 0 forks source link

promisify everything in processTorrentCreationQueue & processTorrentUploadQueue #1

Open radonthetyrant opened 6 years ago

radonthetyrant commented 6 years ago

srv.js

methods: processTorrentCreationQueue, processTorrentUploadQueue

use a hacky way to serialize their queue processing. could be improved to use promises

AviKav commented 6 years ago

Is serialization necessary?

radonthetyrant commented 6 years ago

It should prevent creating/uploading up to 100 torrents at once, therefore that process is serialized.

  1. scrape mangalist page / individual chapters / download images, all in parallel
  2. wait until all is downloaded, then create all the queued up torrents sequentially
  3. wait until all is created, then upload all torrents sequentially
  4. proceed with page+1

could be improved, but works atm as it is.

AviKav commented 6 years ago

What's the downside of concurrent uploads?

ZaneHannanAU commented 6 years ago

@AviKav none unless anidex has a rate limiter

AviKav commented 6 years ago

We can use a rate limiter without serialization