AssetSync / asset_sync

Synchronises Assets between Rails and S3
1.88k stars 346 forks source link

Let configuration define the number of threads to upload concurrently #395

Open oldigor opened 4 years ago

oldigor commented 4 years ago

I guess it is necessary to change only this piece of code

if self.config.concurrent_uploads
        threads = ThreadGroup.new
        # Upload new files
        local_files_to_upload.each do |f|
          next unless File.file? "#{path}/#{f}" # Only files.
          threads.add(Thread.new { upload_file f })
        end
        sleep 1 while threads.list.any? # wait for threads to finish uploading

in thttps://github.com/AssetSync/asset_sync/blob/11d206f316d67df9c8076bad9f5d5b5ba7f73d4c/lib/asset_sync/storage.rb#L245

PikachuEXE commented 4 years ago

I have no idea how to limit thread count properly Any example code from other libraries you can find?

RickCSong commented 4 years ago

Created a PR to enable this :)