uploadcare / uploadcare-ruby

Ruby API client that handles uploads and further operations with files by wrapping Uploadcare Upload and REST APIs.
https://uploadcare.com
MIT License
39 stars 28 forks source link

Batch file storage/deletion #32

Closed vizvamitra closed 7 years ago

vizvamitra commented 7 years ago

closes #21

I've added two new methods:

Both of them accept either an array containing UUIDs/Uploadcare::Api::Files or an Uploadcare::Api::FileList.

Notes

Please note the changes in lib/uploadcare/rest/connections/api_connection.rb. The DELETE /files/storage/ endpoint accepts UUIDs of files to be deleted. It accepts them in the form of a json array in request's body. As far as I know, there is no way to send an unamed array in a url-encoded format (which we use now), so I was forced to change the request params encoding strategy to json. Tests are passing, so I think this won't cause any issues.

vizvamitra commented 7 years ago

@dmitry-mukhin I see two ways to do so:

  1. We limit max input size to 100. One #store_files / #delete_files call - one request. Return value = api response. #store_files / #delete_files methods remain simple, which is good.

    In this case, storage/deletion of FileList becames problematic because we can't tell how much files are there in total before they all will be loaded. To handle this we can add separate methods, #store and #delete, to FileList class. They'll use #store_files and #delete_files internaly, which will be called multiple times if there is more than 100 files in a FileList. To provide user with API responses, we can make this methods to acceps a block in which we'll pass each response:

list = api.file_list

list.delete do |response| # this block will be called for each internat #delete_files request
  if response['problems'].any?
    # ...
  end
end
  1. We add the same block as above to #store_files and #delete_files methods, passing each API response into it. In this case those methods became smart:
uuids # => # array of > 100 uuids
problems = {}

api.delete_files(uuids) do |response| # will be called (uuids.size / 100 + 1)  @times
  problems.merge!(response['problems']) if response['problems'].any?
end
dmitry-mukhin commented 7 years ago

We've decided to go with the 2nd option.

vizvamitra commented 7 years ago

@dmitry-mukhin I've made the final (I hope so) changes. Please review again.

vizvamitra commented 7 years ago

@dm, :tada:

dm commented 7 years ago

@vizvamitra ?

dmitry-mukhin commented 7 years ago

@dm I believe @vizvamitra meant @dmitry-mukhin :p but you can rejoice with us too!

vizvamitra commented 7 years ago

@dm my mistake, sorry for that

dm commented 7 years ago

@dmitry-mukhin @vizvamitra No worries. Be excellent to each other and party on dudes! 👍 🎉