multipart

[masudhossain]

• Issues are for bug reports only.

• Have a question or unsure if it's a bug, then please inquire on the forum.

Brief Description

I'm trying to upload large files to digitalocean spaces with shrine + jquery file uploader. I am testing file uploads with 3mb files and multipart set to 1mb (so 3 pieces).

Expected behavior

I expected it to return the full video file compiled.

Actual behavior

When uploading with multipart to digitalocean spaces with shrine + jquery file uploader (multipart enabled), it returns only 1 piece of the total file and that is what's saved to our rails database.

Simplest self-contained example code to demonstrate issue

initliazers/shrine.rb

require "shrine/storage/s3"

s3_options = {
  access_key_id:     Rails.application.secrets.digitalocean_spaces_key,
  secret_access_key: Rails.application.secrets.digitalocean_spaces_secret,
  # region:           'nyc3',
  region:           'nyc3',
  # endpoint:          'https://nyc3.digitaloceanspaces.com',
  endpoint:          'https://nyc3.digitaloceanspaces.com',
  bucket:            Rails.application.secrets.digitalocean_spaces_bucket,
}

Shrine.storages = {
  # Shrine::Storage::S3.new(public: true, **s3_options)
  cache: Shrine::Storage::S3.new(prefix: "cache", upload_options: {acl: "public-read"}, **s3_options),
  store: Shrine::Storage::S3.new(prefix: "store", upload_options: {acl: "public-read"}, **s3_options),
}

Shrine.plugin :activerecord
Shrine.plugin :direct_upload
Shrine.plugin :restore_cached_data

System configuration

Ruby version: gem 'rails', '~> 5.0.2' ruby 2.4.2

Shrine version: shrine 2.12.0)

[benkoshy]

Hi masudhossain,

Just to clarify what you are trying to do:

Question 1: Are you wanting to upload your video file:

• directly to your Digital Ocean bucket without touching your rails app OR
• you want to upload to your app (which will then send it to the bucket)?

Question 2:

If you are uploading directly to your bucket (and not via your rails app), won't you need to make direct use of a gem like this to help you with the multi-part uploads: https://shrinerb.com/docs/getting-started#uppy-s3-multipart

https://github.com/janko/uppy-s3_multipart

I'm not sure about jquery compatibility, but it might be worth considering switching to uppy (or incorporating your tasks with uppy) to handle your uploads, because it works fairly seamlessly with shrine.

I might be mistaken, so perhaps x2 check with some folks more knowledgeable.

rgds Ben

[masudhossain]

Hi masudhossain,

Just to clarify what you are trying to do:

Question 1: Are you wanting to upload your video file:

• directly to your Digital Ocean bucket without touching your rails app OR
• you want to upload to your app (which will then send it to the bucket)?

Question 2:

If you are uploading directly to your bucket (and not via your rails app), won't you need to make direct use of a gem like this to help you with the multi-part uploads: https://shrinerb.com/docs/getting-started#uppy-s3-multipart

https://github.com/janko/uppy-s3_multipart

I'm not sure about jquery compatibility, but it might be worth considering switching to uppy (or incorporating your tasks with uppy) to handle your uploads, because it works fairly seamlessly with shrine.

I might be mistaken, so perhaps x2 check with some folks more knowledgeable.

rgds Ben

Question 1: upload through my rails app to my digitalocean bucket

question 2: We're using jquery file uploader inside our rails app to do the upload process. So user will click the upload button and then upload.

[benkoshy]

Hi masudhossain

would you be able to spin up a minimal rails app of the above? It would go a long way towards solving the problem. (Unfortunately, I'm not experienced enough with this library to debug it based on the initialiser files.)

[janko]

Shrine's presign_endpoint plugin (or direct_upload plugin on older versions) doesn't support multipart uploads. As @BKSpurgeon said, you'll need to use uppy-s3_multipart for that, though I don't know whether jQuery-File-Upload's multipart feature has some specific interface the endpoints need to conform to.

FWIW, I've been there, Shrine recommended jQuery-File-Upload originally before Uppy came along. But today I would never recommend it over Uppy, so I would suggest making the switch if it's possible.