Minio compatibility

[mohanklein]

Hey! Is there a way of pre signing urls for multipart uploads to minio with this package? sorry, I am really lost by all the possibilities right now. I want to use minio as storage and have the user directly upload huge files via browser without knowing the credentials :-)

Cheers!

[janko]

You should just be able to configure the AWS SDK to point to Minio:

require "uppy/s3_multipart"

resource = Aws::S3::Resource.new(
  access_key_id:     "<MINIO_ACCESS_KEY>", # "AccessKey" value
  secret_access_key: "<MINIO_SECRET_KEY>", # "SecretKey" value
  endpoint:          "<MINIO_ENDPOINT>",   # "Endpoint"  value
  region:            "us-east-1",
  force_path_style:  true,
)

bucket = resource.bucket("<MINIO_BUCKET>") # name of the bucket you created

Uppy::S3Multipart::App.new(bucket: bucket)

[mohanklein]

yeah man! thank you so much, works nicely!

uppy client using AwsS3Multipart now returns a shared link like:

http://192.168.1.2:9000/user-uploads/33bc50e10510af9cfcb8a2edca9ba4c0.wav?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=L12BDEGQ6WAJ96UPZDMF%2F20181204%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20181204T141428Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=41bc9dd86cb3f1dc351eecbe772155b21c125063c61303135874ee7cc0bb76f8

I set the bucket to have download permissions via mc policy myminio/user-uploads. Is there a simple way of directly returning the "public download" url without the X-Amz... params so I don't have to put more storage knowledge into my frontend?

Sorry for my questions but I am really havin a hard time understanding all that huge uploads/object storage stuff as I am an "old school guy" but I need it and it makes perfectly sense to me :-)

[janko]

You could just strip the query parameters on the frontend.

uppy.on('upload-success', function (file, data, uploadURL) {
  var publicURL = uploadURL.split('?')[0]
  // ...
end

Note that for public links to work you need to mark your S3 bucket as public in the AWS console (by default it's private).

[janko]

Going to close this, let me know if you still have issues regarding this.

[janko]

I've just released version 0.2.0 which adds a :public option for making uploads public and returning a public URL without query parameters:

Uppy::S3Multipart::App.new(bucket: bucket, public: true)

[mohanklein]

@janko-m nice! tough I forked due to a lack of time as I couldn't find a quick abstract solution for a PR and needed to go on in the project for which I use your repo ... but maybe some hints of my customization: Added acl: ... so it becomes "overrideable":

r.post "multipart" do
    ...
    result = client_call(:create_multipart_upload, key: key, content_type: content_type, content_disposition: content_disposition, acl: 'private')
    ...

now I can create public downloads directly without an extra API call just like so:

UPPY_S3_MULTIPART_APP = Uppy::S3Multipart::App.new(bucket: bucket, options: {
  create_multipart_upload: {
    acl: 'public-read'
  }
})

And I couldn't find an abstract/customizable way to decide about Content-disposition as I needed attachment mainly. The challenge with your overrides logic in client_call method seems to be that one, me for example :-), may want to keep the posted filename from r.params["filename"] but change content-disposition to something else than inline. With the overrides logic there is no separation between A and B in Content-disposition: A; filename=B; I know one could add this to every request later like http://bucket.XXX.com/123.wav?response-content-disposition=attachment but main business logic should be decided at the point of upload and could be manipulated later in special cases.

[janko]

Added acl: ... so it becomes "overrideable"

The :acl should already be overridable for #create_multipart_upload (it's even shown in the README), see the #client_call implementation 😉

Note that if you only override :acl, you will still get back presigned URLs with query parameters at the end. That's why I also added the #object_url operation with :public option, so that you can tell it to return public URLs.

options: { object_url: { public: true } }

And since these are two separate things that need to be done, that's why I also added a :public app initialize option that sets up both of these things.

Uppy::S3Multipart::App.new(bucket: bucket, public: true)
# is equivalent to
Uppy::S3Multipart::App.new(bucket: bucket, options: {
  create_multipart_upload: { acl: "public-read" },
  object_url: { public: true },
})

And I couldn't find an abstract/customizable way to decide about Content-disposition as I needed attachment mainly

Yeah, for Content-Disposition you cannot currently separate the disposition from the filename. So to do what you want you'd have to do this:

Uppy::S3Multipart::App.new(bucket: bucket, options: {
  create_multipart_upload: -> (request) {
    filename = request.params["filename"]
    { content_disposition: "attachment; filename=\"#{CGI.escape(filename)}\"" }
  }
})

I'm planning on creating a gem soon that makes it easier to create Content-Disposition headers in correct fomat, so that you'd be able to do:

Uppy::S3Multipart::App.new(bucket: bucket, options: {
  create_multipart_upload: -> (request) {
    filename = request.params["filename"]
    { content_disposition: ContentDisposition.(disposition: "attachment", filename: filename) }
  }
})

Not really shorter, but easier to remember. Maybe we could also add :filename and :disposition parameters that lets you specify only filename or only disposition.

[janko]

Updated the comment that I submitted too early 😄

[mohanklein]

thank you so much! I am back using your official repo :-) How would you implement user auth in an existing rails app in which I mount uppy-s3_multipart like so in routes.rb: mount UPPY_S3_MULTIPART_APP => "/s3". I use JWT for auth in my rails api. Off course I already auth the user in frontend but someone might directly post to http://railsapp.../s3 to get signed urls

[janko]

Well, first of all, any authentication you're doing on the Rails controller level won't be called for requests to the uppy-s3_multipart app (because it's not touching Rails controllers). So you need to do the authentication on the Rack level. You can use constraints to hook into the Rack level:

Rails.application.routes.draw do
  constraints(-> (request) { ... }) do
    mount Shrine.uppy_s3_multipart => "/s3/multipart"
  end
end

See this Shrine wiki page for some examples, they apply equally to uppy-s3_multipart.

Honestly, Rails makes authenticating Rack endpoints really tricky, because of the separation between the router and the controller. With Warden it works in a straightforward way, because with it you have a Rack middleware around your whole app, which allows you to "halt" a request via throw (which is what what Warden's #authenticate! method uses internally IIRC).

Maybe you can search on StackOverflow how to return an early response from inside the constraints block.