Can anyone confirm if this gem works in Rails 6?

[desnw]

Can anyone confirm if this gem works in Rails 6?

[mcohen9]

Just upgraded my app to Rails 6 getting error on this one. AccessDenied

Invalid according to Policy: Policy Condition failed: ["starts-with", "$utf8", ""]

And I added utf hidden field then i get another error:

Invalid according to Policy: Policy Condition failed: ["starts-with", "$key", "uploads/"]

Not sure how to fix this.

[desnw]

Hello mcohen9,

I had opened a ticket with Amazon Web Services and they told me the same thing that the issue was in the policy the "starts-with", "$utf-8" line had to be removed. Actually what they told me was much more cryptic. I am still very new to Ruby on Rails and web development.

I was able to fix the UPLOAD itself by going into the gem itself and in the form_helper.rb file I simply removed that line.

BEFORE: def policy_data { expiration: @options[:expiration], conditions: [ ["starts-with", "$utf8", ""], ["starts-with", "$key", @options[:key_starts_with]], ["starts-with", "$x-requested-with", ""], ["content-length-range", 0, @options[:max_file_size]], ["starts-with","$content-type", @options[:content_type_starts_with] ||""], {bucket: @options[:bucket]}, {acl: @options[:acl]}, {success_action_status: "201"} ] + (@options[:conditions] || []) } end

AFTER: def policy_data { expiration: @options[:expiration], conditions: [ ["starts-with", "$key", @options[:key_starts_with]], ["starts-with", "$x-requested-with", ""], ["content-length-range", 0, @options[:max_file_size]], ["starts-with","$content-type", @options[:content_type_starts_with] ||""], {bucket: @options[:bucket]}, {acl: @options[:acl]}, {success_action_status: "201"} ] + (@options[:conditions] || []) } end I simply removed the first line under the policy data ["starts-with", "$utf8", ""] and that fixed the upload issue itself.

However I am having another issue where it is not coming back after the upload takes place. It puts it in the bucket in the uploads folder and at this point there is a second function that is supposed to take place but isn't.
It never starts the second function that moves it to its perm folder and then goes back and deletes the file out of the upload folder.
I am still SO new to this and have been struggling for about 3 weeks on this issue trying everything I can think of. I also tried to pull in Almost all of the newer commits of this gem and tried a couple others that were forked from this gem where there were notes were other people also removed that $UTF-8 line but they did other things as well.
Unfortunately whatever the other things were that they changed are causing a different post error when it tries to upload to S3..

If you do this it will pull in the very latest commit but again this ended up causing another issue and I am just not knowledgeable enough to understand WHY.
gem 's3_direct_upload', '0.1.7', github: 'waynehoover/s3_direct_upload', ref: '6f6decc75fdf89888d7f729fc89f78e90d91cece'

I can CONFIRM with v0.1.7 if you go into the gem itself and go into the form_helper.rb file and you remove that line in the policy itself then it seems to do at least the upload part right. I myself in the app I am working on is then having a secondary issue, maybes yours will not.

What I don't know is even if I do figure out how to fix this gem I don't know then I how make it where I can pull it in from Heroku. I am modifying the gem on my Windows Laptop which is great for local DEV and PROD but I can't pull that onto Heroku that I am aware of. I am assuming I would have to FORK the 0.1.7 version of the gem into my own GitHub and then pull from there or learn how to make a gem. But I haven't got it all working yet on my app. I have a second app I had to upgrade and it is also not working on that APP. I haven't been able to work on the second app much because I am still struggling to get this first App working with this gem.

I don't want to abandon paperclip as I am just too new to change EVERYTHING. So I have been trying to figure out how to get this gem working, OR use just the jquery-file-upload itself and not use this gem. I would be happy at this point if I could find something that works that can handle the direct upload function properly. It's a shame that the development of this gem just stopped as it is still quite valuable IMO. I am trying my best to figure out how to get it to work.

This is what I know for now, as soon as I have something else figured out I will update this post. Scott

[desnw]

Here is what Amazon told me which is what led me to remove that one line:

From the image attached to the case correspondence I was able to extract S3 request ID and using which I performed log dive and observed the reason for 403 access denied was 'Error Reason: Invalid according to Policy: Policy Condition failed: ["starts-with", "$utf8", ""]'. Usually this error occurs when there is something wrong with the upload policy. The policy required for making authenticated requests using HTTP POST is a UTF-8 and base64-encoded document written in JavaScript Object Notation (JSON) that specifies conditions that the request must meet. Depending on how you design your policy document, you can control the access granularity per-upload, per-user, for all uploads, or according to other designs that meet your needs.

You can check the correct policy by referring the following link - https://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html

Please refer the following document to learn more about POST Policy - POST Policy

Additionally I went ahead and searched online and found an article which describes the same error - https://github.com/dwilkie/carrierwave_direct/issues/199

I would request you to go through the above article and perform changes that are advised.

Therefore in order to resolve this issue, you will have to edit the POST policy that is used to upload images using forms to your S3 bucket.

Please refer the following document to learn about S3 request IDs - https://aws.amazon.com/premiumsupport/knowledge-center/s3-request-id-values/

Maybe with this MUCH MORE technical information someone else more skilled than myself can fix this gem and get it working properly? Until then I am still trying to get it working.

Scott

[desnw]

I was able to get this working with Rails 6 as far as the upload portion of it. If you pull the latest version of this gem as a plugin instead by entering in this in your gemfile:

gem 's3_direct_upload', github: 'waynehoover/s3_direct_upload', ref: '6f6decc75fdf89888d7f729fc89f78e90d91cece'

That will get you the latest version that has the policy fixed as well as some other things added into it. This however will require you to add one additional item to your file_field_tag.

data: { url: s3_uploader_url }

So for example my file_field_tag is setup as such: (using HAML) = file_field_tag(:file, id: "before_photo", multiple: true, data: { url: s3_uploader_url })

If you pull the latest version and you use the data: attribute it seems to work properly as far as the upload portion. I am still having an issue with the callback but I don't know if that is to due to this GEM or something else in my own code. I was able to get a couple of sample apps working properly with Rails 6, aws-sdk-v1, paperclip, s3_direct_upload using the latest version with the data: attribute as I demonstrated above.

[desnw]

I believe my issue with this gem is resolved. If I find that the issue with the callback is this gem I will come back and re-open this.

[benjaminwood]

Thank @desnw for documenting the issues you ran into here. For anyone else that's getting 403's from AWS, you may also be able to solve the issue with the rubygems release by adding the utf8 field back into the form manually

= hidden_field_tag 'utf8', '✓'

This hidden field was included in the form by default before this change in Rails 6.0: https://github.com/rails/rails/pull/32125

[kashiftufail]

Thanks @benjaminwood

its working for me. i'm using ruby 3.2.2 & Rails 7.0.8.

my s3_direct_upload at this version (0.1.7)