Active Storage does not handle HEAD requests properly with AWS S3 backend

soirju commented 5 years ago

Steps to reproduce

A javascript HEAD XHR request to a resource on the web server is redirected to a HEAD request on the S3 server. The resulting redirect URL is calculated using the url method in rails/activestorage/lib/active_storage/service/s3_service.rb based on the :get method only, including the pre-signed key. This key allows only GET requests in S3, no HEAD request.

Expected behavior

Redirect works with 200 - OK

Actual behavior

Redirected HEAD request gives 403 - forbidden

System configuration

Rails version: 5.2.2

Ruby version: 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]

Example:

Initial header request

eedchme@DE-00000657:~$ curl --head "https://www.meyer-kohlscheid.com/weissensing/rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBHQT09IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--93210b5d911fd57dd7a9fd5436e503b979f674aa/test.mp3" HTTP/1.1 302 Found Date: Thu, 18 Apr 2019 15:02:50 GMT Server: Apache/2.4.29 (Ubuntu) Cache-Control: max-age=300, private Referrer-Policy: strict-origin-when-cross-origin X-Permitted-Cross-Domain-Policies: none X-XSS-Protection: 1; mode=block X-Request-Id: 949557ce-9553-40ad-882c-f513932087b4 X-Download-Options: noopen X-Runtime: 0.011524 X-Frame-Options: SAMEORIGIN X-Content-Type-Options: nosniff X-Powered-By: Phusion Passenger 5.0.30 Transfer-Encoding: chunked Location: https://s3.eu-central-1.amazonaws.com/weissensing.meyer-kohlscheid.com/P681vxjhfjbWfRiFtrrc4w9c?response-content-disposition=attachment%3B%20filename%3D%22test.mp3%22%3B%20filename%2A%3DUTF-8%27%27test.mp3&response-content-type=audio%2Fmpeg&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI7CJAEZLGQS2X6OQ%2F20190418%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20190418T150250Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=15303b6e4fa6bb3bc6467a181efedab3ba90b789d6c632f3f9036296381725ce Status: 302 Found Content-Type: text/html; charset=utf-8

Follow re-direct (HEAD request) 403 - Forbidden

eedchme@DE-00000657:~$ curl --head "https://s3.eu-central-1.amazonaws.com/weissensing.meyer-kohlscheid.com/P681vxjhfjbWfRiFtrrc4w9c?response-content-disposition=attachment%3B%20filename%3D%22test.mp3%22%3B%20filename%2A%3DUTF-8%27%27test.mp3&response-content-type=audio%2Fmpeg&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI7CJAEZLGQS2X6OQ%2F20190418%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20190418T150545Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=3c40852c09b0d4dd5968703b8fa6b1296c28468cd5779bf43377c5c5e32e2bad" HTTP/1.1 403 Forbidden x-amz-request-id: E8878A449D6D70B5 x-amz-id-2: LUAJcAU4e6Mskjrrb5R6Nn03H6GuGyVurnMfJAcQGByEjdiZNmcXlVgcF6JxE5tmitAnkLNEwAs= Content-Type: application/xml Transfer-Encoding: chunked Date: Thu, 18 Apr 2019 15:06:02 GMT Server: AmazonS3

Follow same re-direct (GET request) - Working!

eedchme@DE-00000657:~$ curl "https://s3.eu-central-1.amazonaws.com/weissensing.meyer-kohlscheid.com/P681vxjhfjbWfRiFtrrc4w9c?response-content-disposition=attachment%3B%20filename%3D%22test.mp3%22%3B%20filename%2A%3DUTF-8%27%27test.mp3&response-content-type=audio%2Fmpeg&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI7CJAEZLGQS2X6OQ%2F20190418%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20190418T150545Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=3c40852c09b0d4dd5968703b8fa6b1296c28468cd5779bf43377c5c5e32e2bad" --output test.mp3 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 9296k 100 9296k 0 0 8729k 0 0:00:01 0:00:01 --:--:-- 8729k

rails-bot[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not been commented on for at least three months. The resources of the Rails team are limited, and so we are asking for your help. If you can still reproduce this error on the 5-2-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open. Thank you for all your contributions.

vkalach commented 2 years ago

this one is still actual Rails 6.0.3.5

ledowong commented 4 months ago

Having the same issue in rails 7. Ideal if there is an option for us to override the hard-code :get in here would be nice.

My application scenario:

frontend ask for a export file.
the file not ready yet, background job start generate the export file, frontend retry the same endpoint every 3 seconds, until the file is ready.
the export file is ready and uploaded to s3 in background job
endpoint return the signed s3 url to frontend
frontend JS trigger browser download the signed s3 url
Due to s3 cache delay (it is reasonable), s3 return 404 for the signed url.
browser saved the file to file system, however the file content is actually s3 404 XML...

What I want to do, after step 5, frontend use JS to check if the file is ready, using HEAD request to s3.

ledowong commented 4 months ago

a monkey patch in rails-api/config/initializers/active_storage_s3_monkey_patch.rb

require 'active_storage/service/s3_service'

module ActiveStorage
  class Service::S3Service < Service
    private

    def private_url(key, expires_in:, filename:, disposition:, content_type:, **client_opts)
      method = client_opts.delete(:method)
      if method == :head
        object_for(key).presigned_url method, expires_in: expires_in.to_i, **client_opts
      else
        object_for(key).presigned_url :get, expires_in: expires_in.to_i,
          response_content_disposition: content_disposition_with(type: disposition, filename: filename),
          response_content_type: content_type, **client_opts
      end
    end
  end
end

rails / rails