markevans / dragonfly-s3_data_store

S3 data store for the Dragonfly ruby gem
MIT License
62 stars 58 forks source link

Mime type not detected on existing files #32

Closed cbillen closed 2 years ago

cbillen commented 2 years ago

Recently we migrated a few TB of files over to Wasabi S3 from a NAS. We then enabled the s3 data store with dragonfly. All the files are found and mapped, but, every single one of them comes back with an application/octet-stream mime type, causing browsers not to render and download the files

For instance, I have the following image in one of my model :image_uid => "2021/10/07/6d13jlg995_Five_Start_Big_Center_Star.png", :image_name => "Five Start Big Center Star.png", :image_size => 435695 If I try to get the mime-type: p.image.mime_type => "application/octet-stream" Trying to get info through s3cmd on the file, i can see the mime type is reported `s3cmd info 's3://heatwave-assets/secure_assets/production/2021/10/07/6d13jlg995_Five_Start_Big_Center_Star.png'

s3://heatwave-assets/secure_assets/production/2021/10/07/6d13jlg995_Five_Start_Big_Center_Star.png (object): File size: 435695 Last mod: Thu, 07 Oct 2021 23:45:57 GMT MIME type: image/png Storage: STANDARD MD5 sum: d83f65fcb65b56418be9c0e32703fe17 SSE: AES256 Policy: none CORS: ACL: cbillen: FULL_CONTROL `

With --debug I get the header: 'content-type': 'image/png'

But Dragonfly doesn't see this and returns application/octet-stream

In order to fix this i found out i have to add this header x-amz-meta-json':'{"name":"Five Start Big Center Star.png"}'

Then image.mime_type will return the right type image/png

I could also re-upload to fix the header model.image = model.image.to_file(model.image_name)

With millions of files, doing the above one by one is a nightmare. But there are other indicator in the content type header, in the image file name in the model, which could all have been used to determine the mime_type, so why weren't they?

Is there an alternative to going in these files one by one by telling dragonfly to leverage a database attribute or the content type returned by s3?

Thank you

cbillen commented 2 years ago

I created a pull request to support the native content type header https://github.com/markevans/dragonfly-s3_data_store/pull/33 Thank you