Amazon S3 problem in MODX 3

[rsdesign007]

Summary: When uploading a file to an Amazon S3 bucket using the Amazon S3 Media Source in MODX 3 the file is not made public.

Files upload fine, but they don't have Public Read access when uploaded through MODX 3.

It is then necessary to login to Amazon S3 Management Console, go to the Amazon S3 Bucket and manually change each file's ACL to: Everyone (public access), Object or Object ACL: Read.

This problem does not occur in MODX 2x.

Step to reproduce: Setup Amazon S3 bucket. Setup MODX 3 Amazon S3 Mediasource. Upload file to Amazon.

Observed behavior: After uploading the file is not publicly available on our website. It is necessary to login to Amazon S3 and manually add "Object or Object ACL: Read" to make each file public.

This is the Amazon error message:

` AccessDenied

Access Denied YBA14M63CWYT2K9F 9aedOZHCRnEqK5mRCnoJith3PxtwGORYXkJ7H7o9SLgouC2n/mx6LyTHOZAJoeNZynXJ5dSKxEU=

`

Expected behavior: The file, video or image should be publicly viewable on the front end of our website.

Environment: MODX 3.0.3, PHP 8.1, MODX Cloud, Nginx.

[JoshuaLuckers]

@matdave I remember you had experience with the S3 integration and I thought your insights could help in resolving this.

[matdave]

This is generally an issue with MIME types. I've seen this most often happen with SVG files and PDFs. @rsdesign007 is this happening to ALL files, or just specific file types?

[rsdesign007]

Hi Mat,

Thanks for looking into this.

So far, we've only used Amazon S3 in MODX 3 for larger MP4 videos. I just uploaded a JPG image through MODX 3 to test if that works.

Unfortunately uploading a jpg image to Amazon S3 through MODX 3 has the same problem.

This jpg is NOT publicly viewable after uploading through MODX 3:

https://powers-static.s3.amazonaws.com/pw-s_main-blog_suzanne_2x3_2x.jpg

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>YE97XGMX638YRKSM</RequestId>
<HostId>baqINEMP0nS+wWD07p9uQooXO0OdoEn6qn3cQL2EfchNY5OI9LyhPxFKHwkoRbYEaKot+ud3OEE=</HostId>
</Error>

I then uploaded a second jpg through MODX 3, then logged into Amazon S3 and manually changed the permissions to Everyone (public access) Read and then it worked.

This jpg IS publicly viewable after I logged into Amazon and manually changing the permissions:

https://powers-static.s3.amazonaws.com/pw-s_main-parallax_9309-valley-hill_2x.jpg

PERMISSIONS ADDED IN AMAZON: Object owner (your AWS account)

Canonical ID: 719932e99a691eb530ef7e4da21b08497b024191c53f36d989d0b7759947ea5c

Everyone (public access)

Group: http://acs.amazonaws.com/groups/global/AllUsers

Object Object ACL Read Read

I hope this helps.

Thanks,

Chris

[JoshuaLuckers]

@rsdesign007 you mentioned it works fine in 2.x. Just to clarify, are you using the same bucket in 2.x?

[rsdesign007]

Hi @JoshuaLuckers,

Yes the two example images uploaded above are an S3 Bucket that is shared by two different MODX Clouds. Both websites are for the same customer, so we wanted to allow them to share their images between the 2 websites.

One cloud is using: MODX Version: 3.0.3-pl PHP Version: 8.1

The other cloud is using: MODX Version: 2.8.5-pl PHP Version: 8.1 It is also using the AWS S3 Media Source Extra

We have been using the MODX 2.8.5 website for several years with the Amazon S3 Bucket that's now shared with the MODX 3.0.3 website, and it has always worked correctly with MODX 2.

We just setup the new MODX Version: 3.0.3-pl website a couple weeks ago.

I just went to the MODX Version: 2.8.5-pl website to confirm that it is still working correctly, since I have not uploaded any files to Amazon S3 from there in a while.

I noticed a new problem today on the MODX 2.8.5 website that seems related.

SUMMARY: We upgraded the MODX 2.8.5-pl Cloud to PHP 8.1 a few months ago. The default MODX 2 Amazon S3 Media Source did not work with PHP 8.1 so we installed the AWS S3 Media Source Extra on this Cloud which solved the problem.

I noticed that today that the the image I uploaded yesterday through the MODX 3 website (above) without READ permissions was causing the MODX 2 Cloud to not be able to see the Amazon S3 Bucket?

After logging into Amazon and changing the permissions to READ on this image (uploaded through MODX 3), then the MODX 2 website using the same AWS S3 Media Source Extra could once again see and upload files to the Amazon S3 Bucket and works correctly again.

Changed Permissions to READ: https://powers-static.s3.amazonaws.com/pw-s_main-blog_suzanne_2x3_2x.jpg

The MODX 2.8.5 website now works correctly again with the same Amazon S3 Bucket.

The MODX 3.0.3 website still does NOT work correctly with Amazon S3

I'm not sure if this is at all related since MODX 2 and MODX 3 use an entirely different system for S3 Storage, but I thought I would include this information if it helps.

Thanks,

Chris

[krava77]

Hi @rsdesign007, @JoshuaLuckers I suppose the issue in the core/src/Revolution/Sources/modS3MediaSource.php I see the $visibility_files variable is false by default, but no chances to change it from the manager. I've added the following code to the 37 row of the file, and now I can manage it from the Manager-Media Sources-visibility would be great if modx dev team add this fix to the next release. $this->visibility_files = $this->xpdo->getOption('visibility', $properties, Visibility::PRIVATE);

[rsdesign007]

Hi @krava77,

Last month I added your fix at line 37 to a MODX 3.0.3 website, and your fix worked perfect! So I added it to all of our MODX 3 websites.

Thank you!

Update to your solution...

Today I upgraded a website to MODX 3.0.4. The fix at line 37 was not in the modS3MediaSource.php file after upgrading, but I tried uploading a video to Amazon S3 anyway to test it.

The new video uploaded would not work. I again received the below error from Amazon.

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>GJ45C376F9BBX46H</RequestId>
<HostId>J4p2e5e1oCOx8h2ZEScax6wKztuE4CeEmPsYlmB0Ox+znlJQI2Csj9WnC4JoWh5byBmppx6/nnM=</HostId>
</Error>

Next, I added your solution back at line 37 in MODX 3.0.4, uploaded another video to S3 through MODX and this time the video was publicly viewable and worked correctly.

It looks like your suggestion was not added into MODX 3.0.4. The problem still exists.

The visibility public setting in the Amazon S3 Media Source does not work in MODX 3.0.4 without adding your fix.

Chris

[rthrash]

Hi @rsdesign007 this issue is still open and now slated for the next patch (or minor) release.