search

edwardspec / mediawiki-aws-s3

Extension:AWS allows MediaWiki to use Amazon S3 (instead of the local directory) to store images.
https://www.mediawiki.org/wiki/Extension:AWS
GNU General Public License v2.0
42 stars 32 forks source link

Could not write file "mwstore://AmazonS3/local-public/... #48

Closed ajmichels closed 2 years ago

ajmichels commented 2 years ago

I recently started getting these errors and I am struggling to figure out why.

I am on PHP 7.4, MediaWiki 1.35, Extension:AWS 0.11.1.

I did recently update my composer dependencies. Per the MediaWiki documentation I removed my composer.lock file and ran composer install

Files are still being ''read'' from the bucket correctly via the CloudFront distribution that I created.

I verified that the AWS credentials I am using are still working correctly. I used the AWS CLI with those same credentials to interact with the S3 bucket on the EC2 instance that the wiki is running on.

I also tried using the latest code from the extension's repo (a08f5d) and then doing the composer update again.

To be clear, this was working just fine a up until mid Dec 2021 (the last time a file was uploaded). The only thing that has changed since then was that I updated the composer dependencies and I enabled the VisualEditor functionality.

Any help would be appreciated.

Here are the versions Composer is using for this extension's dependencies:

  - Locking aws/aws-sdk-php (3.209.17)
  - Locking composer/installers (v1.12.0)

And here is the error I am seeing in the debug logs (some information obfuscated):

<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF (truncated...)
 AccessDenied (client): Access Denied - <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF*******</RequestId><HostId>u6uU*************************************************</HostId></Error>
[error] [de39d4fe79d16409eda7a6cf] /wiki/Special:Upload   ErrorException from line 1104 of /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php: PHP Warning: doCreateInternal: S3Exception: Error executing "PutObject" on "******/Shopify_Photoshop_Actions.atn.zip"; AWS HTTP error: Client error: `PUT *******/Shopify_Photoshop_Actions.atn.zip` resulted in a `403 Forbidden` response:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF (truncated...)
 AccessDenied (client): Access Denied - <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF*******</RequestId><HostId>u6uU*************************************************</HostId></Error>
#0 [internal function]: MWExceptionHandler::handleError()
#1 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(1104): trigger_error()
#2 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(1031): AmazonS3FileBackend->logException()
#3 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(347): AmazonS3FileBackend->runWithExceptionHandling()
#4 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(369): AmazonS3FileBackend->doCreateInternal()
#5 /var/www/html/includes/libs/filebackend/FileBackendStore.php(187): AmazonS3FileBackend->doStoreInternal()
#6 /var/www/html/includes/libs/filebackend/fileop/StoreFileOp.php(74): FileBackendStore->storeInternal()
#7 /var/www/html/includes/libs/filebackend/fileop/FileOp.php(301): StoreFileOp->doAttempt()
#8 /var/www/html/includes/libs/filebackend/FileOpBatch.php(176): FileOp->attempt()
#9 /var/www/html/includes/libs/filebackend/FileOpBatch.php(132): FileOpBatch::runParallelBatches()
#10 /var/www/html/includes/libs/filebackend/FileBackendStore.php(1308): FileOpBatch::attempt()
#11 /var/www/html/includes/libs/filebackend/FileBackend.php(484): FileBackendStore->doOperationsInternal()
#12 /var/www/html/includes/filerepo/FileRepo.php(1336): FileBackend->doOperations()
#13 /var/www/html/includes/filerepo/LocalRepo.php(587): FileRepo->publishBatch()
#14 /var/www/html/includes/filerepo/LocalRepo.php(562): LocalRepo->skipWriteOperationIfSha1()
#15 /var/www/html/includes/filerepo/FileRepo.php(1232): LocalRepo->publishBatch()
#16 /var/www/html/includes/filerepo/LocalRepo.php(587): FileRepo->publish()
#17 /var/www/html/includes/filerepo/LocalRepo.php(558): LocalRepo->skipWriteOperationIfSha1()
#18 /var/www/html/includes/filerepo/file/LocalFile.php(1963): LocalRepo->publish()
#19 /var/www/html/includes/filerepo/file/LocalFile.php(1908): LocalFile->publishTo()
#20 /var/www/html/includes/filerepo/file/LocalFile.php(1415): LocalFile->publish()
#21 /var/www/html/includes/upload/UploadBase.php(944): LocalFile->upload()
#22 /var/www/html/includes/specials/SpecialUpload.php(579): UploadBase->performUpload()
#23 /var/www/html/includes/specials/SpecialUpload.php(214): SpecialUpload->processUpload()
#24 /var/www/html/includes/specialpage/SpecialPage.php(600): SpecialUpload->execute()
#25 /var/www/html/includes/specialpage/SpecialPageFactory.php(635): SpecialPage->run()
#26 /var/www/html/includes/MediaWiki.php(307): MediaWiki\SpecialPage\SpecialPageFactory->executePath()
#27 /var/www/html/includes/MediaWiki.php(940): MediaWiki->performRequest()
#28 /var/www/html/includes/MediaWiki.php(543): MediaWiki->main()
#29 /var/www/html/index.php(53): MediaWiki->run()
#30 /var/www/html/index.php(46): wfIndexMain()
#31 {main}
edwardspec commented 2 years ago

1) Works for me under the same conditions (PHP 7.4, MediaWiki 1.35, Extension:AWS 0.11.1) after updating composer dependencies: the file was uploaded correctly. So we can rule the possibility that some breaking changes (e.g. in aws/aws-sdk-php) could have happened in upstream libraries when updating Composer dependencies.

2) Please double-check everything from https://aws.amazon.com/premiumsupport/knowledge-center/s3-403-forbidden-error/ You mentioned that you were able to upload via AWS CLI, so a big suspect there is "Block Public Access" part (you might have been uploading S3 object as "private", but extension is using "public-read", as the wiki is public, and this behavior hasn't been overridden with $wgFileBackends['s3']['privateWiki'] = true;).

ajmichels commented 2 years ago

Yes, this was the issue. I actually found this right before seeing your response. Specifically this setting:

Screen Shot 2022-02-04 at 4 07 44 PM

Our Wiki is private. I am confused how this was once working before and I didn't previous have this privateWiki setting set. Is this something that changed recently... am very confused by how this issue just started happening without me making any configuration change... Strange.

Everything is working now even with all public access blocked on the bucket.

Thank you for your help.