search

edwardspec / mediawiki-aws-s3

Extension:AWS allows MediaWiki to use Amazon S3 (instead of the local directory) to store images.
https://www.mediawiki.org/wiki/Extension:AWS
GNU General Public License v2.0
42 stars 32 forks source link

Can we exclude certain file types from local-public cache? #53

Closed davidhaslip closed 1 year ago

davidhaslip commented 1 year ago

I use $wgAWSLocalCacheMinSize to speed up rendering of page thumbnails from PDF files, but audio and video files are also captured and stored in the local cache. With a lot of audio or video files it becomes expensive to store them locally. Can a setting be added to allow excluding certain file types and prevent them from being saved locally? Perhaps: $wgAWSLocalCacheExclude ?

edwardspec commented 1 year ago

I have added a new configuration option:

// Files with these extensions won't be cached. 
$wgAWSLocalCacheExcludeFileExtensions = [ 'ogg', 'mp4' ]; // Default: [ 'ogg' ]

Please try it out (git clone -b local-cache-exclude-file-extensions) and let me know if it works for you.

davidhaslip commented 1 year ago

Awesome, thank you for adding this feature. I was getting an error however:

Error from line 136 of /home/public_html/mediawiki/vendor/guzzlehttp/guzzle/src/functions.php: Undefined class constant 'VERSION'

Not sure if this helps: `#0 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/Handler/GuzzleV6/GuzzleHandler.php(40): GuzzleHttp\default_user_agent()

1 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php(87): Aws\Handler\GuzzleV6\GuzzleHandler->__invoke()

2 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/ClientSideMonitoring/AbstractMonitoringMiddleware.php(126): Aws\WrappedHttpHandler->__invoke()

3 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/S3/PermanentRedirectMiddleware.php(43): Aws\ClientSideMonitoring\AbstractMonitoringMiddleware->__invoke()

4 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/S3/PutObjectUrlMiddleware.php(41): Aws\S3\PermanentRedirectMiddleware->__invoke()

5 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/Middleware.php(126): Aws\S3\PutObjectUrlMiddleware->__invoke()

6 /mediawiki/extensions/AWS/vendor/guzzlehttp/promises/src/FulfilledPromise.php(41): Aws\Middleware::Aws{closure}()

7 /mediawiki/extensions/AWS/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\Promise\FulfilledPromise::GuzzleHttp\Promise{closure}()

8 /mediawiki/extensions/AWS/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\Promise\TaskQueue->run()

9 /mediawiki/extensions/AWS/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\Promise\Promise->invokeWaitFn()

10 /mediawiki/extensions/AWS/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\Promise\Promise->waitIfPending()

11 /mediawiki/extensions/AWS/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\Promise\Promise->invokeWaitList()

12 /mediawiki/extensions/AWS/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()

13 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/AwsClientTrait.php(58): GuzzleHttp\Promise\Promise->wait()

14 /mediawiki/extensions/AWS/vendor/aws/aws-sdk-php/src/AwsClientTrait.php(86): Aws\AwsClient->execute()

15 /mediawiki/extensions/AWS/s3/AmazonS3FileBackend.php(603): Aws\AwsClient->__call()

16 /mediawiki/extensions/AWS/s3/AmazonS3FileBackend.php(566): AmazonS3FileBackend->statUncached()

17 /mediawiki/includes/libs/filebackend/FileBackendStore.php(728): AmazonS3FileBackend->doGetFileStat()

18 /mediawiki/includes/libs/filebackend/FileBackendStore.php(648): FileBackendStore->getFileStat()

19 /mediawiki/includes/filerepo/FileRepo.php(1425): FileBackendStore->fileExists()

20 /mediawiki/includes/filerepo/FileRepo.php(1407): FileRepo->fileExistsBatch()

21 /mediawiki/includes/filerepo/file/File.php(1170): FileRepo->fileExists()

22 /mediawiki/includes/page/ImagePage.php(433): File->transform()

23 /mediawiki/includes/page/ImagePage.php(142): ImagePage->openShowImage()

24 /mediawiki/includes/actions/ViewAction.php(80): ImagePage->view()

25 /mediawiki/includes/MediaWiki.php(532): ViewAction->show()

26 /mediawiki/includes/MediaWiki.php(313): MediaWiki->performAction()

27 /mediawiki/includes/MediaWiki.php(945): MediaWiki->performRequest()

28 /mediawiki/includes/MediaWiki.php(548): MediaWiki->main()

29 /mediawiki/index.php(53): MediaWiki->run()

30 /mediawiki/index.php(46): wfIndexMain()

31 {main}`

edwardspec commented 1 year ago

Are you using the installation method with composer.local.json (see README)? The old installation method (with composer install in the directory extension/AWS/) was replaced due to Guzzle-related problems similar to this.

davidhaslip commented 1 year ago

Ok thanks. I followed those instructions and it's working as expected now. I appreciate it a lot!