search

statamic / cms

The core Laravel CMS Composer package
https://statamic.com
Other
4.06k stars 530 forks source link

Unable to browse S3 bucket with a lot of files #7317

Open carlosdubus opened 1 year ago

carlosdubus commented 1 year ago

Bug description

Seems like this library tries fetch ALL files under a bucket when browsing a S3 asset container. Is there a reason for this?

The whole sites hangs when trying to browse a s3 bucket with a lot of files, needing to restart nginx.

Changing recursive to false I can see the top level of the bucket, but not subfolders.

vendor/statamic/cms/src/Assets/AssetContainerContents.php

$files = $this->filesystem()->listContents('/', false); // settting this to false fixed the issue for the top level, but can't browse subfolders

How to reproduce

Create a S3 asset container with a bucket with a lot of files.

Logs

No response

Environment

Environment
Application Name: Laravel
Laravel Version: 9.46.0
PHP Version: 8.1.13
Composer Version: 2.5.1
Environment: local
Debug Mode: ENABLED
URL: localhost
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: statamic
Database: mysql
Logs: stack / single
Mail: smtp
Queue: sync
Session: file

Filament
Packages: filament, forms, notifications, support, tables
Version: v2.16.59
Views: NOT PUBLISHED

Statamic
Addons: 1
Antlers: regex
Stache Watcher: Enabled
Static Caching: Disabled
Version: 3.3.63 Solo

Statamic Addons
doublethreedigital/runway: 2.6.3

Installation

Existing Laravel app

Antlers Parser

None

Additional details

No response

ryanmitchell commented 11 months ago

Version 3 of Eloquent Driver has a database driven asset query builder to bypass this behaviour, you should see significant performance improvements with a large number of remote storage files.