static-urls-cache folder not clearing

DanielDarrenJones commented 3 months ago

Bug description

We have a multisite setup where we have static caching enabled, the server went down this morning as it ran out of disk space, I tracked this down to the storage/statamic/static-urls-cache folder which is currently sitting at 41GB:

We clear the static cache when we deploy on Laravel forge, however this only seems to clear the public/static folder:

The sites domains were changed when we sent live, and it does appear that the old staging URL's still exist within the public/store folder, however these are dramatically smaller in size that the storage/statamic/static-urls-cache folder at 831 MB:

How to reproduce

Not sure on the exact reproduction steps here, it may be something to do with out specific setup. However, we created a multisite setup, enabled static caching, and have attempted to clear this with php artisan statamic:static:clear.

Logs

No response

Environment

Environment
Application Name: Pelorus
Laravel Version: 10.45.1
PHP Version: 8.2.12
Composer Version: 2.7.2
Environment: production
Debug Mode: OFF
URL: pelorustravel.com/
Maintenance Mode: OFF

Cache
Config: CACHED
Events: NOT CACHED
Routes: CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: redis
Database: mysql
Logs: stack / single
Mail: smtp
Queue: redis
Session: redis

Statamic
Addons: 9
Antlers: runtime
Sites: 8 (Travel (GB), Travel (US), Aviation (GB), and 5 more)
Stache Watcher: Enabled
Static Caching: full
Version: 4.50.0 PRO

Statamic Addons
aryehraber/statamic-captcha: 1.10.0
doefom/currency-fieldtype: 1.2.3
duncanmcclean/cookie-notice: 8.1.2
gerttimmerman/statamic-zapier: 1.5
jacksleight/statamic-bard-texstyle: 3.1.6
ndx/statamic-bard-color-picker: 1.2.0
ryanmitchell/statamic-translation-manager: 1.4.1
statamic/eloquent-driver: 3.1.2
withcandour/aardvark-seo: 3.0.1

Statamic Eloquent Driver
Asset Containers: eloquent
Assets: eloquent
Blueprints: file
Collection Trees: file
Collections: file
Entries: file
Forms: file
Global Sets: file
Global Variables: file
Navigation Trees: file
Navigations: file
Revisions: file
Taxonomies: file
Terms: file

Installation

Fresh statamic/statamic site via CLI

Antlers Parser

Runtime (default)

Additional details

I have included our config files for sites, static-caching and cache below.

sites.php

```php [ 'travel_gb' => [ 'name' => 'Travel (GB)', 'locale' => 'en_GB', 'url' => env('TRAVEL_URL','https://pelorustravel.com/'), 'lang' => 'en_GB', ], 'travel_us' => [ 'name' => 'Travel (US)', 'locale' => 'en_US', 'url' => env('TRAVEL_US_URL','https://pelorustravel.com/us/'), 'lang' => 'en_US', ], 'aviation_gb' => [ 'name' => 'Aviation (GB)', 'locale' => 'en_GB', 'url' => env('AVIATION_URL','https://pelorusaviation.com/'), 'lang' => 'en_GB', ], 'aviation_us' => [ 'name' => 'Aviation (US)', 'locale' => 'en_US', 'url' => env('AVIATION_US_URL','https://pelorusaviation.com/us/'), 'lang' => 'en_US', ], 'yachting_gb' => [ 'name' => 'Yachting (GB)', 'locale' => 'en_GB', 'url' => env('YACHTING_URL','https://pelorusyachting.com/'), 'lang' => 'en_GB', ], 'yachting_us' => [ 'name' => 'Yachting (US)', 'locale' => 'en_US', 'url' => env('YACHTING_US_URL','https://pelorusyachting.com/us/'), 'lang' => 'en_US', ], 'group_gb' => [ 'name' => 'Group (GB)', 'locale' => 'en_GB', 'url' => env('GROUP_URL','https://pelorusx.com/'), 'lang' => 'en_GB', ], 'group_us' => [ 'name' => 'Group (US)', 'locale' => 'en_US', 'url' => env('GROUP_US_URL','https://pelorusx.com/us/'), 'lang' => 'en_US', ], ], ]; ```

static-caching.php

```php env('STATAMIC_STATIC_CACHING_STRATEGY', null), /* |-------------------------------------------------------------------------- | Caching Strategies |-------------------------------------------------------------------------- | | Here you may define all of the static caching strategies for your | application as well as their drivers. | | Supported drivers: "application", "file" | */ 'strategies' => [ 'half' => [ 'driver' => 'application', 'expiry' => null, ], 'full' => [ 'driver' => 'file', 'path' => [ 'travel_gb' => public_path('static') . '/pelorustravel.com/', 'travel_us' => public_path('static') . '/pelorustravel.com/', 'aviation_gb' => public_path('static') . '/pelorusaviation.com/', 'aviation_us' => public_path('static') . '/pelorusaviation.com/', 'yachting_gb' => public_path('static') . '/pelorusyachting.com/', 'yachting_us' => public_path('static') . '/pelorusyachting.com/', 'group_gb' => public_path('static') . '/pelorusx.co/', 'group_us' => public_path('static') . '/pelorusx.co/', ], 'lock_hold_length' => 0, ], ], /* |-------------------------------------------------------------------------- | Exclusions |-------------------------------------------------------------------------- | | Here you may define a list of URLs to be excluded from static | caching. You may want to exclude URLs containing dynamic | elements like contact forms, or shopping carts. | */ 'exclude' => [ 'class' => null, 'urls' => [ // ], ], /* |-------------------------------------------------------------------------- | Invalidation Rules |-------------------------------------------------------------------------- | | Here you may define the rules that trigger when and how content would be | flushed from the static cache. See the documentation for more details. | If a custom class is not defined, the default invalidator is used. | | https://statamic.dev/static-caching | */ 'invalidation' => [ 'class' => null, 'rules' => [ 'globals' => [ 'branding' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ], 'footer' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ], 'navigation' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ] ], 'navigation' => [ 'footer_column_1' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ], 'footer_column_2' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ], 'footer_column_3' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ], 'main_menu' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ], 'site_selector' => [ 'urls' => [ '/*', $travel.'*', $yachting.'*', $aviation.'*', $group.'*', ] ] ] ], ], /* |-------------------------------------------------------------------------- | Ignoring Query Strings |-------------------------------------------------------------------------- | | Statamic will cache pages of the same URL but with different query | parameters separately. This is useful for pages with pagination. | If you'd like to ignore the query strings, you may do so. | */ 'ignore_query_strings' => false, /* |-------------------------------------------------------------------------- | Replacers |-------------------------------------------------------------------------- | | Here you may define replacers that dynamically replace content within | the response. Each replacer must implement the Replacer interface. | */ 'replacers' => [ \Statamic\StaticCaching\Replacers\CsrfTokenReplacer::class, \Statamic\StaticCaching\Replacers\NoCacheReplacer::class, ], /* |-------------------------------------------------------------------------- | Warm Queue |-------------------------------------------------------------------------- | | Here you may define the name of the queue that requests will be pushed | onto when warming the static cache using the static:warm command. | */ 'warm_queue' => null, ]; ```

cache.php

```php env('CACHE_DRIVER', 'file'), /* |-------------------------------------------------------------------------- | Cache Stores |-------------------------------------------------------------------------- | | Here you may define all of the cache "stores" for your application as | well as their drivers. You may even define multiple stores for the | same cache driver to group types of items stored in your caches. | | Supported drivers: "apc", "array", "database", "file", | "memcached", "redis", "dynamodb", "octane", "null" | */ 'stores' => [ 'apc' => [ 'driver' => 'apc', ], 'array' => [ 'driver' => 'array', 'serialize' => false, ], 'database' => [ 'driver' => 'database', 'table' => 'cache', 'connection' => null, 'lock_connection' => null, ], 'file' => [ 'driver' => 'file', 'path' => storage_path('framework/cache/data'), ], 'memcached' => [ 'driver' => 'memcached', 'persistent_id' => env('MEMCACHED_PERSISTENT_ID'), 'sasl' => [ env('MEMCACHED_USERNAME'), env('MEMCACHED_PASSWORD'), ], 'options' => [ // Memcached::OPT_CONNECT_TIMEOUT => 2000, ], 'servers' => [ [ 'host' => env('MEMCACHED_HOST', '127.0.0.1'), 'port' => env('MEMCACHED_PORT', 11211), 'weight' => 100, ], ], ], 'redis' => [ 'driver' => 'redis', 'connection' => 'cache', 'lock_connection' => 'default', ], 'dynamodb' => [ 'driver' => 'dynamodb', 'key' => env('AWS_ACCESS_KEY_ID'), 'secret' => env('AWS_SECRET_ACCESS_KEY'), 'region' => env('AWS_DEFAULT_REGION', 'us-east-1'), 'table' => env('DYNAMODB_CACHE_TABLE', 'cache'), 'endpoint' => env('DYNAMODB_ENDPOINT'), ], 'octane' => [ 'driver' => 'octane', ], 'static_cache' => [ 'driver' => 'file', 'path' => storage_path('statamic/static-urls-cache'), ], ], /* |-------------------------------------------------------------------------- | Cache Key Prefix |-------------------------------------------------------------------------- | | When utilizing the APC, database, memcached, Redis, or DynamoDB cache | stores there might be other applications using the same cache. For | that reason, you may prefix every cache key to avoid collisions. | */ 'prefix' => env('CACHE_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_cache_'), ]; ```

duncanmcclean commented 3 months ago

What does your config/cache.php file look like?

DanielDarrenJones commented 3 months ago

@duncanmcclean I have edited the original issue above to include this under additional details, thanks!

ryanmitchell commented 3 months ago

You should be able to clear that store manually using artisan cache:clear static_cache

At least as a temporary fix.

DanielDarrenJones commented 3 months ago

Thanks @ryanmitchell I have added this to our deploy script for the time being to ensure this is cleared at the same time as the rest of the static cache, it seems to have done the trick!

statamic / cms