search

statamic / ssg

The official Statamic Static Site Generator
229 stars 23 forks source link

Generation fails using {{ glide:asset_field }} tag pair, produces "Unable to copy file from source:// to cache://" error #110

Open ncla opened 1 year ago

ncla commented 1 year ago

Reproducible repository (branch: ssg-glide-cache-bug): https://github.com/ncla/statamic-bugs/tree/ssg-glide-cache-bug

To reproduce, simply have an asset field, and have such templating:

{{ glide:assets_field }}
        {{ height }}
        {{ width }}
{{ /glide:assets_field }}

Then run commands in such order:

php please ssg:clear
php please glide:clear
php please ssg:generate

Which will produce the following output:

Static site destination directory cleared.
[✔] Glide path cache cleared.
[✔] Found 0 images.
[✔] Deleted empty directories.
Your Glide image cache is now so very, very empty.
You may be able to speed up site generation significantly by installing spatie/fork and using multiple workers (requires PHP 8+).
[✔] Gathered content to be generated
[✔] Generated 2 content files
[✘] / (Unable to copy file from source://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png to cache://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png)
[✔] /var/www/html/public/css copied to /var/www/html/storage/app/static/css
[✔] /var/www/html/public/js copied to /var/www/html/storage/app/static/js

Static site generated into /var/www/html/storage/app/static
[!] 1/2 pages not generated

If I remove try .. catch in Generate class over here https://github.com/statamic/ssg/blob/d5664b7b9aca4352d06ea4c5e2f27722a7d6e06c/src/Generator.php#L280-L291 I can retrieve helpful exceptions as otherwise they are not logged or outputted anywhere otherwise.

From quick glance (looking at the exceptions posted lower) it seems the culprit might be the Glide cache recent changes that were implemented, as the failing code in Glide.php tag is here https://github.com/statamic/cms/blob/1ef7efc0de680cdc82655a20ac54cae11c579fe7/src/Tags/Glide.php#L112

Attributes::from(GlideManager::cacheDisk()->getDriver(), $path);

Using normal glide tag without pair like this..

{{ glide:assets_field fit="max" }}

..does not trigger that part of the code, and does not fail the SSG generation.

Stacktraces, three exceptions:

[2022-08-12 17:38:19] local.ERROR: Unable to copy file from source://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png to cache://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png {"exception":"[object] (Statamic\\StaticSite\\NotGeneratedException(code: 0): Unable to copy file from source://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png to cache://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png at /var/www/html/vendor/statamic/ssg/src/Page.php:38)
[stacktrace]
#0 /var/www/html/vendor/statamic/ssg/src/Generator.php(281): Statamic\\StaticSite\\Page->generate()
#1 /var/www/html/vendor/statamic/ssg/src/ConsecutiveTasks.php(12): Statamic\\StaticSite\\Generator->Statamic\\StaticSite\\{closure}()
#2 /var/www/html/vendor/statamic/ssg/src/Generator.php(193): Statamic\\StaticSite\\ConsecutiveTasks->run()
#3 /var/www/html/vendor/statamic/ssg/src/Generator.php(92): Statamic\\StaticSite\\Generator->createContentFiles()
#4 /var/www/html/vendor/statamic/ssg/src/Commands/StaticSiteGenerate.php(62): Statamic\\StaticSite\\Generator->generate()
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Statamic\\StaticSite\\Commands\\StaticSiteGenerate->handle()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#8 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(139): Illuminate\\Container\\Container->call()
#11 /var/www/html/vendor/symfony/console/Command/Command.php(308): Illuminate\\Console\\Command->execute()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(124): Symfony\\Component\\Console\\Command\\Command->run()
#13 /var/www/html/vendor/symfony/console/Application.php(998): Illuminate\\Console\\Command->run()
#14 /var/www/html/vendor/symfony/console/Application.php(299): Symfony\\Component\\Console\\Application->doRunCommand()
#15 /var/www/html/vendor/symfony/console/Application.php(171): Symfony\\Component\\Console\\Application->doRun()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Application.php(102): Symfony\\Component\\Console\\Application->run()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\\Console\\Application->run()
#18 /var/www/html/please(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#19 {main}

[previous exception] [object] (League\\Flysystem\\UnableToCopyFile(code: 0): Unable to copy file from source://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png to cache://containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png at /var/www/html/vendor/league/flysystem/src/UnableToCopyFile.php:37)
[stacktrace]
#0 /var/www/html/vendor/league/flysystem/src/MountManager.php(333): League\\Flysystem\\UnableToCopyFile::fromLocationTo()
#1 /var/www/html/vendor/league/flysystem/src/MountManager.php(243): League\\Flysystem\\MountManager->copyAcrossFilesystem()
#2 /var/www/html/vendor/statamic/cms/src/Imaging/Attributes.php(20): League\\Flysystem\\MountManager->copy()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(337): Statamic\\Imaging\\Attributes->from()
#4 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(112): Illuminate\\Support\\Facades\\Facade::__callStatic()
#5 [internal function]: Statamic\\Tags\\Glide->Statamic\\Tags\\{closure}()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(560): array_map()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(719): Illuminate\\Support\\Arr::map()
#8 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(121): Illuminate\\Support\\Collection->map()
#9 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(37): Statamic\\Tags\\Glide->generate()
#10 [internal function]: Statamic\\Tags\\Glide->__call()
#11 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(161): call_user_func()
#12 [internal function]: Statamic\\View\\Antlers\\Engine::renderTag()
#13 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(586): call_user_func_array()
#14 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(179): Statamic\\View\\Antlers\\Parser->parseCallbackTags()
#15 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(134): Statamic\\View\\Antlers\\Parser->parse()
#16 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(97): Statamic\\View\\Antlers\\Parser->parseView()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(139): Statamic\\View\\Antlers\\Engine->get()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(122): Illuminate\\View\\View->getContents()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(91): Illuminate\\View\\View->renderContents()
#20 /var/www/html/vendor/statamic/cms/src/View/View.php(98): Illuminate\\View\\View->render()
#21 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(154): Statamic\\View\\View->render()
#22 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(45): Statamic\\Http\\Responses\\DataResponse->contents()
#23 /var/www/html/vendor/statamic/cms/src/Entries/Entry.php(423): Statamic\\Http\\Responses\\DataResponse->toResponse()
#24 /var/www/html/vendor/statamic/ssg/src/Page.php(45): Statamic\\Entries\\Entry->toResponse()
#25 /var/www/html/vendor/statamic/ssg/src/Page.php(36): Statamic\\StaticSite\\Page->write()
#26 /var/www/html/vendor/statamic/ssg/src/Generator.php(281): Statamic\\StaticSite\\Page->generate()
#27 /var/www/html/vendor/statamic/ssg/src/ConsecutiveTasks.php(12): Statamic\\StaticSite\\Generator->Statamic\\StaticSite\\{closure}()
#28 /var/www/html/vendor/statamic/ssg/src/Generator.php(193): Statamic\\StaticSite\\ConsecutiveTasks->run()
#29 /var/www/html/vendor/statamic/ssg/src/Generator.php(92): Statamic\\StaticSite\\Generator->createContentFiles()
#30 /var/www/html/vendor/statamic/ssg/src/Commands/StaticSiteGenerate.php(62): Statamic\\StaticSite\\Generator->generate()
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Statamic\\StaticSite\\Commands\\StaticSiteGenerate->handle()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(139): Illuminate\\Container\\Container->call()
#37 /var/www/html/vendor/symfony/console/Command/Command.php(308): Illuminate\\Console\\Command->execute()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(124): Symfony\\Component\\Console\\Command\\Command->run()
#39 /var/www/html/vendor/symfony/console/Application.php(998): Illuminate\\Console\\Command->run()
#40 /var/www/html/vendor/symfony/console/Application.php(299): Symfony\\Component\\Console\\Application->doRunCommand()
#41 /var/www/html/vendor/symfony/console/Application.php(171): Symfony\\Component\\Console\\Application->doRun()
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Application.php(102): Symfony\\Component\\Console\\Application->run()
#43 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\\Console\\Application->run()
#44 /var/www/html/please(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#45 {main}

[previous exception] [object] (League\\Flysystem\\UnableToRetrieveMetadata(code: 0): Unable to retrieve the visibility for file at location: containers/assets/uuu.png/8aeb6030920f4be1c711736371e4b7b7.png.  at /var/www/html/vendor/league/flysystem/src/UnableToRetrieveMetadata.php:49)
[stacktrace]
#0 /var/www/html/vendor/league/flysystem/src/UnableToRetrieveMetadata.php(34): League\\Flysystem\\UnableToRetrieveMetadata::create()
#1 /var/www/html/vendor/league/flysystem/src/Local/LocalFilesystemAdapter.php(385): League\\Flysystem\\UnableToRetrieveMetadata::visibility()
#2 /var/www/html/vendor/league/flysystem/src/Filesystem.php(147): League\\Flysystem\\Local\\LocalFilesystemAdapter->visibility()
#3 /var/www/html/vendor/league/flysystem/src/MountManager.php(329): League\\Flysystem\\Filesystem->visibility()
#4 /var/www/html/vendor/league/flysystem/src/MountManager.php(243): League\\Flysystem\\MountManager->copyAcrossFilesystem()
#5 /var/www/html/vendor/statamic/cms/src/Imaging/Attributes.php(20): League\\Flysystem\\MountManager->copy()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(337): Statamic\\Imaging\\Attributes->from()
#7 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(112): Illuminate\\Support\\Facades\\Facade::__callStatic()
#8 [internal function]: Statamic\\Tags\\Glide->Statamic\\Tags\\{closure}()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(560): array_map()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(719): Illuminate\\Support\\Arr::map()
#11 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(121): Illuminate\\Support\\Collection->map()
#12 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(37): Statamic\\Tags\\Glide->generate()
#13 [internal function]: Statamic\\Tags\\Glide->__call()
#14 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(161): call_user_func()
#15 [internal function]: Statamic\\View\\Antlers\\Engine::renderTag()
#16 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(586): call_user_func_array()
#17 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(179): Statamic\\View\\Antlers\\Parser->parseCallbackTags()
#18 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(134): Statamic\\View\\Antlers\\Parser->parse()
#19 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(97): Statamic\\View\\Antlers\\Parser->parseView()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(139): Statamic\\View\\Antlers\\Engine->get()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(122): Illuminate\\View\\View->getContents()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(91): Illuminate\\View\\View->renderContents()
#23 /var/www/html/vendor/statamic/cms/src/View/View.php(98): Illuminate\\View\\View->render()
#24 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(154): Statamic\\View\\View->render()
#25 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(45): Statamic\\Http\\Responses\\DataResponse->contents()
#26 /var/www/html/vendor/statamic/cms/src/Entries/Entry.php(423): Statamic\\Http\\Responses\\DataResponse->toResponse()
#27 /var/www/html/vendor/statamic/ssg/src/Page.php(45): Statamic\\Entries\\Entry->toResponse()
#28 /var/www/html/vendor/statamic/ssg/src/Page.php(36): Statamic\\StaticSite\\Page->write()
#29 /var/www/html/vendor/statamic/ssg/src/Generator.php(281): Statamic\\StaticSite\\Page->generate()
#30 /var/www/html/vendor/statamic/ssg/src/ConsecutiveTasks.php(12): Statamic\\StaticSite\\Generator->Statamic\\StaticSite\\{closure}()
#31 /var/www/html/vendor/statamic/ssg/src/Generator.php(193): Statamic\\StaticSite\\ConsecutiveTasks->run()
#32 /var/www/html/vendor/statamic/ssg/src/Generator.php(92): Statamic\\StaticSite\\Generator->createContentFiles()
#33 /var/www/html/vendor/statamic/ssg/src/Commands/StaticSiteGenerate.php(62): Statamic\\StaticSite\\Generator->generate()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Statamic\\StaticSite\\Commands\\StaticSiteGenerate->handle()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(139): Illuminate\\Container\\Container->call()
#40 /var/www/html/vendor/symfony/console/Command/Command.php(308): Illuminate\\Console\\Command->execute()
#41 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(124): Symfony\\Component\\Console\\Command\\Command->run()
#42 /var/www/html/vendor/symfony/console/Application.php(998): Illuminate\\Console\\Command->run()
#43 /var/www/html/vendor/symfony/console/Application.php(299): Symfony\\Component\\Console\\Application->doRunCommand()
#44 /var/www/html/vendor/symfony/console/Application.php(171): Symfony\\Component\\Console\\Application->doRun()
#45 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Application.php(102): Symfony\\Component\\Console\\Application->run()
#46 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\\Console\\Application->run()
#47 /var/www/html/please(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#48 {main}
"}

Support details:

Environment
Application Name: Statamic
Laravel Version: 9.22.1
PHP Version: 8.1.7
Composer Version: 2.3.8
Environment: local
Debug Mode: ENABLED
URL: responsive-alt-bug.test
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: NOT CACHED

Drivers
Broadcasting: log
Cache: statamic
Database: mysql
Logs: stack / single
Mail: smtp
Queue: sync
Session: file

Statamic
Addons: 1
Antlers: regex
Version: 3.3.28 Solo

Statamic Addons
statamic/ssg: 1.2.0

This is separate, organized, continuation issue of https://github.com/statamic/ssg/issues/91 as it was getting a bit messy there.

brad-sf commented 1 year ago

Same issue for me on the following setup...

Environment
Application Name: Evergrace
Laravel Version: 9.26.1
PHP Version: 8.0.22
Composer Version: 2.3.5
Environment: local
Debug Mode: ENABLED
URL: evergrace.test
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Cache: statamic
Database: mysql
Logs: stack / single
Mail: smtp
Queue: sync
Session: file

Statamic
Addons: 7
Antlers: runtime
Version: 3.3.31 PRO

Statamic Addons
4rn0/statamic-v3-image-optimizer: 1.0.15
aryehraber/statamic-logbook: 2.1.0
statamic/collaboration: 0.4.0
statamic/ssg: 1.2.0
swiftmade/statamic-clear-assets: 1.1.0
webographen/statamic-admin-log: 1.0.6
withcandour/aardvark-seo: 2.0.28
brad-sf commented 1 year ago

This workaround allowed me to build and copied all my images across :)

https://github.com/statamic/ssg/issues/91#issuecomment-1083455400

Be sure to add it after the $directory variable (ie line 113 for me)

schwartzmj commented 1 year ago

Just wanted to give this a little bump and mention I've also been having this issue. I've been unable to SSG sites and have been hosting them with caching set to full instead for now.

ncla commented 1 year ago

This also affects undocumented glide:data_url tag as well, and to some extent Responsive Images addon as it uses the same logic as this tag to retrieve asset image contents, which is used to generate blurry placeholder. The error is a bit slightly different but it is same source of problem. https://github.com/spatie/statamic-responsive-images/issues/178

I can confirm that workaround posted in #91, adding this code Generator.php after line with $directory.. variable lets ssg:generate run without errors.

config([
    'statamic.assets.image_manipulation.cache' => true,
    'statamic.assets.image_manipulation.cache_path' => $this->config['destination'].'/'.$directory,
]);
RiaanZA commented 1 year ago

I've managed to get this working by setting glide.override to false in the ssg.php config file. Not sure how long that setting has been there, but I think it was added specifically to solve this issue.

'glide' => [
        'directory' => 'img',
        'override' => false,
    ],

Then also remember to add your image directory to the "copy" array

public_path('img') => 'img',