statamic / ssg

The official Statamic Static Site Generator
229 stars 23 forks source link

getimagesize errors on site generate #91

Open fitzage opened 2 years ago

fitzage commented 2 years ago

I’m not sure where exactly the issue is here because I updated to Statamic 3.3 as well as SSG 1.0 at the same time, since it kind of all depended on each other.

When I attempt to generate my site statically, a few pages generate and then I get a glide error. At first I thought it was social images, but it’s not just social images. As far as I’ve been able to track down, the generator is for some reason trying to run “getimagesize” on versions of images that haven’t yet been generated by glide. I’m not even sure why it’s trying to do that here, and not sure why glide is even generating some of these images. I’m not actively using it in these cases (although I imagine it’s in use in the background for the social images.

I did figure out this much: if I visit the page with the image in question on my localhost, the image that was missing gets generated. I can go into storage/statamic/glide/containers and dig down to that folder and the image doesn’t exist. But as soon as I load the page, it gets generated. So the SSG is expecting these to be generated, but they’re not getting generated unless I go visit the actual page, which creates a pretty big issue for the SSG.

This happens even if I disable multiple workers, so it’s not related to the other bug of occasional issues with multiple workers.

[✘] /blog/2020/01/15/the-year-of-the-mobile-edge (getimagesize(/Users/fitzage/Source/mexv3/storage/statamic/glide/containers/social/yotme-blog-post.jpg/2df2be5e837c23016d3700f17a23f082.jpg): Failed to open stream: No such file or directory)
imacrayon commented 2 years ago

I just ran into this too after the same upgrade. I dunno if it matters but I'm using the S3 driver.

fitzage commented 2 years ago

I’m not using S3 on this site, so I don’t think it makes a difference.

imacrayon commented 2 years ago

The Glide Tag is using the cache path set by the CMS config instead of the SSG config here: https://github.com/statamic/cms/blob/e69c1f434acc4a267832dc20ea7397ea1623bb34/src/Tags/Glide.php#L110.

In order to fix this the SSG plugin needs to be able to override the cache paths set here: https://github.com/statamic/cms/blob/e69c1f434acc4a267832dc20ea7397ea1623bb34/src/Imaging/GlideServer.php#L34

fitzage commented 2 years ago

I’m not even using Glide on the pages in question, which makes it extra weird. I’m guessing it’s related to SEO Pro using Glide.

fitzage commented 2 years ago

@imacrayon I’m not sure this is the whole story, because I have cache disabled in the assets.php file, so I would think it shouldn’t even be caching the glide images, unless the SSG is overriding that.

fitzage commented 2 years ago

OK, I take that back. Caching on/off doesn’t apply because it has to generate them anyway, so they have to be “cached.”

But I think you may be on to something. So my Glide path is set to img in the ssg.php config file, but the error is because it's trying to deal with images in a completely different directory than that.

from ssg.php:

    'glide' => [
        'directory' => 'img',
    ],

But error is showing …/storage/statamic/glide/containers/… as path.

So perhaps since glide isn’t putting images into that path during SSG generate, it can’t find them for the next step unless they’ve already been generated by the CMS on the dynamic side.

imacrayon commented 2 years ago

Yep, I'm pretty sure that's what's going on. If I add this after line 101 in Glide.php. SSG starts working again for me.

config([
    'statamic.assets.image_manipulation.cache' => true,
    'statamic.assets.image_manipulation.cache_path' => $this->config['destination'].'/'.$directory,
]);
fitzage commented 2 years ago

@imacrayen Would you be able to do a pull request with that fix? Maybe we could actually get this fixed faster. :-)

jasonvarga commented 2 years ago

Hey I believe this is related to https://github.com/statamic/cms/issues/5662 which I'm investigating now.

imacrayon commented 2 years ago

https://github.com/statamic/cms/pull/5724 doesn't quite fix this b/c the cachePath method is causing the issue.

The SSG command overrides Glide's internal cache path here: https://github.com/statamic/ssg/blob/1acc7b897852e63c0c00ef02a8349f2259210b48/src/Generator.php#L109 but GlideServer.php still returns the cache path set in config/statamic/assets.php.

fitzage commented 2 years ago

So this error did not occur on a new site built from the basic starter kit. Granted, it’s a simple site that I haven’t done anything fancy on, but the issue is not universal.

On another site, instead of getting getimagesize errors, I just get File not found at path errors that seem to be related to the same problem as when I opened this ticket. But they’re referring to image paths inside the generated output folder, and when I get the error and go look at the path it mentions, the image actually exists.

For example:

[✘] /press-release/pico-announces-appointment-of-ann-neidenbach-to-board-of-directors (File not found at path: containers/miscellaneous/social-ann-neidenbach.png/ea9e31750d5ebe2020d81e9aeacc0e47.png)

But here is the image:

image

So that would seem to support @imacrayon’s point that the issue is with the cache path not being respected.

fitzage commented 2 years ago

So I noticed the fresh site I set up uses Laravel 9, so I went through all the shenanigans of updated to that (which also means disabling the lovely Review module for now). I also generally made sure other composer modules were updated to match the new starter file.

Now I get a slightly more robust error message:

[✘] /blog/why-quality-of-market-data-matters-more-in-volatile-markets (Unable to copy file from source://containers/blog/6-why-quality-of-market-data-matters-more-in-volatile-markets/social.png/872432f2ee6d0c639f4ea1a35857b477.png to cache://containers/blog/6-why-quality-of-market-data-matters-more-in-volatile-markets/social.png/872432f2ee6d0c639f4ea1a35857b477.png)

I’m not sure what this source containers folder is, but I don’t think it exists.

conradfuhrman commented 2 years ago

@fitzage, by chance did you ever figure out the last error message? I'm running into this too and the cause seems to be related to the seo-pro addon in our case. We're using github actions to deploy to S3/CloudFront and this is the only hangup we're currently encountering.

fitzage commented 2 years ago

@conradfuhrman Nope, I’ve had to back burner the upgrades. I did just build a site without SEO Pro that works fine, so it makes sense that this is the issue.

conradfuhrman commented 2 years ago

Thanks for the quick response! I'll make sure to make a ticket for SEO Pro and reference this. Any other issues with Glide you experienced and SSG? We also just backed out SEO Pro and had a perfect deploy so we'll just have to implement our own SEO fields (or try a different addon).

fitzage commented 2 years ago

@conradfuhrman I don’t think I’m actually using glide anywhere else on the sites in question, so I’m not sure about other issues.

ncla commented 1 year ago

Running into this issue too:

[✘] /project/xxx (Unable to copy file from source://containers/assets/3-1628160583.jpg/db6bdb5908d65358fe28a6668cbff84a.jpg to cache://containers/assets/3-1628160583.jpg/db6bdb5908d65358fe28a6668cbff84a.jpg)

The file does exist when I check through terminal

➜  project git:(master) ✗ find . -name "db6bdb5908d65358fe28a6668cbff84a.jpg"
./storage/app/static/img/containers/assets/3-1628160583.jpg/db6bdb5908d65358fe28a6668cbff84a.jpg

ls -lh output

-rw-r--r-- 1 ncla ncla 329K jūl 26 14:14 db6bdb5908d65358fe28a6668cbff84a.jpg

Once this error occurs during ssg:generate and if you visit the page through browser that failed generating, you get helpful stacktrace:

[2022-07-26 11:33:01] local.ERROR: Unable to copy file from source://containers/assets/Snacks.png/57b5e228804ab42dfbb4dc4357b47326.png to cache://containers/assets/Snacks.png/57b5e228804ab42dfbb4dc4357b47326.png {"userId":"33c6b322-954c-4288-8c81-270233928178","exception":"[object] (League\\Flysystem\\UnableToCopyFile(code: 0): Unable to copy file from source://containers/assets/Snacks.png/57b5e228804ab42dfbb4dc4357b47326.png to cache://containers/assets/Snacks.png/57b5e228804ab42dfbb4dc4357b47326.png at /var/www/html/vendor/league/flysystem/src/UnableToCopyFile.php:37)
[stacktrace]
#0 /var/www/html/vendor/league/flysystem/src/MountManager.php(333): League\\Flysystem\\UnableToCopyFile::fromLocationTo()
#1 /var/www/html/vendor/league/flysystem/src/MountManager.php(243): League\\Flysystem\\MountManager->copyAcrossFilesystem()
#2 /var/www/html/vendor/statamic/cms/src/Imaging/Attributes.php(20): League\\Flysystem\\MountManager->copy()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(337): Statamic\\Imaging\\Attributes->from()
#4 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(112): Illuminate\\Support\\Facades\\Facade::__callStatic()
#5 [internal function]: Statamic\\Tags\\Glide->Statamic\\Tags\\{closure}()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(560): array_map()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(719): Illuminate\\Support\\Arr::map()
#8 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(121): Illuminate\\Support\\Collection->map()
#9 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(37): Statamic\\Tags\\Glide->generate()
#10 [internal function]: Statamic\\Tags\\Glide->__call()
#11 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(161): call_user_func()
#12 [internal function]: Statamic\\View\\Antlers\\Engine::renderTag()
#13 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(586): call_user_func_array()
#14 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(347): Statamic\\View\\Antlers\\Parser->parseCallbackTags()
#15 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(290): Statamic\\View\\Antlers\\Parser->parseLoopInstance()
#16 [internal function]: Statamic\\View\\Antlers\\Parser->Statamic\\View\\Antlers\\{closure}()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(560): array_map()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(719): Illuminate\\Support\\Arr::map()
#19 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(291): Illuminate\\Support\\Collection->map()
#20 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(225): Statamic\\View\\Antlers\\Parser->parseLoopVariables()
#21 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(177): Statamic\\View\\Antlers\\Parser->parseVariables()
#22 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(134): Statamic\\View\\Antlers\\Parser->parse()
#23 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(97): Statamic\\View\\Antlers\\Parser->parseView()
#24 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(139): Statamic\\View\\Antlers\\Engine->get()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(122): Illuminate\\View\\View->getContents()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(91): Illuminate\\View\\View->renderContents()
#27 /var/www/html/vendor/statamic/cms/src/View/View.php(98): Illuminate\\View\\View->render()
#28 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(154): Statamic\\View\\View->render()
#29 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(45): Statamic\\Http\\Responses\\DataResponse->contents()
#30 /var/www/html/vendor/statamic/cms/src/Structures/Page.php(384): Statamic\\Http\\Responses\\DataResponse->toResponse()
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(815): Statamic\\Structures\\Page->toResponse()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(802): Illuminate\\Routing\\Router::toResponse()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(725): Illuminate\\Routing\\Router->prepareResponse()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}()
#35 /var/www/html/vendor/withcandour/aardvark-seo/src/Http/Middleware/RedirectsMiddleware.php(16): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): WithCandour\\AardvarkSeo\\Http\\Middleware\\RedirectsMiddleware->handle()
#37 /var/www/html/vendor/statamic/cms/src/StaticCaching/Middleware/Cache.php(47): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\StaticCaching\\Middleware\\Cache->handle()
#39 /var/www/html/vendor/statamic/cms/src/Http/Middleware/AuthGuard.php(14): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\AuthGuard->handle()
#41 /var/www/html/vendor/statamic/cms/src/Http/Middleware/AddViewPaths.php(28): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\AddViewPaths->handle()
#43 /var/www/html/vendor/statamic/cms/src/Http/Middleware/Localize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#44 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\Localize->handle()
#45 /var/www/html/vendor/statamic/cms/src/Http/Middleware/HandleToken.php(13): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#46 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\HandleToken->handle()
#47 /var/www/html/vendor/statamic/cms/src/Http/Middleware/StacheLock.php(29): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#48 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\StacheLock->handle()
#49 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#50 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle()
#51 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(78): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#52 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken->handle()
#53 /var/www/html/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#54 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\View\\Middleware\\ShareErrorsFromSession->handle()
#55 /var/www/html/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#56 /var/www/html/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(64): Illuminate\\Session\\Middleware\\StartSession->handleStatefulRequest()
#57 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Session\\Middleware\\StartSession->handle()
#58 /var/www/html/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#59 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse->handle()
#60 /var/www/html/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#61 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Cookie\\Middleware\\EncryptCookies->handle()
#62 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#63 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(726): Illuminate\\Pipeline\\Pipeline->then()
#64 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(703): Illuminate\\Routing\\Router->runRouteWithinStack()
#65 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(667): Illuminate\\Routing\\Router->runRoute()
#66 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(656): Illuminate\\Routing\\Router->dispatchToRoute()
#67 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(167): Illuminate\\Routing\\Router->dispatch()
#68 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()
#69 /var/www/html/vendor/statamic/cms/src/Http/Middleware/DisableFloc.php(18): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#70 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\DisableFloc->handle()
#71 /var/www/html/vendor/statamic/cms/src/Http/Middleware/CheckMultisite.php(14): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#72 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\CheckMultisite->handle()
#73 /var/www/html/vendor/statamic/cms/src/Http/Middleware/CheckComposerJsonScripts.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#74 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\CheckComposerJsonScripts->handle()
#75 /var/www/html/vendor/statamic/cms/src/Http/Middleware/PoweredByHeader.php(19): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#76 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Statamic\\Http\\Middleware\\PoweredByHeader->handle()
#77 /var/www/html/vendor/barryvdh/laravel-debugbar/src/Middleware/InjectDebugbar.php(66): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#78 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Barryvdh\\Debugbar\\Middleware\\InjectDebugbar->handle()
#79 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#80 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#81 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()
#82 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#83 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#84 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()
#85 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#86 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()
#87 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#88 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()
#89 /var/www/html/vendor/fruitcake/laravel-cors/src/HandleCors.php(38): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#90 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Fruitcake\\Cors\\HandleCors->handle()
#91 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#92 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\\Http\\Middleware\\TrustProxies->handle()
#93 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#94 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(142): Illuminate\\Pipeline\\Pipeline->then()
#95 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(111): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter()
#96 /var/www/html/public/index.php(55): Illuminate\\Foundation\\Http\\Kernel->handle()
#97 /var/www/html/server.php(21): require_once('...')
#98 {main}

This exception will only go away if I do php please glide:clear && php artisan cache:clear, but then I stumble upon this exception elsewhere when I decide to ssg:generate again. It seems that if Glide cache retrieval/generation fails once, then it is stuck forever?

php please support:details output

Environment
Application Name: xxx
Laravel Version: 9.21.6
PHP Version: 8.0.20
Composer Version: 2.3.7
Environment: local
Debug Mode: ENABLED
URL: xxx.com.test
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: statamic
Database: mysql
Logs: stack / single
Mail: smtp
Queue: sync
Session: file

Statamic
Addons: 5
Antlers: regex
Version: 3.3.23 PRO

Statamic Addons
goldnead/statamic-collapse-fieldtype: 1.0.4
spatie/statamic-responsive-images: 2.13.0
statamic/ssg: 1.1.0
withcandour/aardvark-seo: 2.0.26
xndbogdan/statamic-bard-text-color: 2.3

As you can see, I am not using SEO Pro. I do not think this is issue with an addon I have installed, just a general Glide & SSG issue.

Happy to send the git repository to core team.

tao commented 1 year ago

I've started getting the same problem on Github Actions. I was going crazy trying to figure it out but it's the same message:

(Unable to copy file from source://containers/assets/journals/camelot-journal.png/xxx.png to
 cache://containers/assets/journals/camelot-journal.png/xxx.png)
ncla commented 1 year ago

I have dug up another, earlier exception stack trace from the one I posted earlier. I don't know why exceptions in console output are so bare bones, we are shooting ourselves in the foot by not having exception stack traces that can be easily accessed. Here's the exception:

[previous exception] [object] (League\\Flysystem\\UnableToRetrieveMetadata(code: 0): Unable to retrieve the visibility for file at location: containers/assets/1-1628166187.jpg/58464d75481f5a8a53230a0a4043722e.jpg.  at /var/www/html/vendor/league/flysystem/src/UnableToRetrieveMetadata.php:49)
[stacktrace]
#0 /var/www/html/vendor/league/flysystem/src/UnableToRetrieveMetadata.php(34): League\\Flysystem\\UnableToRetrieveMetadata::create()
#1 /var/www/html/vendor/league/flysystem/src/Local/LocalFilesystemAdapter.php(388): League\\Flysystem\\UnableToRetrieveMetadata::visibility()
#2 /var/www/html/vendor/league/flysystem/src/Filesystem.php(147): League\\Flysystem\\Local\\LocalFilesystemAdapter->visibility()
#3 /var/www/html/vendor/league/flysystem/src/MountManager.php(329): League\\Flysystem\\Filesystem->visibility()
#4 /var/www/html/vendor/league/flysystem/src/MountManager.php(243): League\\Flysystem\\MountManager->copyAcrossFilesystem()
#5 /var/www/html/vendor/statamic/cms/src/Imaging/Attributes.php(20): League\\Flysystem\\MountManager->copy()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(337): Statamic\\Imaging\\Attributes->from()
#7 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(112): Illuminate\\Support\\Facades\\Facade::__callStatic()
#8 [internal function]: Statamic\\Tags\\Glide->Statamic\\Tags\\{closure}()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(560): array_map()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(719): Illuminate\\Support\\Arr::map()
#11 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(121): Illuminate\\Support\\Collection->map()
#12 /var/www/html/vendor/statamic/cms/src/Tags/Glide.php(37): Statamic\\Tags\\Glide->generate()
#13 [internal function]: Statamic\\Tags\\Glide->__call()
#14 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(161): call_user_func()
#15 [internal function]: Statamic\\View\\Antlers\\Engine::renderTag()
#16 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(586): call_user_func_array()
#17 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(347): Statamic\\View\\Antlers\\Parser->parseCallbackTags()
#18 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(290): Statamic\\View\\Antlers\\Parser->parseLoopInstance()
#19 [internal function]: Statamic\\View\\Antlers\\Parser->Statamic\\View\\Antlers\\{closure}()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(560): array_map()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(719): Illuminate\\Support\\Arr::map()
#22 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(291): Illuminate\\Support\\Collection->map()
#23 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(225): Statamic\\View\\Antlers\\Parser->parseLoopVariables()
#24 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(177): Statamic\\View\\Antlers\\Parser->parseVariables()
#25 /var/www/html/vendor/statamic/cms/src/View/Antlers/Parser.php(134): Statamic\\View\\Antlers\\Parser->parse()
#26 /var/www/html/vendor/statamic/cms/src/View/Antlers/Engine.php(97): Statamic\\View\\Antlers\\Parser->parseView()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(139): Statamic\\View\\Antlers\\Engine->get()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(122): Illuminate\\View\\View->getContents()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/View/View.php(91): Illuminate\\View\\View->renderContents()
#30 /var/www/html/vendor/statamic/cms/src/View/View.php(98): Illuminate\\View\\View->render()
#31 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(154): Statamic\\View\\View->render()
#32 /var/www/html/vendor/statamic/cms/src/Http/Responses/DataResponse.php(45): Statamic\\Http\\Responses\\DataResponse->contents()
#33 /var/www/html/vendor/statamic/cms/src/Entries/Entry.php(423): Statamic\\Http\\Responses\\DataResponse->toResponse()
#34 /var/www/html/vendor/statamic/ssg/src/Page.php(40): Statamic\\Entries\\Entry->toResponse()
#35 /var/www/html/vendor/statamic/ssg/src/Page.php(31): Statamic\\StaticSite\\Page->write()
#36 /var/www/html/vendor/statamic/ssg/src/Generator.php(271): Statamic\\StaticSite\\Page->generate()
#37 /var/www/html/vendor/spatie/fork/src/Task.php(92): Statamic\\StaticSite\\Generator->Statamic\\StaticSite\\{closure}()
#38 /var/www/html/vendor/spatie/fork/src/Fork.php(154): Spatie\\Fork\\Task->execute()
#39 /var/www/html/vendor/spatie/fork/src/Fork.php(128): Spatie\\Fork\\Fork->executeInChildTask()
#40 /var/www/html/vendor/spatie/fork/src/Fork.php(103): Spatie\\Fork\\Fork->forkForTask()
#41 /var/www/html/vendor/spatie/fork/src/Fork.php(182): Spatie\\Fork\\Fork->runTask()
#42 /var/www/html/vendor/spatie/fork/src/Fork.php(76): Spatie\\Fork\\Fork->startRunning()
#43 /var/www/html/vendor/spatie/fork/src/Fork.php(69): Spatie\\Fork\\Fork->waitFor()
#44 /var/www/html/vendor/statamic/ssg/src/ConcurrentTasks.php(18): Spatie\\Fork\\Fork->run()
#45 /var/www/html/vendor/statamic/ssg/src/Generator.php(190): Statamic\\StaticSite\\ConcurrentTasks->run()
#46 /var/www/html/vendor/statamic/ssg/src/Generator.php(90): Statamic\\StaticSite\\Generator->createContentFiles()
#47 /var/www/html/vendor/statamic/ssg/src/Commands/StaticSiteGenerate.php(62): Statamic\\StaticSite\\Generator->generate()
#48 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Statamic\\StaticSite\\Commands\\StaticSiteGenerate->handle()
#49 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#50 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#51 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#52 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#53 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(139): Illuminate\\Container\\Container->call()
#54 /var/www/html/vendor/symfony/console/Command/Command.php(291): Illuminate\\Console\\Command->execute()
#55 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(124): Symfony\\Component\\Console\\Command\\Command->run()
#56 /var/www/html/vendor/symfony/console/Application.php(998): Illuminate\\Console\\Command->run()
#57 /var/www/html/vendor/symfony/console/Application.php(299): Symfony\\Component\\Console\\Application->doRunCommand()
#58 /var/www/html/vendor/symfony/console/Application.php(171): Symfony\\Component\\Console\\Application->doRun()
#59 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Application.php(102): Symfony\\Component\\Console\\Application->run()
#60 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\\Console\\Application->run()
#61 /var/www/html/please(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#62 {main}
"}
ncla commented 1 year ago

I was able to narrow this issue down to Glide tag pairs. I opened a new separate issue. https://github.com/statamic/ssg/issues/110

Templating like this

{{ glide:assets_field }}
        {{ height }}
        {{ width }}
{{ /glide:assets_field }}

caused this error for me. Maybe you folks have tag pairs in your projects also? The failing assets were always consistently the same for me, not random, so it was not difficult to narrow this issue down.

fitzage commented 1 year ago

@ncla I haven’t directly used Glide the sites in question. It’s SEO Pro for me.

ncla commented 1 year ago

You may not be using it directly, but the addon does contain Glide tags, and I think this code for example from SEO Pro

https://github.com/statamic/seo-pro/blob/4565e0bed9f8871ad09637c35c841ef2a4e1fd76/resources/views/meta.antlers.html#L55-L59

Will trigger the same error.

jsbls commented 1 year ago

FWIW, if anyone needs a quick workaround, I set up a Gist with the code for Generator.php.diff. Needs Composer Patches (per @ncla's suggestion).

Just create a folder named patches/ at root level and add the contents to a Generator.php.diff file.

It's probably not the cleanest/best way about it but it worked for me.

https://gist.github.com/jsbls/d0858e1e254652c1deb71b063904c778

imacrayon commented 1 year ago

https://github.com/statamic/ssg/pull/104 resolved this for me. Add the new config to config/statamic/ssg.php:

'glide' => [
    'override' => false,
],
cutefluffypuppy commented 1 year ago

I found that SSG and Glide (to Netlify) needed two configuration file adjustments. (Should also fix #123.)

config/statamic/assets.php - statically generate images:

    'image_manipulation' => [
        'cache' => true,
        'cache_path' => public_path('img'),
    ],

config/statamic/ssg.php - disable override and copy the images:

    'copy' => [
        public_path('img') => 'img',
    ],
...
    'glide' => [
        'directory' => 'img',
        'override' => false,
    ],

But note that setting the cache & cache_path in assets.php may break your local editing - hence the reason why Glide.php should really inject the cache & cache_path setting at the time SSG is built (see https://github.com/statamic/ssg/issues/91#issuecomment-1083455400 above).

martyf commented 1 year ago

Full disclosure, I don't personally use SSG day-to-day so haven't felt confident enough to do a PR as I'm unsure of the bigger implications, however rather than changing vendor files above, I developed a little workaround for a client so that they can maintain local edit-ability of images (i.e. not cached): blog post

Basically it is moving this logic to the AppServiceProvider and re-configuring at runtime to do all of these changes when ssg:generate is run, so now the code is part of the app (rather than editing a vendor file or constantly tweaking config files).

fitzage commented 1 year ago

@martyf None of the other workarounds have solved my problem, but I’ll give your fix a shot.

fitzage commented 1 year ago

@martyf Yay! I can finally upgrade to 3.3, and hopefully beyond.

fitzage commented 12 months ago

@martyf Yep, this is needed all the way into 4.0, but it also works just fine in 4.0. Woohoo!

martyf commented 12 months ago

Happy to hear it has helped - it was for Statamic 4 that it was originally written (as the client raised the issue during the 3-to-4 process), but glad it has worked on 3 too.

Not sure if this is too opinionated for a PR as it does it in a way that works, but may not work for everyone too. Again, not a SSG user myself (except for very hands-off for this one client) so unsure of the bigger picture ramifications.

jsbls commented 12 months ago

@martyf Works like a charm! Thanks!