Error logs: Unable to set visibility for file

stuartcusackie commented 1 year ago

Bug description

I'm noticing random but frequent errors appearing in my server logs. They seem to happen randomly. I don't think they are affecting the visitor experience.

Example 1 - From asset field Unable to set visibility for file containers/images/4.jpg/7a74d6248bee8f56088387bf4e26032e.webp

$image = Statamic::tag('glide:generate')->src($image)->params(['width' => 300, 'height' => 300, 'fm' => 'webp', 'fit' => 'crop_focal'])[0];

Example 2 - From local path Unable to set visibility for file paths/images/table-no-space-middle1.jpg/989e5c2d05020abc66c4d34890951412.jpg

@foreach (Statamic::tag('glide:generate')->src('/images/table-no-space-middle1.jpg')->width(1200)->height(627)->fit('crop_focal')->fm('jpg') as $image)

How to reproduce

Not sure. Happens randomly.

I am using a DigitalOcean droplet and a DigitalOcean space.
The errors seem to happen in the droplet's filesystem and not on the space.
The errors occur in views where I am using the glide:generate tag. I am also using Spatie's responsive images but no errors seem to be emanating from there.
Save Cached Images = false
Clearing the glide cache reduces the errors temporarily. Seems to be a problem with older caches.

Logs

[2023-01-01 12:03:09] production.ERROR: Unable to set visibility for file paths/images/table-no-space-middle1.jpg/989e5c2d05020abc66c4d34890951412.jpg. {"view":{"view":"/home/forge/www.website.com/resources/views/vendor/statamic-seo/seo.blade.php","data":[]},"exception":"[object] (Spatie\\LaravelIgnition\\Exceptions\\ViewException(code: 0): Unable to set visibility for file paths/images/table-no-space-middle1.jpg/989e5c2d05020abc66c4d34890951412.jpg. at /home/forge/www.website.com/vendor/league/flysystem/src/UnableToSetVisibility.php:33)
[stacktrace]
#0 /home/forge/www.website.com/vendor/league/flysystem/src/Local/LocalFilesystemAdapter.php(448): League\\Flysystem\\UnableToSetVisibility::atLocation()
#1 /home/forge/www.website.com/vendor/league/flysystem/src/Local/LocalFilesystemAdapter.php(346): League\\Flysystem\\Local\\LocalFilesystemAdapter->setPermissions()
#2 /home/forge/www.website.com/vendor/league/flysystem/src/Local/LocalFilesystemAdapter.php(124): League\\Flysystem\\Local\\LocalFilesystemAdapter->setVisibility()
#3 /home/forge/www.website.com/vendor/league/flysystem/src/Local/LocalFilesystemAdapter.php(103): League\\Flysystem\\Local\\LocalFilesystemAdapter->writeToFile()
#4 /home/forge/www.website.com/vendor/league/flysystem/src/Filesystem.php(69): League\\Flysystem\\Local\\LocalFilesystemAdapter->writeStream()
#5 /home/forge/www.website.com/vendor/league/flysystem/src/MountManager.php(369): League\\Flysystem\\Filesystem->writeStream()
#6 /home/forge/www.website.com/vendor/league/flysystem/src/MountManager.php(245): League\\Flysystem\\MountManager->copyAcrossFilesystem()
#7 /home/forge/www.website.com/vendor/statamic/cms/src/Imaging/Attributes.php(20): League\\Flysystem\\MountManager->copy()
#8 /home/forge/www.website.com/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(338): Statamic\\Imaging\\Attributes->from()
#9 /home/forge/www.website.com/vendor/statamic/cms/src/Tags/Glide.php(138): Illuminate\\Support\\Facades\\Facade::__callStatic()
#10 [internal function]: Statamic\\Tags\\Glide->Statamic\\Tags\\{closure}()
#11 /home/forge/www.website.com/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(560): array_map()
#12 /home/forge/www.website.com/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(739): Illuminate\\Support\\Arr::map()
#13 /home/forge/www.website.com/vendor/statamic/cms/src/Tags/Glide.php(147): Illuminate\\Support\\Collection->map()
#14 /home/forge/www.website.com/vendor/statamic/cms/src/Tags/FluentTag.php(115): Statamic\\Tags\\Glide->generate()
#15 /home/forge/www.website.com/vendor/statamic/cms/src/Tags/FluentTag.php(136): Statamic\\Tags\\FluentTag->fetch()
#16 /home/forge/www.website.com/resources/views/vendor/statamic-seo/seo.blade.php(17): Statamic\\Tags\\FluentTag->getIterator()
#17 /home/forge/www.website.com/vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php(109): require('...')
#18 /home/forge/www.website.com/vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php(110): Illuminate\\Filesystem\\Filesystem::Illuminate\\Filesystem\\{closure}()
... MORE

Environment

Environment
Laravel Version: 9.44.0
PHP Version: 8.1.7
Composer Version: 2.5.1
Environment: production
Debug Mode: OFF
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: statamic
Database: mysql
Logs: stack / single, slack
Mail: smtp
Queue: sync
Session: file

Statamic
Addons: 6
Antlers: regex
Stache Watcher: Disabled
Static Caching: half
Version: 3.3.61 PRO

Statamic Addons
aerni/livewire-forms: 4.0.0
jonassiewertsen/statamic-livewire: 2.9.0
pecotamic/sitemap: 1.4.0
spatie/statamic-responsive-images: 2.14.4
stuartcusackie/statamic-cache-requester: 1.0.2
stuartcusackie/statamic-seo: 0.3.2

Installation

Fresh statamic/statamic site via CLI

Antlers Parser

None

Additional details: file references

I logged into the server to investigate the troublesome file: /storage/statamic/glide/containers/images/4.jpg/7a74d6248bee8f56088387bf4e26032e.webp

I ran a search for references to this file in /storage/framework/cache/glide: grep -ri "7a74d6248bee8f56088387bf4e26032e" /home/forge/www.mywebsite.com/storage/framework/cache/glide

This found two references in folders /glide/7f/06/ and /glide/65/ff/. Is this normal? Should there be two references to the same file?

edalzell commented 1 year ago

@stuartcusackie is this site a multi-site?

stuartcusackie commented 1 year ago

@edalzell Yes indeed. This is happening on a multi-site. It doesn't seem to happen on single sites - still checking my error logs...

imacrayon commented 1 year ago

I'm using DO Spaces to store my Glide manipulations and I see this error when using SSG and too many workers. If I use a single worker, my site builds fine, but the more workers I add the more likely I am to encounter a "Unable to set visibility for file" error. The images that tend to cause this error are ones commonly found on most pages in my website. My hunch is that there's some kind of read/write conflict happening when multiple workers try to simultaneously manipulate the same image. I dunno if this same cause could relate to multi-sites in some way?

duncanmcclean commented 1 year ago

We get this same error too - we're not using multi-site.

BrandonOldenhof commented 1 year ago

We're getting this in multisites too

j3ll3yfi5h commented 1 year ago

Just happened randomly on a "normal" singlesite

edalzell commented 1 year ago

FYI folks you can stop this from appearing in your logs by add this to your app/Exceptions/Handler.php:

j3ll3yfi5h commented 1 year ago

In my case it was a visible error code, not just in the logs...

BrandonOldenhof commented 1 year ago

In my case it was a visible error code, not just in the logs...

This happened for me and my team as well, but only if the environment is NOT in production. When the environment is in production, everything is logged but the visitors don't get the error page.

It feels a bit like the error is thrown somewhere in the internals but doesn't actually stop execution.

stuartcusackie commented 1 year ago

Just to update on this. One of my traffic-heavy sites is experiencing CPU overloads and server crashes, and those seem to be happening at the exact same time that these errors appear in the logs.

It might be possible that this error causes a spike in CPU usage, or maybe this error is the result of an overstressed CPU. I wonder if anyone else is seeing this kind of behaviour...

I have added some additional details above after performing some investigation.

BrandonOldenhof commented 1 year ago

I was able to come up with a couple of workarounds for this issue because of the way we use our templates.

Workaround 1: validation rules on asset fields.

Use the dimensions validation rule for assets field and specifiy the exact dimensions that you require for the images. This works well for images that don't need resizing when the browser changes size, such as social share images. This workaround means you can use $asset->url() or $asset->absoluteUrl() directly instead of depending on Glide.

Workaround 2: generate srcsets on (image) asset upload.

You need to build a couple of things to get this approach to work:

You need to create a custom helper class that generates the srcset for you.
You need an event listener that listens for the AssetUploaded and AssetSaved events, generates a srcset for your image and updates the asset.
You need a command that iterates over your existing assets and adds the srcset attribute to each asset that needs it.

Then all you have to do is replace Glide with attribute calls on the asset. I'm using $asset->srcset myself.

I can probably create some gists if people need it.

☝️ I used both workarounds to remove Glide from our frontend completely; the only place we're using Glide is when an asset is uploaded or saved.

bennettblack commented 1 year ago

Experiencing this too with the eloquent driver and Statamic::tag('glide:generate') in Blade. This does not fail silently for me - throws a 500 in both Production and Local environments.

@foreach(Statamic::tag('glide:generate')->src($post->assetObject)->fit('crop_focal') as $image)
<img src="{{ $image['url'] }}" alt="{{ $image['alt'] ?? '' }}" class="object-cover h-full w-full rounded ">
@endforeach

Laravel Version: 10.13.1 PHP Version: 8.2.6 Statamic Version: 3.5.0 PRO Environment: local

heidkaemper commented 1 year ago

Same here. It's especially noticeable when running the static:warm command in production. That produces random errors 500 with UnableToDeleteFile and UnableToSetVisibility showing up in monitoring.

PHP 8.1 Statamic 4.11.0 Laravel 10.15.0 Multi-site Assets cache = false

sjclark commented 1 year ago

Also having this, on a single (non-multi) site, can't narrow it down to a schedule, seems to happen sporadically a few times a week;

[2023-08-24 20:56:22] production.ERROR: Unable to set visibility for file containers/assets/social-share-images/Social sharing - 1200 x 628.jpg/2e6867acc9a37569fcab8b1d0f53cd14.jpg. {"exception":"[object] (League\\Flysystem\\UnableToSetVisibility(code: 0): Unable to set visibility for file containers/assets/social-share-images/Social sharing - 1200 x 628.jpg/2e6867acc9a37569fcab8b1d0f53cd14.jpg. at /home/ploi/site.com.au/vendor/league/flysystem/src/UnableToSetVisibility.php:33)
[stacktrace]
#0 /home/ploi/site.com.au/vendor/league/flysystem-local/LocalFilesystemAdapter.php(460): League\\Flysystem\\UnableToSetVisibility::atLocation()
#1 /home/ploi/site.com.au/vendor/league/flysystem-local/LocalFilesystemAdapter.php(358): League\\Flysystem\\Local\\LocalFilesystemAdapter->setPermissions()
#2 /home/ploi/site.com.au/vendor/league/flysystem-local/LocalFilesystemAdapter.php(128): League\\Flysystem\\Local\\LocalFilesystemAdapter->setVisibility()
#3 /home/ploi/site.com.au/vendor/league/flysystem-local/LocalFilesystemAdapter.php(107): League\\Flysystem\\Local\\LocalFilesystemAdapter->writeToFile()

grantholle commented 11 months ago

This is happening to me (500 using an external storage vendor like in #7193) and can't figure out how to fix it. It only started after doing a glide:clear. I've regenerated assets, cleared all cache from everything. Anyone know how to get it working again

statamic / cms