Unable to read file from location: asset meta on s3

goellner commented 11 months ago

Bug description

Getting hit with a lot of Unable to read file from location: media/1/.meta/xxx.mp4.yaml errors.

When we upload images directly to S3 Statamic does not generate the .meta files, which is to be expected. Would be good if Statamic tries to create the .meta files if they don't exist instead of throwing an error and making the asset browser unusable.

Also php please assets:meta does not work, throws the same error

How to reproduce

Setup S3 asset container upload image directly open asset browser and enter folder where a new asset has been uploaded

Logs

No response

Environment

Environment
Application Name: SWAG SWAG SWAG
Laravel Version: 10.20.0
PHP Version: 8.1.22
Composer Version: 2.5.8
Environment: production
Debug Mode: ENABLED
URL: app.asdf.com
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: redis
Database: mysql
Logs: prod / mezmo, flare
Mail: smtp
Queue: redis
Scout: meilisearch
Session: redis

Statamic
Addons: 3
Antlers: runtime
Stache Watcher: Enabled
Static Caching: Disabled
Version: 4.19.0 PRO

Statamic Addons
aerni/advanced-seo: 2.1.4
aryehraber/statamic-uuid: 2.2.0
statamic/eloquent-driver: 2.6.1

Statamic Eloquent Driver
Asset Containers: file
Assets: eloquent
Blueprints: file
Collection Trees: file
Collections: eloquent
Entries: eloquent
Forms: eloquent
Global Sets: file
Global Variables: file
Navigation Trees: file
Navigations: eloquent
Revisions: eloquent
Taxonomies: eloquent
Terms: eloquent

Installation

Fresh statamic/statamic site via CLI

Antlers Parser

None

Additional details

No response

duncanmcclean commented 11 months ago

This might have the same cause as #7350, which is thrown when using Glide.

FrankPeters commented 9 months ago

We are using an SFTP driver using the official Spatie's package and we are also getting these .meta not found errors quite frequently. I thought it was related to the visibility not being public in order for the folders to be created with the correct rights, but that does not seem to solve the problem.

Example of our config:

      'assets_main' => [
          'driver' => 'sftp',
          'host' => '[...]',
          'username' => '*',
          'privateKey' => '/home/forge/[...]/sftp_ssh_key',
          'root' => '[...]/public/assets/main',
          'visibility' => 'public',
          'throw' => true,
          'url' => '[...]/assets/main',
      ],

Maybe unrelated, but we also notice that there's a huge delay after uploading a file that ends with the file being uploaded but the .meta error being triggered (sometimes on a different file that was already on the file system but misses the .meta file), so the user never gets to see a success message.

FrankPeters commented 9 months ago

It seemed to be caused by the 'throw' => true,. After disabling that, the issue went away.

ryanmitchell commented 8 months ago

@goellner are you still seeing this issue? If so could you provide a full stack trace?

goellner commented 4 months ago

@ryanmitchell sorry, didn't see the replys. I think it was, because I manually uploaded assets to the S3 bucket. Not sure if this still happens, but Statamic didn't try to create the metadata files, it just threw an error

goellner commented 2 months ago

@ryanmitchell just running into this again. I am on 4.58.2 now and every asset I upload to S3 breaks the asset viewer. Have to manually create a corresponding yaml file. The asset is uploaded. I saw that there are a some fixes for meta on the v5 branch, just can't update yet because of some dependencies

goellner commented 2 months ago

if this could be related, i have a custom root folder in filesystems.php

AWS_ACCESS_KEY_ID=xxx
AWS_SECRET_ACCESS_KEY=xxx
AWS_DEFAULT_REGION=eu-central-1
AWS_BUCKET=xxx
AWS_USE_PATH_STYLE_ENDPOINT=false
AWS_URL=https://assets.example.com

'cms' => [
    'driver' => 's3',
    'key' => env('AWS_ACCESS_KEY_ID'),
    'secret' => env('AWS_SECRET_ACCESS_KEY'),
    'region' => env('AWS_DEFAULT_REGION'),
    'bucket' => env('AWS_BUCKET'),
    'url' => env('AWS_URL'),
    'endpoint' => env('AWS_ENDPOINT'),
    'use_path_style_endpoint' => env('AWS_USE_PATH_STYLE_ENDPOINT', false),
    'throw' => true,
    'root' => 'cms',
],

statamic / cms