Statamic Cloudinary asset container not working with folders

[DmitrySidorenkoShim]

Bug description

I understand it is not directly a bug of Statamic itself but maybe someone can help me with this issue

so, in 'Assets' if you have 'Cloudinary' asset container and any folders in Cloudinary like 'auto mapping folder' and other nested folders in it loading of a list takes too much time, and at the end id shows 'The container is empty'

I thought it is an issue of Cloudinary itself but it works perfectly without any folders

• upload, list, preview, destroy, rename, etc.

Cloudinary package is only as flysystem disk driver and it does return a list of all files and directories from Cloudinary I also tried all variants of data in asset_meta trying to make it work

How to reproduce

So, install yoelpc4/laravel-cloudinary (I don't know why, but the official package cloudinary-labs/cloudinary-laravel is not fully working in this case) Fill out .envcredentials:

FILESYSTEM_DISK=cloudinary

CLOUDINARY_API_KEY=
CLOUDINARY_API_SECRET=
CLOUDINARY_CLOUD_NAME=
CLOUDINARY_SECURE=true

Register cloudinary driver configuration in config/filesystems.php at disks section (example in the package readme) In Cloudinary 'Upload Presets' add 'asset folder', for example: 'some/images'

After all this setup is done goto Statamic Assets, create a new asset container 'Cloudinary' which uses 'cloudinary' disk and upload some sample.jpg (on this step Statamic will show an error " Hochladen fehlgeschlagen. Die Datei ist möglicherweise größer als von deinem Server erlaubt. " ["Upload failed. The file size may be larger than your server allows."]) the image is successfully uploaded to cloudinary though check the image is in the folder in Cloudinary something like: https://res.cloudinary.com/.../image/upload/v1.../some/images/sample.jpg and it is in asset_meta in database

but after this there is no chance to see it in Statamic

any help on this would be appreciated 🙏

Logs

No response

Environment

Environment
Application Name: Bergwelten Backend
Laravel Version: 10.48.20
PHP Version: 8.3.9
Composer Version: 2.7.7
Environment: local
Debug Mode: ENABLED
URL: localhost
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: statamic
Database: mysql
Logs: daily
Mail: smtp
Queue: sync
Session: file

Sentry
Enabled: MISSING DSN
Environment: local
Laravel SDK Version: 4.8.0
PHP SDK Version: 4.9.0
Release: NOT SET
Sample Rate Errors: 100%
Sample Rate Performance Monitoring: NOT SET
Sample Rate Profiling: NOT SET
Send Default PII: DISABLED

Statamic
Addons: 3
Antlers: runtime
Sites: 1
Stache Watcher: Enabled
Static Caching: Disabled
Version: 4.58.2 PRO

Statamic Addons
rbmh/cdc-user-area: dev-master
rbmh/global-functions: 2.2.0
statamic/eloquent-driver: 3.4.1

Statamic Eloquent Driver
Asset Containers: eloquent
Assets: eloquent
Blueprints: eloquent
Collection Trees: eloquent
Collections: eloquent
Entries: eloquent
Forms: eloquent
Global Sets: eloquent
Global Variables: eloquent
Navigation Trees: eloquent
Navigations: eloquent
Revisions: eloquent
Taxonomies: eloquent
Terms: eloquent

Installation

Existing Laravel app

Additional details

I don't know if it is important but I have tried this:

    $rawFlysystemDirectoryListing        = \Illuminate\Support\Facades\Storage::disk('cloudinary')
        ->getDriver()->listContents('/', true);
    $files = collect($rawFlysystemDirectoryListing)
        ->keyBy('path')
        ->map(fn ($file) => \App\Services\TestService::normalizeFlysystemAttributes($file))
        ->pipe(fn ($files) => \App\Services\TestService::ensureMissingDirectoriesExist($files))
        ->sortKeys();

like you have it in \Statamic\Assets\AssetContainerContents::all() in $files I have a full list of files and directories from Cloudinary

as I wrote, any help would be greatly appreciated

[duncanmcclean]

This seems to be an issue with the Eloquent Driver's implementation of the asset stuff.

I don't see the "This container is empty" error when using flat-files, but I do as soon as I switch to the Eloquent Driver.

Moving this issue to the eloquent-driver repository.

[ryanmitchell]

What gets populated in your assets_meta table? Did you try running the sync command?

[DmitrySidorenkoShim]

What gets populated in your assets_meta table? Did you try running the sync command?

here is what I have in assets_meta in database:

<id> cloudinary / sample.jpg sample jpg sample.jpg {"data": []} null 2024-08-28 13:30:11 2024-08-28 13:30:11

may sound stupid but sorry I didn't heart about any sync command

[ryanmitchell]

You can read about it here - https://github.com/statamic/eloquent-driver?tab=readme-ov-file#syncing-assets If you are using an existing file system it will populate it with assets already added.

[DmitrySidorenkoShim]

@ryanmitchell thanks for the quick response

I will try this php please eloquent:sync-assets sync command

is it going through all asset containers and synchronizing files for discs they using? and the same for Cloudinary storage disk if I have asset container?

[ryanmitchell]

Yes it adds any pre-existing files to the database, and as the eloquent driver uses the database to build the folder list it will also then show you any folders.

[DmitrySidorenkoShim]

there are looks like some issues with the command I tried it on the dev server, and it is just frozen on this:

for the last 20 mins or so... 🤔 are you using chunking? because we have around 62 thousand of images there

and I tried the command locally (I have there my dev Cloudinary account credentials) it was synced but not correctly

see hightailed entries (duplicates with wrong extensions) it might be because you are doing something like: $ext = Str::afterLast($path, '.'); instead of something like: $ext1 = File::extension(\Illuminate\Support\Facades\Storage::disk('cloudinary')->path($uri)); could it be possible? 😌

[ryanmitchell]

Yes it is chunking where possible - if you are finding issues on your install please feel free to open a PR with the changes required.

[DmitrySidorenkoShim]

@ryanmitchell

excuse me if I was not polite in any of my messages and thank you for your quick response you moved me from dead point 👍

[DmitrySidorenkoShim]

ok, sadly I couldn't make sync command working with Cloudinary... (I also had no much time for this, doing this in my free time) the main issue is because of Cloudinary return a list with paths without extensions

@ryanmitchell can you maybe point to the place (a class and method) where the package gets an extension for Asset, please? 🙏 it can help to get further 😌

[ryanmitchell]

What do you mean an extension for asset? You can bind your own model and class (see the config) or you can bind your own AssetContainerContents - similar to how we do: https://github.com/statamic/eloquent-driver/blob/38371abc8dd58189b221e2efa136ef44a1468d79/src/ServiceProvider.php#L262-L264

[ryanmitchell]

@duncanmcclean did some digging on this and as you said the Flysystem driver doesn't seem to return an extension.

This unfortunately means we can't work with it - it may be worth raising an issue on that repo to see if they would consider including the extension in the same way other Flysystem adaptors do.