Array to string conversion during eloquent sync

[GioChocolateBro]

I just installed the 3.0 update. I have a sizeable filesystem with lots of folders in digitalocean spaces.

After installation the eloquent:sync-assets command throws an exception when it reaches the end of processing the root folder and trying to move on to nested folders.

local.ERROR: Array to string conversion {"exception":"[object] (ErrorException(code: 0): Array to string conversion at /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Stringable.php:33)
[stacktrace]
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php(254): Illuminate\\Foundation\\Bootstrap\\HandleExceptions->handleError(2, 'Array to string...', '/var/www/html/v...', 33)
#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Stringable.php(33): Illuminate\\Foundation\\Bootstrap\\HandleExceptions->Illuminate\\Foundation\\Bootstrap\\{closure}(2, 'Array to string...', '/var/www/html/v...', 33)
#2 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Str.php(68): Illuminate\\Support\\Stringable->__construct(Array)
#3 /var/www/html/vendor/statamic/cms/src/Support/Str.php(314): Illuminate\\Support\\Str::of(Array)
#4 /var/www/html/vendor/statamic/eloquent-driver/src/Assets/AssetContainerContents.php(75): Statamic\\Support\\Str::__callStatic('of', Array)
#5 [internal function]: Statamic\\Eloquent\\Assets\\AssetContainerContents->Statamic\\Eloquent\\Assets\\{closure}(Array, 0)
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Arr.php(869): array_filter(Array, Object(Closure), 1)
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Collection.php(387): Illuminate\\Support\\Arr::where(Array, Object(Closure))
#8 /var/www/html/vendor/statamic/eloquent-driver/src/Assets/AssetContainerContents.php(76): Illuminate\\Support\\Collection->filter(Object(Closure))
#9 /var/www/html/vendor/statamic/cms/src/Assets/AssetContainer.php(354): Statamic\\Eloquent\\Assets\\AssetContainerContents->filteredDirectoriesIn('', false)
#10 /var/www/html/vendor/statamic/eloquent-driver/src/Commands/SyncAssets.php(87): Statamic\\Assets\\AssetContainer->folders('')
#11 /var/www/html/vendor/statamic/eloquent-driver/src/Commands/SyncAssets.php(46): Statamic\\Eloquent\\Commands\\SyncAssets->processFolder(Object(Statamic\\Assets\\AssetContainer))
#12 /var/www/html/vendor/statamic/eloquent-driver/src/Commands/SyncAssets.php(37): Statamic\\Eloquent\\Commands\\SyncAssets->processContainer(Object(Statamic\\Assets\\AssetContainer))
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Collections/Traits/EnumeratesValues.php(236): Statamic\\Eloquent\\Commands\\SyncAssets->Statamic\\Eloquent\\Commands\\{closure}(Object(Statamic\\Assets\\AssetContainer), 0)
#14 /var/www/html/vendor/statamic/eloquent-driver/src/Commands/SyncAssets.php(37): Illuminate\\Support\\Collection->each(Object(Closure))
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Statamic\\Eloquent\\Commands\\SyncAssets->handle()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(211): Illuminate\\Container\\Container->call(Array)
#21 /var/www/html/vendor/symfony/console/Command/Command.php(326): Illuminate\\Console\\Command->execute(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(181): Symfony\\Component\\Console\\Command\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))
#23 /var/www/html/vendor/symfony/console/Application.php(1063): Illuminate\\Console\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#24 /var/www/html/vendor/symfony/console/Application.php(320): Symfony\\Component\\Console\\Application->doRunCommand(Object(Statamic\\Eloquent\\Commands\\SyncAssets), Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#25 /var/www/html/vendor/symfony/console/Application.php(174): Symfony\\Component\\Console\\Application->doRun(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(201): Symfony\\Component\\Console\\Application->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#27 /var/www/html/please(37): Illuminate\\Foundation\\Console\\Kernel->handle(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#28 {main}
"} 

When I try to access the container through the CP it's empty. I also don't see any of the processed assets in my database. Just the ones that were already there (the only assets that had meta data)

[ryanmitchell]

Interesting. Can you share a dd of $this->directories() inserted somewhere around line 69 of that file? You seem to be getting a nested directory list, which shouldn't be the case and I just want to confirm.

[ryanmitchell]

I've opened a PR here with the changes I think are required. Can you test it and let me know if it works for you?

[GioChocolateBro]

Hi @ryanmitchell ,

Thanks for the quick response. It does continue now but gets stuck into a recursive loop. It's been processing - re-processing the same folder for quite some time now.

It processed a couple of folder and then got stuck on a double nested folder. root > events > event-name > file.png

it keeps processing folder event, then event-1, event, event-1 and so on.

Also I still don't see any new entries in my assets_meta table. It's supposed to enter any processed items into the table right? This is what will give the performance gains in terms of not having to talk to digitalocean?

[GioChocolateBro]

[ryanmitchell]

Does this change help? https://github.com/ryanmitchell/eloquent-driver/commit/b7dc24eadc7a9cf3c1abdfee9f6b6bad54a5475f

Laravel can try to be clever with database inserts and batch them, so it's possible youre not seeing updates due to that. Also do make sure you've got assets set to eloquent in the eloquent-driver config.

[GioChocolateBro]

Initially it throws an exception because $folder isn't accessible for the if statement on line 89 ->each(function ($subfolder) use ($container, $folder) {

When I added it, it got stuck on the first folder (not the double nested folder) in the same manner as before.

[ryanmitchell]

@GioChocolateBro I've pushed an update which should fix that.

[GioChocolateBro]

Hi @ryanmitchell ,

Yes this seems to work, thanks a bunch 🌟 ! It's been running for some time now. It will be a while before it finishes syncing.

But it has definitely synced over 3K assets already and there has been no batch insert yet. I checked my config again and it is set to eloquent

    'asset_containers' => [
        'driver' => 'file',
        'model'  => \Statamic\Eloquent\Assets\AssetContainerModel::class,
    ],

    'assets' => [
        'driver' => 'eloquent',
        'model'  => \Statamic\Eloquent\Assets\AssetModel::class,
        'asset'  => \Statamic\Eloquent\Assets\Asset::class,
    ],

Should I still be holding out for a batch insert or is there something else going on?

[ryanmitchell]

Im not sure on that, its inserting fine on my side so you'd need to debug it a bit yourself to see why its not inserting.

[GioChocolateBro]

After exiting the sync and starting it again it will fall into recursion again on the first folder it encounters. This time indefinitely creating deeper levels of meta folders.

[GioChocolateBro]

@ryanmitchell , I've been digging around but I don't see where inside the SyncAsset writeMeta() should be getting called?

Inside the ImportAsset command it's very clearly creating the database records, but inside SyncAsset I can't find any reference to it.

[ryanmitchell]

it happens inside the asset class. I dont have any further time to work on this today (sorry).

[GioChocolateBro]

No worries, your work is greatly appreciated.

[ryanmitchell]

I added some code to prevent .meta folders being processed - maybe that helps your error as well.

[ryanmitchell]

@GioChocolateBro can you pull down the latest version of the PR and see if the changes resolve the issues you have experienced?

[GioChocolateBro]

Hi @ryanmitchell ,

Sorry, I had a busy week and lost track of this PR. I will be checking it out next week and try to upgrade my app to 3.x again. Thanks so much for your quick assistance and all your work on the eloquent driver 🙏 ❤️ .