Error while creating folder after deleting it from gdrive

[gcilinskas]

Steps to replicate:

1. Create folder Storage::disk('google')->createDir('test');
2. Delete 'test' folder in Google drive manually(delete from trash also)
3. Create same folder Storage::disk('google')->createDir('test');

Locally works, but throws this error on live servers (AWS EC2 Instance Ubuntu)

[2021-07-16 06:59:58] staging.ERROR: Unknown exception encountered while retrieving product or dispatching asset download jobs: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "notFound",
    "message": "File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA.",
    "locationType": "parameter",
    "location": "fileId"
   }
  ],
  "code": 404,
  "message": "File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA."
 }
}
 {"asin":"B01LSUQSB0","marketplace":"us","exception":"[object] (Google\\Service\\Exception(code: 404): {
 \"error\": {
  \"errors\": [
   {
    \"domain\": \"global\",
    \"reason\": \"notFound\",
    \"message\": \"File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA.\",
    \"locationType\": \"parameter\",
    \"location\": \"fileId\"
   }
  ],
  \"code\": 404,
  \"message\": \"File not found: 1255h34g2AtWNgO4wjJPzzfHlxh_yIIyA.\"
 }
}
 at /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Http/REST.php:128)
[stacktrace]
#0 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Http/REST.php(103): Google\\Http\\REST::decodeHttpResponse()
#1 [internal function]: Google\\Http\\REST::doExecute()
#2 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Task/Runner.php(182): call_user_func_array()
#3 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Http/REST.php(66): Google\\Task\\Runner->run()
#4 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Client.php(898): Google\\Http\\REST::execute()
#5 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient/src/Service/Resource.php(238): Google\\Client->execute()
#6 /var/www/html/contentx/staging/releases/20211507_122037/vendor/google/apiclient-services/src/Drive/Resource/Files.php(101): Google\\Service\\Resource->call()
#7 /var/www/html/contentx/staging/releases/20211507_122037/vendor/masbug/flysystem-google-drive-ext/src/GoogleDriveAdapter.php(1282): Google\\Service\\Drive\\Resource\\Files->create()
#8 /var/www/html/contentx/staging/releases/20211507_122037/vendor/masbug/flysystem-google-drive-ext/src/GoogleDriveAdapter.php(573): Masbug\\Flysystem\\GoogleDriveAdapter->createDirectory()
#9 /var/www/html/contentx/staging/releases/20211507_122037/vendor/league/flysystem/src/Filesystem.php(263): Masbug\\Flysystem\\GoogleDriveAdapter->createDir()
#10 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Filesystem/FilesystemAdapter.php(787): League\\Flysystem\\Filesystem->createDir()
#11 /var/www/html/contentx/staging/releases/20211507_122037/app/Jobs/GetAsinDataJob.php(95): Illuminate\\Filesystem\\FilesystemAdapter->__call()
#12 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\\Jobs\\GetAsinDataJob->handle()
#13 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#14 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#15 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#16 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#17 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\\Container\\Container->call()
#18 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Bus\\Dispatcher->Illuminate\\Bus\\{closure}()
#19 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#20 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\\Pipeline\\Pipeline->then()
#21 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(120): Illuminate\\Bus\\Dispatcher->dispatchNow()
#22 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Queue\\CallQueuedHandler->Illuminate\\Queue\\{closure}()
#23 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#24 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(122): Illuminate\\Pipeline\\Pipeline->then()
#25 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(70): Illuminate\\Queue\\CallQueuedHandler->dispatchThroughMiddleware()
#26 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(98): Illuminate\\Queue\\CallQueuedHandler->call()
#27 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(428): Illuminate\\Queue\\Jobs\\Job->fire()
#28 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(378): Illuminate\\Queue\\Worker->process()
#29 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(172): Illuminate\\Queue\\Worker->runJob()
#30 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(117): Illuminate\\Queue\\Worker->daemon()
#31 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(101): Illuminate\\Queue\\Console\\WorkCommand->runWorker()
#32 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\\Queue\\Console\\WorkCommand->handle()
#33 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#34 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#35 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#36 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Container/Container.php(651): Illuminate\\Container\\BoundMethod::call()
#37 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Console/Command.php(136): Illuminate\\Container\\Container->call()
#38 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Command/Command.php(299): Illuminate\\Console\\Command->execute()
#39 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Console/Command.php(121): Symfony\\Component\\Console\\Command\\Command->run()
#40 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Application.php(978): Illuminate\\Console\\Command->run()
#41 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Application.php(295): Symfony\\Component\\Console\\Application->doRunCommand()
#42 /var/www/html/contentx/staging/releases/20211507_122037/vendor/symfony/console/Application.php(167): Symfony\\Component\\Console\\Application->doRun()
#43 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Console/Application.php(92): Symfony\\Component\\Console\\Application->run()
#44 /var/www/html/contentx/staging/releases/20211507_122037/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\\Console\\Application->run()
#45 /var/www/html/contentx/staging/releases/20211507_122037/artisan(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#46 {main}

[erikn69]

This package is using a little cache trick, if you delete a folder in Google drive manually, that folder still remains in cache even though it no longer exists in google drive (if you are using a long running script, because on every new instance it starts the connection and cache), maybe the function Storage::disk('google')->getDriver()->getAdapter()->clearCache(); before createDir will be useful to you (maybe with catch, only when there is these exception). https://github.com/masbug/flysystem-google-drive-ext/blob/aa0b678e99fb668da0457c08d62183df6fa47971/src/GoogleDriveAdapter.php#L257-L263 @masbug is this a bug?, maybe i can make a fix

[gcilinskas]

Storage::disk('google')->getDriver()->getAdapter()->clearCache(); This one helped, thanks! Just for information, Since I use queued jobs, I had to call this clearCache command not only once before creating directories, but also in every job class that was using google drive upload

[erikn69]

On limitation's section

Using display paths as identifiers for folders and files requires them to be unique. Unfortunately Google Drive allows users to create files and folders with same (displayed) names. In such cases when unique path cannot be determined this adapter chooses the oldest (first) instance

Concurrent use of same Google Drive might lead to unexpected problems due to heavy caching of file/folder identifiers and file objects.

So, for concurrent threads of queued jobs you have to do this workaround with cache, otherwise you get those errors, or when create a directory duplicate(created at the same time with same name) and you put a unique file or folder inside, the adapter will be able to reach it properly because it chooses the oldest (first), so, create directory, uncache the folder, then put file inside I make a PR with uncacheFolder function, if it is merged it be available on the next release

$disk=\Storage::disk('google');    
$disk->createDir('test/testa');
//manually delete without adapter or other job create it first 
$dir=$disk->getDriver()->getAdapter()->getMetadata('test/testa');
$disk->getDriver()->getAdapter()->getService()->files->delete($dir['id']); 
//uncache
$disk->getDriver()->getAdapter()->uncacheFolder('test/testa'); 
// refresh id and put file
$disk->put('test/testa/file.txt', 'Contents1'); 

[masbug]

@gcilinskas As @erikn69 pointed out, extensive object caching breaks concurrency. The code of this adapter assumes that it's operating alone on a selected drive. One way to partially work around this would be to disable all caching in the code (which would slow everything down considerably)...but that's only half of the fix - there's still the issue of mixing write, read, delete, create operations and you'd still have race conditions to solve. Perhaps you can just change the approach and not work on the same folder from multiple threads/jobs/queues? Or just not delete the folder after you're done with it (I assume you use one job to fill the folder and another to consume what's in it and then delete the folder)?

[gcilinskas]

Yeah, manual human interaction with gdrive is possible on our side so removal of folders might be at random times. We are fetching data from api and saving its assets to gdrive in structured folders hierarchy.

clearCache method helped for now, performance for gdrive upload is not crucial for us. Regarding to racing conditions we did some workarounds with sleep functions, currently seems to work

We don't have options to change approaches in usage of gdrive folders

[masbug]

Ok. So I guess you're doing something similar to:

lRetry:
try {
  ... small set of operations on the drive or a complete set with error mitigation
} catch(\Throwable $e) {
  sleep(rand()%4);
  $adapter->clearCache();
  if($retry-->0)
     goto lRetry;
}