search

spatie / laravel-backup

A package to backup your Laravel app
https://spatie.be/docs/laravel-backup
MIT License
5.65k stars 763 forks source link

AWS HTTP DELETE resulted in a 503 Slow Down response #1067

Closed patriziotomato closed 4 years ago

patriziotomato commented 4 years ago

Getting (from time to time) this exception while cleaning up backups:


Error executing "DeleteObject" on "https://asdf.fra1.digitaloceanspaces.com/backup/asdf/2020-04-27-19-12-02.zip"; AWS HTTP error: Server error: DELETE https://asdf.fra1.digitaloceanspaces.com/backup/asdf/2020-04-27-19-12-02.zip resulted in a 503 Slow Down response:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>SlowDown</Code>
<Message>Please reduce your request rate.</Message> (truncated...)
SlowDown (server): Please reduce your request rate. - <?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>SlowDown</Code>
<Message>Please reduce your request rate.</Message>
<RequestId></RequestId>
</Error>`

Stacktrace:

`
backup::notifications.exception_message_trace
#0 /srv/www/release-2020-04-30_14-01_master/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php(100): Aws\WrappedHttpHandler->parseError(Array, Object(GuzzleHttp\Psr7\Request), Object(Aws\Command), Array)
#1 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(203): Aws\WrappedHttpHandler->Aws\{closure}(Array)
#2 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(174): GuzzleHttp\Promise\Promise::callHandler(2, Array, Array)
#3 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/RejectedPromise.php(40): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}(Array)
#4 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/TaskQueue.php(47): GuzzleHttp\Promise\RejectedPromise::GuzzleHttp\Promise\{closure}()
#5 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(119): GuzzleHttp\Promise\TaskQueue->run()
#6 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(146): GuzzleHttp\Handler\CurlMultiHandler->tick()
#7 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(246): GuzzleHttp\Handler\CurlMultiHandler->execute(true)
#8 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(223): GuzzleHttp\Promise\Promise->invokeWaitFn()
#9 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(267): GuzzleHttp\Promise\Promise->waitIfPending()
#10 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(225): GuzzleHttp\Promise\Promise->invokeWaitList()
#11 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(267): GuzzleHttp\Promise\Promise->waitIfPending()
#12 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(225): GuzzleHttp\Promise\Promise->invokeWaitList()
#13 /srv/www/release-2020-04-30_14-01_master/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
#14 /srv/www/release-2020-04-30_14-01_master/vendor/aws/aws-sdk-php/src/AwsClientTrait.php(58): GuzzleHttp\Promise\Promise->wait()
#15 /srv/www/release-2020-04-30_14-01_master/vendor/league/flysystem-aws-s3-v3/src/AwsS3Adapter.php(186): Aws\AwsClient->execute(Object(Aws\Command))
#16 /srv/www/release-2020-04-30_14-01_master/vendor/league/flysystem/src/Filesystem.php(237): League\Flysystem\AwsS3v3\AwsS3Adapter->delete('Electrolux-DAM-...')
#17 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Filesystem/FilesystemAdapter.php(345): League\Flysystem\Filesystem->delete('Electrolux-DAM-...')
#18 /srv/www/release-2020-04-30_14-01_master/vendor/spatie/laravel-backup/src/BackupDestination/Backup.php(85): Illuminate\Filesystem\FilesystemAdapter->delete(Array)
#19 /srv/www/release-2020-04-30_14-01_master/vendor/spatie/laravel-backup/src/Tasks/Cleanup/Strategies/DefaultStrategy.php(111): Spatie\Backup\BackupDestination\Backup->delete()
#20 /srv/www/release-2020-04-30_14-01_master/vendor/spatie/laravel-backup/src/Tasks/Cleanup/Strategies/DefaultStrategy.php(39): Spatie\Backup\Tasks\Cleanup\Strategies\DefaultStrategy->removeOldBackupsUntilUsingLessThanMaximumStorage(Object(Spatie\Backup\BackupDestination\BackupCollection))
#21 /srv/www/release-2020-04-30_14-01_master/vendor/spatie/laravel-backup/src/Tasks/Cleanup/CleanupJob.php(42): Spatie\Backup\Tasks\Cleanup\Strategies\DefaultStrategy->deleteOldBackups(Object(Spatie\Backup\BackupDestination\BackupCollection))
#22 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Support/Traits/EnumeratesValues.php(197): Spatie\Backup\Tasks\Cleanup\CleanupJob->Spatie\Backup\Tasks\Cleanup\{closure}(Object(Spatie\Backup\BackupDestination\BackupDestination), 0)
#23 /srv/www/release-2020-04-30_14-01_master/vendor/spatie/laravel-backup/src/Tasks/Cleanup/CleanupJob.php(54): Illuminate\Support\Collection->each(Object(Closure))
#24 /srv/www/release-2020-04-30_14-01_master/vendor/spatie/laravel-backup/src/Commands/CleanupCommand.php(42): Spatie\Backup\Tasks\Cleanup\CleanupJob->run()
#25 [internal function]: Spatie\Backup\Commands\CleanupCommand->handle()
#26 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(33): call_user_func_array(Array, Array)
#27 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Container/Util.php(36): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#28 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(91): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#29 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#30 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Container/Container.php(592): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#31 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Console/Command.php(134): Illuminate\Container\Container->call(Array)
#32 /srv/www/release-2020-04-30_14-01_master/vendor/symfony/console/Command/Command.php(255): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#33 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Console/Command.php(121): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#34 /srv/www/release-2020-04-30_14-01_master/vendor/spatie/laravel-backup/src/Commands/BaseCommand.php(16): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#35 /srv/www/release-2020-04-30_14-01_master/vendor/symfony/console/Application.php(912): Spatie\Backup\Commands\BaseCommand->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#36 /srv/www/release-2020-04-30_14-01_master/vendor/symfony/console/Application.php(264): Symfony\Component\Console\Application->doRunCommand(Object(Spatie\Backup\Commands\CleanupCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#37 /srv/www/release-2020-04-30_14-01_master/vendor/symfony/console/Application.php(140): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#38 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Console/Application.php(93): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#39 /srv/www/release-2020-04-30_14-01_master/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#40 /srv/www/release-2020-04-30_14-01_master/artisan(37): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#41 {main}
`
freekmurze commented 4 years ago

I accept a PR that improves this behaviour.

pmochine commented 4 years ago

I still have this issue !