search

dizda / CloudBackupBundle

Be able to backup your database(s) and upload it to the cloud (Dropbox, Amazon S3, GoogleDrive, etc.)
MIT License
196 stars 57 forks source link

Better timeout options #81

Closed Nyholm closed 8 years ago

Nyholm commented 8 years ago

When I upload 100Mb of data I get a time out. This PR solves that issue.

dizda commented 8 years ago

Could it be possible to let the user override a default parameter ?

dizda commented 8 years ago

To make it simple, define a parameter there https://github.com/dizda/CloudBackupBundle/blob/master/Resources/config/services.yml#L1, so a user will be able to override it directly from the parameters.yml

Nyholm commented 8 years ago

Yeah, agreed. This is not usually something you want to change... ever. You have my update this afternoon.

dizda commented 8 years ago

Super! Thanks for the PR!

Nyholm commented 8 years ago

I'm still running in with timeout issues. I tried to update the DriveClient to upload files in chunks but I'm not too sure it works either..

dizda commented 8 years ago

Could you copy the exception you get?

Nyholm commented 8 years ago

Sure: 

{  
   "class":"Google_IO_Exception",
   "message":"Operation timed out after 100000 milliseconds with 0 bytes received",
   "code":28,
   "file":"/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/google/apiclient/src/Google/IO/Curl.php:118",
   "trace":[  
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/google/apiclient/src/Google/IO/Abstract.php:136",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/google/apiclient/src/Google/Http/MediaFileUpload.php:145",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/google/apiclient/src/Google/Http/MediaFileUpload.php:196",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/dizda/cloud-backup-bundle/Dizda/CloudBackupBundle/Client/GoogleDriveClient.php:255",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/dizda/cloud-backup-bundle/Dizda/CloudBackupBundle/Client/GoogleDriveClient.php:193",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/dizda/cloud-backup-bundle/Dizda/CloudBackupBundle/Client/GoogleDriveClient.php:56",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/dizda/cloud-backup-bundle/Dizda/CloudBackupBundle/Manager/ClientManager.php:61",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/dizda/cloud-backup-bundle/Dizda/CloudBackupBundle/Manager/BackupManager.php:69",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/dizda/cloud-backup-bundle/Dizda/CloudBackupBundle/Command/BackupCommand.php:36",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:256",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:846",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:189",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:96",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:120",
      "/home/sites/io.happyr.demo/sf/releases/20151029094226/app/console:27"
   ]
}
Nyholm commented 8 years ago

This is the demo database.. It's less that 5Mb. Im now trying to use the google/client: 1.1.4

dizda commented 8 years ago

I have no idea man..

Nyholm commented 8 years ago

It is an interesting problem. I see the problem on different machines, different applications, different google accounts, file sizes vary between 500kB, 7MB and 100MB. The problem is not even consistent. It shows up about 50% of the times I try...

When I try the example it works every time... At least from the local machine...

I'll investigate further..

Nyholm commented 8 years ago

I created new google API credentials and now it seams to work... Really strange bug.

I don't know if this PR helps to solve the issue. But anyhow; now we upload files in chunks and you may configure the timeout.

dizda commented 8 years ago

Oh ok.

What what are the advantages to upload files in chunks?

Nyholm commented 8 years ago

That's the recommended way of upload big files. Sending large request might fail on slow networks.. and you'll get timeouts..

dizda commented 8 years ago

Alright, so if you're agree, I merge it.

Nyholm commented 8 years ago

Please. Thank you.