dizda / CloudBackupBundle

Be able to backup your database(s) and upload it to the cloud (Dropbox, Amazon S3, GoogleDrive, etc.)
MIT License
196 stars 57 forks source link

Dropbox daily monthly #128

Open mctdev-marius opened 7 years ago

mctdev-marius commented 7 years ago

Dropbox will have 2 directory’s one for monthly and one for daily updates each folder has a limit of backups. If the current date is the first day of the month the backup will upload to the monthly folder if not it will end up in the daily backup folder. When the upload has finished it will check to see if the folder has reached the file limit if so it will delete the oldest file.

jongotlin commented 7 years ago

Hi! Bumped into this issue recently. The problem is Travis is using a global phpunit and it is not working the same as before. Have a look at https://github.com/kompisbyran/Kompisbyran/pull/190 and see if you can solve it this way in this repo too.

dizda commented 7 years ago

Hey @mctdev-marius !

Thanks for this PR, this is quite interesting! You guys always have nice ideas that I had never thought about! 😃 But there is one disadvantage about this, I'm afraid that may complicate the bundle, and not cross-storage compatible for a non-vital feature. I'm sure we can find a simpler way to do it, and managing it directly right from the crontab manager.

What about just specifying the remote_path directly from the command line? Let me show you an exemple:

#every day
1 1 * * * SYMFONY__REMOTE__PATH=/normal/path php app/console --env=prod dizda:backup:start >/dev/null 2>&1
#first day of the month
1 1 1 * * SYMFONY__REMOTE__PATH=/different/path php app/console --env=prod dizda:backup:start >/dev/null 2>&1

More details here http://symfony.com/doc/master/configuration/external_parameters.html

Then you can add more crons to delete oldest files in both directories if more than X items: http://stackoverflow.com/questions/25785/delete-all-but-the-most-recent-x-files-in-bash

What do you guys think?

mctdev-marius commented 7 years ago

Hello @dizda , and thanks for your work on this project! I did not know about the external parameters so thanks for that, it can be very helpful. As for the crons to delete old files it will not work in my case (DropBoxSDK storage). You see, my integration is not meant for all the storage is just (for now) DropboxSDK oriented. In fact all the code add (except for the Travis PHP 7 part) is on the configuration (so i can make the user set up the right parameters) and DropboxSDK client file.

What it dose is uploads the backup archive to different DropBox directory based on the day of month, then it checks the dropbox file list to see if it needs to delete files based on the settings for DropboxSDK. This can be very useful if your dropbox account has a space limit and u don't want to have an infinite number of backups eating up your subscription plan.

I normally have a 5 files for the daily and 2 for monthly.

Please let me know if i'm missing the long term problems that can complicate a already grate bundle (yours).

dizda commented 7 years ago

Yeah sorry, I didn't get than you were talking about removing remote files on the cloud storage rather than the ones on your local...

So I might be agree on the utility to remove files if the total number is higher than a specified amount. But I'm really not sure that other people would need to upload the backup to specific directories depending on the current day. To me, you should handle that on your side, but let's see what other people think?

mctdev-marius commented 7 years ago

Yes i agree, the directory and the limit number based on the folder they end up in can be set via crons so you can make of whit out extra code to decide where to upload the created backup. I add it because i did not know about the parameters sent via cl so i need to keep my monthly/daily backups organized and limit the amount of backups stored meaning i don't end up whit 2 backups if it's the last day of month (say you wanna share only the monthly backups whit your client).

Let's see if i'm the only one needing this running this way or if it can bee useful to others.