search

l3uddz / cloudplow

Automatic rclone remote uploader, with support for multiple remote/folder pairings. UnionFS Cleaner functionality: Deletion of UnionFS whiteout files and their corresponding files on rclone remotes. Automatic remote syncer: Sync between different remotes via a Scaleway server instance, that is created and destroyed at every sync.
GNU General Public License v3.0
344 stars 46 forks source link

"Schedule" is not working yet? or Did I wrong... #59

Closed sodakhaki closed 5 years ago

sodakhaki commented 5 years ago

Hello Sir,

The cloudplow is a great program and I've used it over 3 months without issue. For more comfi internet I would like to run this as per schedule, however it is not working as I planned.

Below is my config.json. Sorry for messy layout as I don't know how to put the code with space/tab.

    "uploader": {
        "google": {
            "check_interval": 30,
            "exclude_open_files": false,
            "max_size_gb": 2,
            "opened_excludes": [
                "/downloads/"
            ],
            "schedule": {
                "allowed_from": "00:05",
                "allowed_until": "12:00",
                "enabled": true
            },
            "size_excludes": [
                "downloads/*"
            ]
        }
    }

And.. Below is a recent log took yesterday. Above setting was used for this operations by reboot.

 2019-05-11 19:45:14,036 - INFO       - git                  - check_version                  - You are using the latest master: 5f6a4c35c2999e8661ddc957b3fa34fff6f1367a
 2019-05-11 19:45:14,036 - INFO       - cloudplow            - <module>                       - Started in run mode
 2019-05-11 19:45:14,037 - INFO       - cloudplow            - <module>                       - Added google uploader to schedule, checking available disk space every 30 minutes
 2019-05-11 20:15:14,792 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 340 GB over the maximum limit of 30 GB
 2019-05-11 20:15:14,793 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 20:15 is not within the allowed upload time periods 00:05 -> 16:00
 2019-05-11 20:45:15,602 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 340 GB over the maximum limit of 30 GB
 2019-05-11 20:45:15,602 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 20:45 is not within the allowed upload time periods 00:05 -> 16:00
 2019-05-11 21:15:16,361 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 340 GB over the maximum limit of 30 GB
 2019-05-11 21:15:16,361 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 21:15 is not within the allowed upload time periods 00:05 -> 16:00
 2019-05-11 21:45:17,143 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 369 GB over the maximum limit of 30 GB
 2019-05-11 21:45:17,144 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 21:45 is not within the allowed upload time periods 00:05 -> 16:00
 2019-05-11 21:57:08,141 - INFO       - lock                 - ensure_lock_folder             - Created lock folder: '/opt/cloudplow/locks'
 2019-05-11 21:57:09,652 - INFO       - git                  - check_version                  - You are using the latest master: 5f6a4c35c2999e8661ddc957b3fa34fff6f1367a
 2019-05-11 21:57:09,652 - INFO       - cloudplow            - <module>                       - Started in run mode
 2019-05-11 21:57:09,662 - INFO       - cloudplow            - <module>                       - Added google uploader to schedule, checking available disk space every 30 minutes
 2019-05-11 22:27:09,706 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 403 GB over the maximum limit of 2 GB
 2019-05-11 22:27:09,707 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 22:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-11 22:57:10,496 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 415 GB over the maximum limit of 2 GB
 2019-05-11 22:57:10,496 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 22:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-11 23:27:11,353 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 442 GB over the maximum limit of 2 GB
 2019-05-11 23:27:11,355 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 23:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-11 23:57:12,326 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 442 GB over the maximum limit of 2 GB
 2019-05-11 23:57:12,326 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 23:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 00:27:13,131 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 442 GB over the maximum limit of 2 GB
 2019-05-12 00:27:13,131 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 00:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 00:57:13,843 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 442 GB over the maximum limit of 2 GB
 2019-05-12 00:57:13,844 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 00:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 01:27:14,546 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 443 GB over the maximum limit of 2 GB
 2019-05-12 01:27:14,546 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 01:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 01:57:15,442 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 444 GB over the maximum limit of 2 GB
 2019-05-12 01:57:15,442 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 01:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 02:27:16,375 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 444 GB over the maximum limit of 2 GB
 2019-05-12 02:27:16,376 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 02:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 02:57:17,315 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 444 GB over the maximum limit of 2 GB
 2019-05-12 02:57:17,316 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 02:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 03:27:18,286 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 444 GB over the maximum limit of 2 GB
 2019-05-12 03:27:18,286 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 03:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 03:57:19,203 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 444 GB over the maximum limit of 2 GB
 2019-05-12 03:57:19,204 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 03:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 04:27:20,127 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 446 GB over the maximum limit of 2 GB
 2019-05-12 04:27:20,127 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 04:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 04:57:21,082 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 446 GB over the maximum limit of 2 GB
 2019-05-12 04:57:21,082 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 04:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 05:27:22,030 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 446 GB over the maximum limit of 2 GB
 2019-05-12 05:27:22,031 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 05:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 05:57:22,979 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 446 GB over the maximum limit of 2 GB
 2019-05-12 05:57:22,980 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 05:57 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 06:27:23,876 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 449 GB over the maximum limit of 2 GB
 2019-05-12 06:27:23,876 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 06:27 is not within the allowed upload time periods 00:05 -> 12:00
 2019-05-12 06:57:24,760 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 449 GB over the maximum limit of 2 GB
 2019-05-12 06:57:24,760 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 06:57 is not within the allowed upload time periods 00:05 -> 12:00`

It seems that the setting is applied well, but running time calculation is not working.

Thanks in advance.

sodakhaki commented 5 years ago

After a day, still same

2019-05-13 03:57:57,585 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 606 GB over the maximum limit of 2 GB
2019-05-13 03:57:57,586 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 03:57 is not within the allowed upload time periods 00:05 -> 12:00
2019-05-13 04:27:58,266 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 606 GB over the maximum limit of 2 GB
2019-05-13 04:27:58,266 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 04:27 is not within the allowed upload time periods 00:05 -> 12:00
2019-05-13 04:57:58,984 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 606 GB over the maximum limit of 2 GB
2019-05-13 04:57:58,984 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 04:57 is not within the allowed upload time periods 00:05 -> 12:00
2019-05-13 05:27:59,657 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 606 GB over the maximum limit of 2 GB
2019-05-13 05:27:59,658 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 05:27 is not within the allowed upload time periods 00:05 -> 12:00
2019-05-13 05:58:00,294 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 606 GB over the maximum limit of 2 GB
2019-05-13 05:58:00,294 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 05:58 is not within the allowed upload time periods 00:05 -> 12:00
2019-05-13 06:28:00,917 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. Local folder size is currently 606 GB over the maximum limit of 2 GB
2019-05-13 06:28:00,917 - INFO       - cloudplow            - scheduled_uploader             - Uploader: google. The current time 06:28 is not within the allowed upload time periods 00:05 -> 12:00
sodakhaki commented 5 years ago

If the service is restarted within the allowed time, then it works well for that instance but next instance, i.e. 2nd upload after 1st upload completion, is not working.

dmemedo commented 5 years ago

got the exact same problem too

desimaniac commented 5 years ago

We'll need to look into this. Labeled it as "bug" for now.

PS @sodakhaki that's are how you put stuff in code blocks.

rackey12 commented 5 years ago

same issue can provide logs and config if needed.

desimaniac commented 5 years ago

Fix has been pushed.