search

l3uddz / cloudplow

Automatic rclone remote uploader, with support for multiple remote/folder pairings. UnionFS Cleaner functionality: Deletion of UnionFS whiteout files and their corresponding files on rclone remotes. Automatic remote syncer: Sync between different remotes via a Scaleway server instance, that is created and destroyed at every sync.
GNU General Public License v3.0
345 stars 46 forks source link

Fix throttle check logic #124

Closed RobLaw0304 closed 2 years ago

RobLaw0304 commented 2 years ago

The current master and develop branch has an issue where cloudplow is not recognizing that a throttle has been applied and therefore sends rclone a throttle command during every check when all conditions are met. The expected behavior is that if cloudplow detects a throttle has already been placed, it does not take an action. This commit resolves this issue by removing the check on the non-existent speed attribute under rate. This is not longer there in rclone so the check was causing success to always be false instead of returning true when throttle was applied.

saltydk commented 2 years ago

What version of rclone are you running that makes you think they removed the speed from core/bwlimit?

saltydk commented 2 years ago

Just tested it on my dev box and it works fine there so you'd need to be more specific as to why you don't think it works. It is also not checking for a speed attribute but for the value set in speed in data['rate']

RobLaw0304 commented 2 years ago

Thank you for the response. I have had this issue since I began using cloudplow. Throttling works, but the messages I receive were not consistent with the state. your response helped me realize that the format of my config file was inconsistent. In order to use decimal points with cloudplow you must specify 3 decimal places for it to correctly detect that throttling has been activated. After correcting my config to use 3 decimal places it correctly detects. That being said I think maybe some improvements in detection could be made so i'm going to take a look at that just for quality of life. Here is how my config was set (couldn't detect that it was successfully throttling though it was):

      "throttle_speeds": {
        "0": "4.5M",
        "1": "1.75M",
        "2": "1.5M",
        "3": "0.5M",
        "4": "0.05M",
        "5": "0.01M",
        "6": "0.001M"
      },

I changed my config to this and now it works as expected.

      "throttle_speeds": {
        "0": "4.500M",
        "1": "1.750M",
        "2": "1.500M",
        "3": "500K",
        "4": "100K",
        "5": "10K",
        "6": "1K"
      },

Here is an excerpt of my current state log showing the throttling behavior current state.log

Given that you helped me solve the immediate issue i'll close this pull request. I will take a look at enhancements to standardizing speed strings so more strings are recognized.

saltydk commented 2 years ago

Happy to help even if it was by accident :P