search

l3uddz / cloudplow

Automatic rclone remote uploader, with support for multiple remote/folder pairings. UnionFS Cleaner functionality: Deletion of UnionFS whiteout files and their corresponding files on rclone remotes. Automatic remote syncer: Sync between different remotes via a Scaleway server instance, that is created and destroyed at every sync.
GNU General Public License v3.0
338 stars 48 forks source link

Size of upload is incorrect in Notification #89

Closed wjbeckett closed 2 years ago

wjbeckett commented 4 years ago

Describe the bug I have set a min-age on files that I was uploaded with Cloudplow as an rclone_extra setting. The setting seems to filter the files for upload correctly, however the notification I receive tells me it is uploading the size of the source directory, not the actual upload size.

To Reproduce Steps to reproduce the behavior:

  1. In rclone_extras, set "--min-age": "365d"
  2. Configure dry_run so nothing is actually uploaded
  3. perform a manual upload and observe the upload size in the notification (in my case, slack)
  4. See the incorrect upload size.

Expected behavior I would expect that the notification I receive should be for the size of the actual upload, not just the size of my source directory.

Screenshots Actual size of media greater than 365d: image

Notification from Cloudplow: image

System Information

efanibi25 commented 3 years ago

I don't know how you would remove this behavior from the program rclone is the one handle the program handle the additonal filtering.

The line in the code that controls that output is path.get_size(rclone_config['upload_folder'], uploader_config['size_excludes'])

So it looks like is is the size of the folder -excluded directories. So the filter you added isn't part of the config file and therefore is not counted

So your right once rclone adds commands that filter the size of that further down it won't be correct

saltydk commented 2 years ago

As stated in the previous comment this is not really something you can expect cloudplow to react to. Can always edit the command used in the path.py if you really want a fix for your particular use case.