search

l3uddz / cloudplow

Automatic rclone remote uploader, with support for multiple remote/folder pairings. UnionFS Cleaner functionality: Deletion of UnionFS whiteout files and their corresponding files on rclone remotes. Automatic remote syncer: Sync between different remotes via a Scaleway server instance, that is created and destroyed at every sync.
GNU General Public License v3.0
344 stars 46 forks source link

Manage service accounts better by rotating #77

Closed seertenedos closed 2 years ago

seertenedos commented 4 years ago

If you upload a lot of data to multiple team dives using a pool of service accounts it would be great if the app remembered how much it has uploaded on each service account recently and pick the one it has uploaded the least on instead of seeming to choose the first service account and working it way though them till it finds one that works fromt he start each time.

It would also be good if in the rclone sleeps you could specify to move to the next key as well.

seertenedos commented 4 years ago

i have 5 jobs configured using the same server key pool and each one starts at servie key 1 and works up each time enven though other jobs and itself on past runs have gone past service key one. it really needs a memory of which one it is up to.

SaintNick1214 commented 2 years ago

This drives me crazy as well. I even have sarotate running in background and cloudplow always decides to use 1.json when it starts and just sits there with errors not changing to another json within a reasonable amount of time.

From log: There is 2850 available service accounts Using service account: /opt/sa/1.json

Sounds silly just looking at it.

saltydk commented 2 years ago

Consdering multiple applications could be using the service accounts I don't see the point of tracking this.