Multiple Cloud Mounts

[rkdavies]

I'm just checking for feature availability.

Is it possible to mount two separate Google Drive mounts?

Or is it just a matter of adding another [DriveName] type/scope drive, etc. to rclone.conf?

Thank you!

[kelinger]

You're not the first to ask but I'll respond with a question:

What are your "intentions" with the second mount? Is it "read-only" or should some newly downloaded files end up on mount 1 while other files end up on mount 2? Are you looking to have the /cloud overlay both remotes?

It certain can be done but, depending on how it will be used, may require changes to every Docker container's YAML (which you can do by copying them to the user-owned container in 801-899 but this really means you're not using that much of OmniStream anymore). Of course, if it's just a read-only mount that ends up in ~/OmniStream/mnt, then it may be possible to set this up with minimal interference to everything else.

[rkdavies]

1 mount would be 100% read-only. No more updates/writes. the 2nd would be Read/Write.

[kelinger]

Could you mount the second cloud from the host instead of OmniStream? If OmniStream is using Google and you're looking to also mount Dropbox (for example), then you'd use a script on the host to mount Dropbox but maybe put it somewhere other than ~/OmniStream/mnt so that any of the cleanup scripts don't accidentally unmount or touch files on Dropbox.

[jaseelye]

So, I created another container and attached it to /home/user/mnt so just outside Omnistream. Looked to be working well but changes aren't being synced back to the Google Drive. Any thoughts on why the normal Omnimount works but this doesn't?

[jaseelye]

If I copy files directly into the Gdrive folder it syncs right away. However, if I use the Cloud folder, the file stays in the Uploadcache folder and never syncs.

[kelinger]

This sounds like an issue with the TurboSync process. Just to confirm:

if you type upsize you should see everything queued for upload to the cloud (you don't need to share the output with us). Is that the case?

Can you check ~/OmniStream/logs/turbosync.log? This should have the results of the last sync attempt (assuming you're seeing files after the upsize command).

[jaseelye]

Turbosync seems to be the issue. Sync works if I manually force Turbosync. Nothing is showing up in the queue but when I manually trigger Turbosync, it works.

[jaseelye]

I was looking at the wrong queue. Since I now have 2 Omnimount containers, I can't use the built in upsize command. I have to do the Turbosync check in the container. Anyway, the files do show up in the queue but do not transfer without forcing the Turbosync. This is happening for both Omnimount containers. I typically force v1.5 but have tried "latest" as well. Any suggestions?

[kelinger]

@jaseelye - Check the cron job and make sure its calling the right TurboSync.

[jaseelye]

Original Cronjob (At minute 0, 15, 30, and 45): 0,15,30,45 * * * * /home/user/OmniStream/bin/turbosync > /home/user/OmniStream/logs/turbosync.log 2>&1

New Cronjob (every 5 min): */5 * * * * /home/user/OmniStream/bin/turbosync > /home/user/OmniStream/logs/turbosync.log 2>&1

I modified Turbosync to include new container:

# Run turbosync inside omnimount
docker exec -i omnimount /root/turbosync
docker exec -i omnimount2 /root/turbosync

This seems to be working well now. Thank you.

@kelinger Do you think 5min is too much? Can Turbosync be triggered by files in the uploadcache directory vs the cronjob? I don't remember having this issue before.

[kelinger]

TurboSync is smart enough to not re-queue files that are already queued and it will also not run multiple instances of itself (for example, if you run TurboSync and it queues 100 files and then you run it again while it's still queuing files, it will exit and run the next time cron calls it).

The TurboSync in bin controls this so if you modified it to trigger both when called, it is probably OK. However, I'm not sure of the other variables though. Earlier you said the second mount was going to be read only but now it sounds like you want to sync files up to both clouds. Is that what you intended?

[jaseelye]

Thank you and great point. The plan IS to have the older mount go read-only. I was just trying to get them both working when I found this issue and just forgot about that part. I can revert back to the original TurboSync once I rename the container host back to omnimount.

Still not sure why TurboSync isn't happening automatically but changing the schedule to every 5min is working. Was I mistaken before? Does TurboSync only get triggered by the Cron Job?