Sync several instances from one command?

[turian]

I'd love to sync several instances from one command, and not run multiple times (redo'ing the S3 sync). Thank you!

[apls777]

@turian Can you, please, tell me more about your specific use case? Do you only want to sync your code or you also want to run the same script on several instances? I remember I had a similar problem when needed to run several experiments in parallel on different machines - same code, but different hyperparameters.

[turian]

Same code, different hyperparameters. I have a local.py which is not synced (based upon local.py.tmpl which is sync'ed) that controls the hyperparams for this machine

spotty sync is so slow I'm resorting to doing a git pull instead of spotty sync.

[turian]

With this in mind, is there also a way to put my id_rsa.pub onto the spotty machines? So I can easily git clone my repo using my SSH key?

Thank you, spotty is a wonderful tool.

[apls777]

spotty sync is so slow I'm resorting to doing a git pull instead of spotty sync.

Is it like really really slow or just slow? If you have thousands of files in your project directory (dataset files, for example), then AWS CLI is super slow, even if those files under the project.syncFilters[].exclude. Is it your case?

With this in mind, is there also a way to put my id_rsa.pub onto the spotty machines? So I can easily git clone my repo using my SSH key?

I think you mean a private key id_rsa, not a public id_rsa.pub. Well, you can do it, but it's not very secure as your key will also be saved in a Spotty S3 bucket. But if it's fine, you can do the following:

• copy your key to the project directory
• use containers[].commands parameter to copy your key to /root/.ssh/id_rsa (don't forget to change permissions on the file to 600 after) or add the key to ssh-agent.

FYI: there is no need to restart the instance to test it, you can restart only the container using the -C flag: spotty start -C.

[apls777]

@turian Just checked the synching time, it takes 10-20 seconds for me at the moment. Is it much slower for you?

[turian]

Now it's about 20 seconds. I mean id_rsa.pub, I thought that is what you use to gain github access. You are right that it will still be in S3. The S3 buckets are secured by default tho?

I am curious if I can disable sync'ing

I have moved to a workflow when I just git push and pull on the local macine

[apls777]

I mean id_rsa.pub, I thought that is what you use to gain github access.

To work with the GitHub account via SSH, you're using a private key (id_rsa, by default). id_rsa.pub is a public key that you imported to your GitHub account to be able to use your private key.

The S3 buckets are secured by default tho?

Yes, a Spotty bucket is private. So, it's fine if you're using your personal AWS account and nobody else has access to it.

I am curious if I can disable sync'ing

You cannot disable synching for the spotty start command, but you can ignore all the files by adding '*' under the project.syncFilters[].exclude parameter. You can disable synching for the spotty run command by using the --no-sync flag.

I have moved to a workflow when I just git push and pull on the local machine

Can you, please, clarify your main problem:

• Is the syncing too slow in general? I mean, if you would need to sync only one instance, would be 20 seconds okay for you? Or do you sync files very often? If that's the case, it, probably, could be solved by replacing aws s3 sync with rsync to sync your local files with the instance directly, should be faster.
• Or the main issue is that you have a group of instances and it's too slow to run several spotty sync commands one-by-one? So if spotty sync would sync a group of instances in parallel for 20 seconds, that's would be okay for you?

[turian]

I think the main issue is that I have many data files that I am excluding. So it's the aws cli here.

Parallel sync is a nice-to-have, but learning about the many data files issue is very useful.