Closed turian closed 3 years ago
@turian Can you, please, tell me more about your specific use case? Do you only want to sync your code or you also want to run the same script on several instances? I remember I had a similar problem when needed to run several experiments in parallel on different machines - same code, but different hyperparameters.
Same code, different hyperparameters. I have a local.py which is not synced (based upon local.py.tmpl which is sync'ed) that controls the hyperparams for this machine
spotty sync is so slow I'm resorting to doing a git pull instead of spotty sync.
With this in mind, is there also a way to put my id_rsa.pub onto the spotty machines? So I can easily git clone my repo using my SSH key?
Thank you, spotty is a wonderful tool.
spotty sync is so slow I'm resorting to doing a git pull instead of spotty sync.
Is it like really really slow or just slow? If you have thousands of files in your project directory (dataset files, for example), then AWS CLI is super slow, even if those files under the project.syncFilters[].exclude
. Is it your case?
With this in mind, is there also a way to put my id_rsa.pub onto the spotty machines? So I can easily git clone my repo using my SSH key?
I think you mean a private key id_rsa
, not a public id_rsa.pub
. Well, you can do it, but it's not very secure as your key will also be saved in a Spotty S3 bucket. But if it's fine, you can do the following:
containers[].commands
parameter to copy your key to /root/.ssh/id_rsa
(don't forget to change permissions on the file to 600 after) or add the key to ssh-agent.FYI: there is no need to restart the instance to test it, you can restart only the container using the -C
flag: spotty start -C
.
@turian Just checked the synching time, it takes 10-20 seconds for me at the moment. Is it much slower for you?
Now it's about 20 seconds. I mean id_rsa.pub, I thought that is what you use to gain github access. You are right that it will still be in S3. The S3 buckets are secured by default tho?
I am curious if I can disable sync'ing
I have moved to a workflow when I just git push and pull on the local macine
I mean id_rsa.pub, I thought that is what you use to gain github access.
To work with the GitHub account via SSH, you're using a private key (id_rsa
, by default). id_rsa.pub
is a public key that you imported to your GitHub account to be able to use your private key.
The S3 buckets are secured by default tho?
Yes, a Spotty bucket is private. So, it's fine if you're using your personal AWS account and nobody else has access to it.
I am curious if I can disable sync'ing
You cannot disable synching for the spotty start
command, but you can ignore all the files by adding '*'
under the project.syncFilters[].exclude
parameter. You can disable synching for the spotty run
command by using the --no-sync
flag.
I have moved to a workflow when I just git push and pull on the local machine
Can you, please, clarify your main problem:
aws s3 sync
with rsync
to sync your local files with the instance directly, should be faster.spotty sync
commands one-by-one? So if spotty sync
would sync a group of instances in parallel for 20 seconds, that's would be okay for you?I think the main issue is that I have many data files that I am excluding. So it's the aws cli here.
Parallel sync is a nice-to-have, but learning about the many data files issue is very useful.
I'd love to sync several instances from one command, and not run multiple times (redo'ing the S3 sync). Thank you!