search

spotty-cloud / spotty

Training deep learning models on AWS and GCP instances
https://spotty.cloud
MIT License
492 stars 43 forks source link

Throw error in spotty download when the file does not exist in the instance #40

Closed jmoralesmat closed 5 years ago

jmoralesmat commented 5 years ago

When downloading a file from the instance to local by using spotty download -f file ./, spotty won't throw any warning or error if the file does not exist in the instance. It would be nice to see this information.

apls777 commented 5 years ago

Under the hood, Spotty is using standard "aws s3 sync" command. The spotty download -f <FILTER> [<FILTER> ...] command is an analogue to aws s3 sync ... --exclude '*' --include <FILTER> [--include <FILTER> ...]. This command won't tell you if the file doesn't exist either, because it's considering your file as an "include" filter. In general, filters may contain wildcards, and everything that matches those filters will be synced to your machine. So, if you don't see a message that a particular file was downloaded to your machine then the file either doesn't exist on a running instance or exactly matches the file on your local machine.

Please, see Use of Exclude and Include Filters for details.

jmoralesmat commented 5 years ago

Good to know, thank you!