support persistent training using aws spot instance without starcluster and EBS volume. still need some more tests. The idea is to use S3 as persistent storage. Download all the required files from S3 to local instance and upload the trained network to S3. In the web console, there is a type of persistent spot instance which can keep spot instance going. What we need is setting the UserData appropriately. This will facilitate training a lot.
UserData
appropriately. This will facilitate training a lot.