search

intel / vck

Volume Controller for Kubernetes
https://ai.intel.com/kubernetes-volume-controller-kvc-data-management-tailored-for-machine-learning-workloads-in-kubernetes/
Apache License 2.0
67 stars 17 forks source link

Sync back with S3 data source. #64

Open Tomcli opened 6 years ago

Tomcli commented 6 years ago

If I have my data downloaded/sync from S3 to hostPath, is there a way for me to sync back my changes in the hostPath back to my S3 storage?

balajismaniam commented 6 years ago

@Tomcli We had discussed about this during our offline meeting. Did you have a chance to try out VCK?

Tomcli commented 6 years ago

@balajismaniam Yes, we tried VCK on our Cluster and it works great. However, when we want to sync back some of our modified data in HostPath back to S3, we have to do it manually on the node. So we would love to see if VCK can provide some help on data upload to S3. Also, I'm wondering can the volumemanager create a cronjob to periodically sync with our data on S3.

balajismaniam commented 6 years ago

Thanks for feedback @Tomcli. For the data upload to S3, as a first cut is it fine if we add the following assumption:

Do you think the above assumptions will work for your use case?

Tomcli commented 6 years ago

Thank you @balajismaniam, I think that should work for us as a first cut.

balajismaniam commented 6 years ago

@Tomcli We added this feature as discussed above in #72.