search

PaddleFlow / paddle-operator

Elastic Deep Learning Training based on Kubernetes by Leveraging EDL and Volcano
Apache License 2.0
31 stars 15 forks source link

Add sample set cache component in paddle-operator #66

Closed xiaolao closed 3 years ago

xiaolao commented 3 years ago
  1. Add custom resource SampleSet and its controller. SampleSet defines a set of data needed for model training, it can be divided into multiple partitions cached in the kubernetes cluster.
  2. Add custom resource SampleJob and its controller. SampleJob defines some data management commands to facilitate users to manage cached data in the cluster.
  3. Add runtime server，it accept requests from the sampleset controller, and perform data management operations on the cache node.