Persistent and Queryable - Githubissues

kube-HPC / hkube

🐟 High Performance Computing over Kubernetes - Core Repo 🎣

http://hkube.io

MIT License

305 stars 20 forks source link

Persistent and Queryable #1009

Closed nassiharel closed 2 years ago

nassiharel commented 3 years ago

In order to allow HKube to be more data queryable and data persistent. We will move some of our objects from Storage/Etcd to MongoDB.

Etcd Data

[ ] algorithms
- [x] builds
- [x] debug
- [ ] executions
- [ ] queue
- [ ] requirements
- [x] store
- [x] versions
[ ] discovery
[ ] drivers
[ ] events
[x] executions
- [x] stored
- [x] running
[ ] jobs
- [x] results
- [x] status
- [ ] tasks
[ ] pipeline drivers
- [ ] queue
- [x] store
- [ ] requirements
[x] pipelines
[x] webhooks
[ ] workers
[x] experiment
[x] tensorboards
[x] triggers
- [x] tree
[ ] streaming
- [ ] statistics

Object Storage

[x] Pipeline README
[x] Algorithm README

Redis Data

[x] Pipeline Graph
[ ] Algorithm Queue Persistency
[ ] Logs

TODO:

[x] enable queries in API server.
[x] need to adapt cleaners.
[x] sync all storage/etcd data to Mongo

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.