search

cylondata / twister2

A composable framework for fast and scalable data analytics
https://twister2.org
Apache License 2.0
57 stars 32 forks source link

Job package uploading through web server pods in Kubernetes #820

Closed ahmet-uyar closed 4 years ago

ahmet-uyar commented 4 years ago

I implemented job package uploading through web server pods in Kubernetes. Many instances of web server pods can run in clusters to handle large workloads. I designed a StatefulSet that runs nginx web servers as uploader servers. I tested and added documentation. I also added configuration parameters.

I think it should be ready to be merged.

supunkamburugamuve commented 4 years ago

@ahmet-uyar there seems to be a conflict

ahmet-uyar commented 4 years ago

@ahmet-uyar there seems to be a conflict

@supunkamburugamuve I think there is no conflict. I am not sure why it says so.

chathurawidanage commented 4 years ago

@ahmet-uyar your branch ahmet/k8s-fixes is 34 commits behind master. If you try to merge master to ahmet/k8s-fixes, this will get automatically resolved. I think the problem is twister2/resource-scheduler/src/java/edu/iu/dsc/tws/rsched/schedulers/k8s/worker/K8sWorkerController.java has changes in master branch within last 9 days. Your branch is out of sync with master for more than 9 days.

ahmet-uyar commented 4 years ago

@chathurawidanage I merged the branch to master on my local machine and resolved the conflicts manually. Your suggestion is better I think. thanks.