att-comdev / halcyon-kubernetes

Ansible playbooks for a kubadm-based kubernetes deployment, on supporting any cloud and any kubeadm-enabled OS.

Apache License 2.0

35 stars 22 forks source link

These commits add support for ceph #23

Closed intlabs closed 7 years ago

intlabs commented 7 years ago

These commits provide support for ceph on the hosts running k8s:

Set up resolv.conf on the hosts to access ceph
Install ceph on the hosts to enable mounting of RBD devices

intlabs commented 7 years ago

Please review these changes prior to merging - I have not validated ubuntu based installs.

v1k0d3n commented 7 years ago

@intlabs Pete, let me try them tonight with Ubuntu and when they all check out I'll add them in. We're planning on deploying Ceph with a "new" StatefulSet (really Kubernetes old petsets). this would work together either way? the ceph stuff is coming very soon....(keep an eye on our Helm repo).

intlabs commented 7 years ago

@v1k0d3n Yeah - this should just enable the kubelet to mount the RBD devices, no matter how they are exposed to the host. It should be totally independent of how ceph is deployed and probably pulls in much more than is strictly necessary. I'll keep an eye on the aic-helm repo for sure - I'm currently working on ceph for kolla-k8s's helm implementation.

v1k0d3n commented 7 years ago

@intlabs made some comments. i'd like to clear them up sometime tomorrow in case anyone wants to attempt to use them with Ubuntu and gets a failure. we should have CI for this project soon (which will help a lot). I have @larryrensing and someone else currently looking at it.

v1k0d3n commented 7 years ago

6 mentions this for reference.

intlabs commented 7 years ago

@v1k0d3n It would be great if we could go over kolla's requirements at some point soon, so that if possible I could retire my fork and point everyone to here. Cheers for all your help. :)

v1k0d3n commented 7 years ago

@intlabs i agree. been trying to sync up with kolla, but it's hard for me to watch IRC all day and still get things done. we were hoping we could show what we're doing for our PoC, and people would take as needed. we're pretty open about our intent; no hiding anything with full intention of merging everything in (if we're allowed to).

can we sync up and find out what's needed from halcyon so we can get them merged into one place? are your changes able to be merged in as is? i noticed you have helm canary and some other useful bits like dns and ceph. it doesn't seem like there is an insurmountable difference between them yet, but i would like them working for both centos and ubuntu (we're a heavy ubuntu shop).