Add Raft partition group to Helm chart

[kuujo]

Atomix 4 and the Atomix k8s controller are just about ready for use in onos-config. There are a few ways we can go about setting up the cluster, but at least initially I think we should just add a Raft PartitionGroup to the onos-config Helm chart and inject the Atomix controller service into the onos-config deployment.

This is just another intermediate step. Eventually we’ll want to deploy shared PartitionGroups for all the services.

[Andrea-Campanella]

@kuujo would there be a way, as it is in current ONOS to have a "single instance" deployment out of k8s for development purposes ? keen to the onos-config command we have at the moment

[kuujo]

I think the short answer is no. We're creating a micro-services architecture, so this is a bridge we'll have to cross eventually (as soon as we have more than one service), and I think the sooner we cross it the better. As David Bainbridge suggested recently, there's a significant benefit to developing and deploying in the same environment, and I think recent production experience confirms that. The transition will be painful initially, but the benefits will be great. And Kubernetes is one of the largest open source communities on Earth, so there are obviously a lot of tools at our disposal.

I think the sooner we move to Kubernetes the better, and the development environment in that case is Minikube (or microk8s for Linux). It still amounts to a one line deployment (we just create manifests for a local testing configuration, e.g. kubectl apply -f deployments/dev/onos-config-3-simulators.yaml). Without making this transition, and with a portion of the developers still developing off k8s, we're bound to continue introducing bugs into the deployment.

I did try to explore whether this would be practical, though. I think it's important that we ensure the stores we're developing with are running the same code as the stores we're deploying, otherwise there's significant overhead to keeping two store implementations in sync, and bugs are bound to be introduced by changes that succeed with one store and fail with the other. So considering that, the closest solution is to essentially either use Docker Compose or do what Docker Compose does: link an Atomix controller, Raft, and onos-config through a Docker bridge network. But TBH that solution is only moderately simpler than just deploying locally in Minikube, but it still requires additional work/maintenance for local development (implementing an Atomix controller for Docker Compose) and comes with the added baggage of introducing potential inconsistencies between development and deployment environments that may make development/testing less reliable.

Still, I'm open to suggestions. But my strong opinion based on what we've seen in the past is that the benefit of developing and deploying in the same environment is well worth the initial cost of pushing local development into Minikube. I would just encourage everyone to make suggestions on how their lives can be made easier with new tools (like the one-line local instance I suggested).

[tomikazi]

I fully concur and would suggest we don’t exert energy in making it simple to do self-contained/standalone server run. That can be left as an exercise for the reader.

Thomas

On Jun 12, 2019, at 18:17, Jordan Halterman notifications@github.com wrote:

I think the short answer is no. We're creating a micro-services architecture, so this is a bridge we'll have to cross eventually (as soon as we have more than one service), and I think the sooner we cross it the better. As David Bainbridge suggested recently, there's a significant benefit to developing and deploying in the same environment, and I think recent production experience confirms that. The transition will be painful initially, but the benefits will be great. And Kubernetes is one of the largest open source communities on Earth, so there are obviously a lot of tools at our disposal.

I think the sooner we move to Kubernetes the better, and the development environment in that case is Minikube (or microk8s https://microk8s.io/ for Linux). It still amounts to a one line deployment (we just create manifests for a local testing configuration, e.g. kubectl apply -f deployments/dev/onos-config-3-simulators.yaml). Without making this transition, and with a portion of the developers still developing off k8s, we're bound to continue introducing bugs into the deployment.

I did try to explore whether this would be practical, though. I think it's important that we ensure the stores we're developing with are running the same code as the stores we're deploying, otherwise there's significant overhead to keeping two store implementations in sync, and bugs are bound to be introduced by changes that succeed with one store and fail with the other. So considering that, the closest solution is to essentially either use Docker Compose or do what Docker Compose does: link an Atomix controller, Raft, and onos-config through a Docker bridge network. But TBH that solution is only moderately simpler than just deploying locally in Minikube, but it still requires additional work/maintenance for local development (implementing an Atomix controller for Docker Compose) and comes with the added baggage of introducing potential inconsistencies between development and deployment environments that may make development/testing less reliable.

Still, I'm open to suggestions. But my strong opinion based on what we've seen in the past is that the benefit of developing and deploying in the same environment is well worth the initial cost of pushing local development into Minikube. I would just encourage everyone to make suggestions on how their lives can be made easier with new tools (like the one-line local instance I suggested).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/onosproject/onos-config/issues/310?email_source=notifications&email_token=ACM2TNJVVFGLVZ3FVER333TP2GNZ7A5CNFSM4HU34NR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXSHKRI#issuecomment-501511493, or mute the thread https://github.com/notifications/unsubscribe-auth/ACM2TNIZTOJT6CZH6Z6NUE3P2GNZ7ANCNFSM4HU34NRQ.

[Andrea-Campanella]

@kuujo thanks for the detailed response. I see your point, and remember David's thus I concur that it's better to deploy everything in k8s (whatever flavour).