Closed BolaNasr closed 5 years ago
at the upper part of this picture It contains an array of shared reverse proxies to follow this steps should implement this via traefik, coredns & etcd Now we will create 0-robot template to install these components ( using flist)
i think this is the first basic phase
@BolaNasr let's start with the core template needed for this setup. So trakeik, coredns and etcd.
Each of these template needs to be "node" template, which means they are meant to be directly installed on a node. (example of such template are zerodb, namespaces, vm...)
Before you start writing anything, please write down here in the issue how you see the schema and the actions of each templates.
Then to create the flist, the way to go is to write a prefab module that build/install the programs and then create an flist out of it. So we have something repeatable to build these flists.
Once we have all this, ping me so we can review the spec of the template and continue with the implementation
schema of etcd :
struct Schema {
listen-peer-urls @1: List(Text); //List of URLs to listen on for peer traffic default: "http://localhost:2380"
listen-client-urls @2 List(Text); //List of URLs to listen on for client traffic default: "http://localhost:2379"
initial-advertise-peer-urls @3: List(Text); //List of this member's peer URLs to advertise to the rest of the cluster. default: "http://localhost:2380"
advertise-client-urls @4: List(Text); //List of this member's client URLs to advertise to the rest of the cluster default: "http://localhost:2379"
}
actions
install // install etcd using flist (https://hub.grid.tf/bola_nasr_1/etcd-3.3.4.flist.md)
uninstall // uninstall etcd
start //start etcd
stop //stop it
schema of traefik:
struct Schema {
user_web @0: Text // your username of traefik web
pass_web @1: Text // your password of traefik web
etcd_server_endpoint @2 : Text //ip of Etcd
etcd_watch @3 : Bool=True //watch changes in Traefik web
}
actions
install // install Traefik using Flist ( https://hub.gig.tech/delandtj/traefik.flist.md )
uninstall // uninstall Traefik
start //start Traefik using config file that created
stop //stop Traefik service
schema of coredns:
struct Schema {
zone @0: Text // default "."
password @1: Text // default none
etcd_server_point @2 :Text //ip of Etcd
upsteram @3 : Text //by default 8.8.8.8:53 8.8.4.4:53
}
actions
install // install coredns using flist file (https://hub.grid.tf/bola_nasr_1/coredns.flist.md)
uninstall //uninstall coredns
start //start coredns server using config file
stop // stop server
@zaibon i think these are the schemas and the actions of each templates as far as i understood
struct Schema {
listen-peer-urls @1: List(Text); //List of URLs to listen on for peer traffic default: "http://localhost:2380"
listen-client-urls @2 List(Text); //List of URLs to listen on for client traffic default: "http://localhost:2379"
initial-advertise-peer-urls @3: List(Text); //List of this member's peer URLs to advertise to the rest of the cluster. default: "http://localhost:2380"
advertise-client-urls @4: List(Text); //List of this member's client URLs to advertise to the rest of the cluster default: "http://localhost:2379"
}
All these url, are they going to be filled by the template ? Or is it something the user is supposed to provide ? Cause in reality what happens is that you create a container, you ask zero-os on which port you can create the port forward so only after you deployed you can actually know the urls where you can reach the etcd.
struct Schema {
user_web @0: Text // your username of traefik web
pass_web @1: Text // your password of traefik web
etcd_server_endpoint @2 : Text //ip of Etcd
etcd_watch @3 : Bool=True //watch changes in Traefik web
}
etcd_server_endpoint @2 : Text //ip of Etcd We should not use the ip directly, but instead having a link to the etcd service, or the name of an etcd client service. If you use the ip directly, there is no way to update that ip after the service is installed. So no self-healing possible in case the etcd change addresses.
struct Schema {
zone @0: Text // default "."
password @1: Text // default none
etcd_server_point @2 :Text //ip of Etcd
upsteram @3 : Text //by default 8.8.8.8:53 8.8.4.4:53
}
Same remark for the etcd ips
All these URLS it's something the user is supposed to provide
you are correct should add the name of an etcd client service and get Ip from it so we replace it to etcd_server_name
For ETCD service implementation, I used this demo as a reference to how an etcd in a cluster should be handled. According to the configuration in this demo, in all the urls attributes, the etcd points to its own url. I only managed to make that work when the etcd container has a zerotier nic, and I use the container zerotier ip along with the etcd port. If I use port forwarding and use the node's ip with the forwarded port, it doesn't work. So is it a right assumption to assume that we will always use zerotier? and should I add validation accordingly?
Accordingly this is the final schema of the etcd service:
struct Schema {
nics @0 :List(Nic); # Configuration of the attached nics to the etcd container
ztIdentity @1 :Text; # ztidentity of the container running 0-etcd
token @2 :Text;
cluster @3 :List(Member);
struct Member {
name @0 :Text;
address @1 :Text;
}
struct Nic {
type @0 :NicType;
id @1 :Text;
config @2 :NicConfig;
name @3 :Text;
ztClient @4 :Text;
hwaddr @5 :Text;
}
struct NicConfig {
dhcp @0 :Bool;
cidr @1 :Text;
gateway @2 :Text;
dns @3 :List(Text);
}
enum NicType {
default @0;
zerotier @1;
vlan @2;
vxlan @3;
bridge @4;
}
}
I assumed there would be a service that manages all etcds that belong to the same cluster, this service would use the actions update_cluster
in the etcd service to update the service with info about all the other etcds in the same cluster.
The following are the actions available in the etcd service:
install
: creates the etcd container and makes sure it is assigned a zerotier ip but doesn't start the etcd process. That way if you want to create a cluster, you can install all the relevant etcd services then update them all with all their urls before starting.start
: start etcd with the right cluster configuration.stop
: stop the etcd process.uninstall
: stop the etcd process and remove the container.connection_info
: returns both the client and peer urlupdate_cluster
: updates the cluster value in self.data. This should be used to update the service with info about all the members of this cluster.insert_record
: inserts a key/value in etcdget_record
: retrieves a value of a key in etcd@zaibon Check my last comment, please.
@despiegk please have a look
1- creating vm on zero-os node and install apache on it. 2 - create Etcd container :
3- create Traefik container :
4- create CoreDns container :
5- load keys and values on etcd:
create 0-robot templates to install these components