Open Discussion. User experience to install the distribution

[angelbarrera92]

Hi team.

Please, take a look to this repository. It contains:

1. Furyfile.yml with static versions of core-modules that has to be installed on every Fury Cluster.
2. kustomization.yaml file with the features that has to be installed on every Fury Cluster.
3. An E2E pipeline test (smoke tests) that tests every module is Running in a vanilla Kubernetes Cluster (provided by the e2e-testing environment project).

The idea is to have both files versioned in a repository (could be this one, renaming it). Then tag the master branch with a new release (as we are doing with all OSS software). This release will be the "distribution" version (as pack of core modules).

So. Let's say we've got:

Furyfile.yml:

versions:
  networking: v1.0.0
  monitoring: v1.3.0
  logging: v1.2.1
  ingress: v1.4.1
  dr: v1.2.0-rc3

bases:
  - name: networking/calico
  - name: monitoring/prometheus-operator
  - name: monitoring/prometheus-operated
  - name: monitoring/grafana
  - name: monitoring/goldpinger
  - name: monitoring/kubeadm-sm
  - name: monitoring/kube-state-metrics
  - name: monitoring/node-exporter
  - name: logging/elasticsearch-single
  - name: logging/cerebro
  - name: logging/curator
  - name: logging/fluentd
  - name: logging/kibana
  - name: ingress/cert-manager
  - name: ingress/nginx
  - name: ingress/forecastle
  - name: dr/velero

And kustomization.yaml

bases:
  # Networking
  - ./vendor/katalog/networking/calico
  # Monitoring
  - ./vendor/katalog/monitoring/prometheus-operator
  - ./vendor/katalog/monitoring/prometheus-operated
  - ./vendor/katalog/monitoring/grafana
  - ./vendor/katalog/monitoring/goldpinger
  - ./vendor/katalog/monitoring/kubeadm-sm
  - ./vendor/katalog/monitoring/kube-state-metrics
  - ./vendor/katalog/monitoring/node-exporter
  # Logging
  - ./vendor/katalog/logging/elasticsearch-single
  - ./vendor/katalog/logging/cerebro
  - ./vendor/katalog/logging/curator
  - ./vendor/katalog/logging/fluentd
  - ./vendor/katalog/logging/kibana
  # Ingress
  - ./vendor/katalog/ingress/cert-manager
  - ./vendor/katalog/ingress/nginx
  - ./vendor/katalog/ingress/forecastle
  # DR
  - ./vendor/katalog/dr/velero/velero-prem
  - ./vendor/katalog/dr/velero/velero-restic

And pipeline passed (smoke test testing that everything is running). We can create a v1.0.0 tag to this repository. Meaning that a new Fury Distribution Release is available.

Ok.

Hands ON!

What should be the user/devops/operator/delivery-team-member experience?

How someone can install this distribution?

Define the flow in the comments as proposal please.

Initial scenario: I've got a GKE/AKS/Kind/K8S Vanilla cluster and i want to install Fury Distribution on it.

[angelbarrera92]

Here are my 2cents:

1st Proposal

I've got a new/empty git repository. I know there is Github Repository with the Distribution. I go to the Releases page of the distribution and there are two files that i have to download.

So:

$ pwd
/home/user/git/my-awesome-empty-repo
$ curl -LOs https://github.com/sighupio/fury-distribution/v1.0.0/furyfile.yml
$ curl -LOs https://github.com/sighupio/fury-distribution/v1.0.0/kustomization.yml

Dummy commands

Then, following the well written docs I run:

$ pwd
/home/user/git/my-awesome-empty-repo
$ ls
furyfile.yml kustomization.yml
$ furyctl install -H
$ ls
furyfile.yml kustomization.yml vendor/
$ kustomize build
$ kustomize build | kubectl apply -f -

And after a couple of minutes everything is working.

[angelbarrera92]

2nd Proposal

I know there is Github Repository with the Distribution. I go to the Releases page of the distribution and there are a binary that i have to download.

So:

$ curl -LOs https://github.com/sighupio/fury-distribution/v1.0.0/fury

Dummy commands

Then, following the well written docs I run:

$ ls
fury
$ fury version
v1.0.0
$ fury manifests apply --kubeconfig=/tmp/kind-config

Dummy commands

And after a couple of minutes everything is working.

[angelbarrera92]

3rd Proposal

I know there is Github Repository with furyctl. I go to the Releases page of the fury client and there are a binary that i have to download.

So:

$ curl -LOs https://github.com/sighupio/furyctl/v1.0.0/furyctl

Dummy commands

Then, following the well written docs I run:

$ ls
furyctl
$ furyctl version
v1.0.0
$ furyctl manifests apply --kubeconfig=/tmp/kind-config

Dummy commands

And after a couple of minutes everything is working.

[MarioUhrik]

HI @angelbarrera92 . I like your 3rd proposal. There's some neat simplicity to it.

First, I think it might be nice if there's one designated "entrance gateway" GitHub project for people new to Fury to look at. Right now, there's about 10 publicly visible projects and imho it's not clear for newcomers to figure out what they should be looking at, and where to start.

I like the idea of furyctl (or whatever else) being the sole entrance gateway to everything related to Fury. Maybe the first command could be something like furyctl init, which will:

• set up the directory structure, modeling how our infra repositories are named and organized.
  • You can also throw in some extra stuff, like your above manifests/bases/kustomization.yaml.
  • I think it's good to enforce uniformity in a way like this
  • Shouldn't hurt to automate stuff that has no reason not to be automated
• create an example Furyfile.yml
  • put as much unique syntax elements in it as possible, so that people don't have to search docs for syntax and features. Furyfile.yml is the truly unique thing someone new to Fury won't know, so it's good to present it to the users on a golden plate.
  • you can also make it a standard "core" pack of modules (monitoring + logging) as you proposed above. This "core" pack of modules should be picked carefully, because it may not be obvious for a person new to Fury which module even exists, or which module do they want/need. Perhaps a good solution is to have every existing module present in Furyfile.yml, but comment out all that you don't consider part of the "core" pack.
• nothing if Furyfile.yml already exists (safety feature)

---

Initial scenario: I've got a GKE/AKS/Kind/K8S Vanilla cluster and i want to install Fury Distribution on it.

No matter the entry point, I think it could work to present Fury usage in 3 main segments:

• Terraform
• Ansible
• kustomize | kubectl apply

In this case, just make it clear enough to people that they can skip the first two and start with kustomize | kubectl apply, if they already have a working cluster that they want to keep.

---

Lastly, I'd like to emphasize that I think the

well written docs

part is extremely important. https://github.com/sighupio/fury-kubernetes-aws README.md is already turning out pretty well, but at this point, who knows if I could successfully follow it without prior experience :man_shrugging: . @bernardhalas maybe that's a good first task for new coworkers?

[jnardiello]

Here is my proposal:

curl -LOs https://github.com/sighupio/furyctl/v1.0.0/furyctl
furyctl init --version=1.blabla

This is going to do 3 things:

• Pull furyfile and vendor
• Pull kustomization.yml
• run furyctl install

So far, nothing has been yet applied to the actual cluster Then:

kustomize build | kubectl apply -f -

So, we keep furyctl purely client side as a vendoring tool, which imho is a plus. I mean, it’s basically #1 with a bit of refactoring of furyctl. And it leaves the door open to future changes

[bernardhalas]

@angelbarrera92 thank you for a well-prepared proposal. My 2 cents:

• In general, I'm in line with the comments from @MarioUhrik and @jnardiello
• I'd prefer to avoid using fury for applying the kubernetes manifests (which rules out options 2 and 3) - I don't see any reason atm why not to use kubectl for that
• I also agree that adding the task for initializing the environment with furyctl init would be of a benefit as it simplifies newcomers' installations

I'm actually inclined to modify @jnardiello 's proposal in the following way:

$ furyctl init --version=1.blabla
$ furyctl install
$ kustomize build | kubectl apply -f -

So that each of the furyctl tasks has a purpose and they don't overlap. What do you think?

[ralgozino]

Hi!

I like the simplicity of @jnardiello's proposal and personally don't mind that furyctl init does also the furyctl install part. In fact, I would deprecate furyctl install or rename it to something else like furyctl pull / download because it is a little misleading, it doesn't actually install anything, it just dowloads all the vendorized modules.

As @MarioUhrik pointed out, furyctl also downloads terraform modules and ansible manifests. I think that to deploy the fury distribution we won't be using those, but we will use that feature while setting up the cluster, right?

TLDR; +1 to Jacopo's proposal

[angelbarrera92]

I agree 100% with @ralgozino, the furyctl install command has to be renamed to furyctl vendor/download/pull.

Maybe while developing this new furyctl feature, we can mark as deprecated furyctl install command, pointing/copying the functionality to furyctl vendor/download/pull adding furyctl init command.

Something like:

furyctl v0.2.0 contains:

• furyctl init
• furyctl install (deprecation warning, use vendor)
• furyctl vendor

furyctl v0.3.0 contains:

• furyctl init
• furyctl vendor

Thanks for your time!

[jnardiello]

Right now furyctl is used only internally. I would not worry too much about the deprecation process @ralgozino @angelbarrera92 let's use whatever we think make sense (remove install) and push it upstream.

What @bernardhalas suggested does make sense, dividing the steps into

$ furyctl init --version=1.blabla
$ furyctl install
$ kustomize build | kubectl apply -f -

Separating init and install will give the opportunity to everyone to inspect the output of furyctl init, which is something I'd like to do when installing a production cluster.

Can we finalise this issue/decision if we all agree?

[angelbarrera92]

Right now furyctl is used only internally. I would not worry too much about the deprecation process @ralgozino @angelbarrera92 let's use whatever we think make sense (remove install) and push it upstream.

What @bernardhalas suggested does make sense, dividing the steps into

$ furyctl init --version=1.blabla
$ furyctl install
$ kustomize build | kubectl apply -f -

Separating init and install will give the opportunity to everyone to inspect the output of furyctl init, which is something I'd like to do when installing a production cluster.

Can we finalise this issue/decision if we all agree?

I have plans to use furyctl in the public madrid workshop. But i can changr the slides. @lnovara will review this issue later today. Tomorrow we will take a decission.

Thanks!!!

[lnovara]

@angelbarrera92 I like your first proposal using the furyctl init && furyctl vendor && kustomize build approach. Maybe for the init subcommand we can wrap something like cookiecutter to avoid reinventing the wheel too much. 😄

From my understanding, the furyctl init --version=<version> will bootstrap the repository and furyctl vendor will populate the vendor/ folder, right? From where furyagent init is gonna take the informations about which are the modules (and modules versions) composing the target distribution version?

[angelbarrera92]

Ok. I think we've got enough feedback to start working on it. I will create a trello card with all the information. As a recap:

$ ls .
.. .
$ furyctl init --version 1.0.0
$ ls .
kustomization.yaml Furyfile.yaml .. .
$ furyctl vendor
$ ls .
kustomization.yaml Furyfile.yaml vendor .. .
kustomize build | kubectl apply -f -

After the first implementation/release i will open a discussion to design the init flags. SPOILER:

• HA flag (for example will deploy elasticsearch triple)
• cloud=aws (for example will deploy velero with the aws provider)
• --enable-private-ingress (for example will deploy dual nginx).

But, for the first iteration we will implement just the --version flag.

Thanks for your time!

As soon as i've got a trello url, i will paste here

[phisco]

I like the furyctl init --version=1... && furyctl vendor && kustomize build approach too. As @lnovara was suggesting we should integrate cookie cutter or something similiar, but as of now I would focus just on the manifest part, not bothering about setting up also terraform/ansible. Let's keep the focus here on the distribution, we can always add flags to manage also the underlying installers afterwards.

[angelbarrera92]

I will try to implement it tomorrow morning, but my golang skils aren't as good as i wanted. But let's see what happens ;)

I think we can close the issue as there is an approved proposal ;)

Thanks for your time!