Automate deployment of viya4-deployment on top of viya4-iac-azure

[demorphica]

hello, we use Jenkins for our DevOps We will use terraform to deploy infra using https://github.com/sassoftware/viya4-iac-azure/ on Azure After this is complete, Jenkins should be able to use the tfstate outputs & some additional configuration file to kickoff the deployment of viya4-deployment on the cluster

How can we create a configuration file based deployment strategy, is there a script we can use to launch the install ?

[thpang]

That repo is not compatible with the viya4-deployment code base in terms of the tfstate file. If you need to stand up an Azure cluster and have it compatible with this repo you'd need to start here: https://github.com/sassoftware/viya4-iac-azure If you're looking for sample files for this repo to base your scripts on those are located here.

[demorphica]

apologies i had meant to reference the same repo you mentioned - edited my message

i saw the examples you referenced, for Jenkins we would need to define a job, which would need eg. a bash script or set of kubectl commands that Jenkins can execute on the cluster with the tfstate & additional configuration

do we have a reference/example of the install commands to help define the install script for the Jenkins job

[thpang]

Yes, in Jenkins you would have to run the scripts just like a user with the same privileges, etc. No rocket science here. The scripts will work in any environment you create with the correct permissions and tooling.

[demorphica]

Thanks @thpang we know how to define Jenkins jobs & get them all the permissions & file they require. This will be stored in a git deployment repo linked to the job & a webhook commit to a branch of this repo would cause Jenkins to run the job, which would retrieve the configuration from the repo & configure & launch the deployment

However we are not experts on the installation process for SAS VIYA. This information seems fragmented across this documentation https://go.documentation.sas.com/doc/en/itopscdc/v_024/itopswlcm/home.htm?fromDefault= & it looks like someone needs to collate this into an install script after careful reading of this documentation. This is a time consuming task & would require us to become experts in your install process

In order to move forward faster, could we please have some reference install script or a documented end to end sequence of commands that we can customize & adapt into an install script. This would help us get started

[thpang]

If you look at the docs for this repo you should do the following:

1. Run the baseline,install item first to install the needed baseline items.
2. Next run viya,install if there are not modifications you need to perform with our order, i.e your order will deploy and run without any customization. If there is customerization needed move on below
3. If you need customization, set the DEPLOY flag in your ansible vars file to false. Make the needed changes, i.e. copy items from the examples directory as directed into the site-config directory created. Make the adjustments.
4. Now that those are complete, set the DEPLOY flag back to true and re-run the viya,install step again.

If there are other items you need to adjust for you'll need to code that up. There are no scripts given the large number of variants and combinations of the SAS code base. You'd have to work with another team setting up your reference install script.

To be honest I am not really sure what you're looking for there. The code base here will cut hours off of your deployment.

Best of luck.

[demorphica]

Thanks for your response @thpang,

The core problem we are trying to solve is to deploy SAS VIYA in a configurable, end to end automated manner which does not require a human being to operate various systems & run commands one by one somewhere

As a devops team we deploy many software stacks on bare metal, linux, docker or kubernetes. As you mentioned, there are a large number of variants for any software configuration. We collect the configurable parameters into a single configuration file & create install scripts which read & use this configuration. This way we can customize the configuration file & deploy a variant in an end to end automatable manner.

The requester of the deployment commits their customized configuration file to the deployment repo, this file has the variant of the configuration they require, this causes the Jenkins job to launch, which runs the install script. The install script picks up the configuration defined by the requester & deploys the software without requiring further human intervention

In order to solve the problem for SAS VIYA deployment, https://github.com/sassoftware/viya4-deployment/tree/main/examples has examples of the ansible configuration, however there is not a readme file on this page that explains these files, what they are for, & the end to end sequence of ansible & kubectl commands to cause the deployment to happen.

Could this be made available please

[thpang]

Ah, the file for the content of those files is here Those example files, simply have the relevant items needed for a deployment based on if you're using the Viya 4 IAC code bases to help, i.e. you have a tfstate file, or not. If you have questions after reading through the docs there let me know.

[demorphica]

thanks @thpang this solves the problem of the variants & the configuration file

the only thing that we can't find is a single definitive listing of all ansible & kubernetes command sequence that must be carried out.

from what we see, you do have comprehensive documentation meant for a human admin user here https://go.documentation.sas.com/doc/en/itopscdc/v_024/itopswlcm/home.htm?fromDefault= , however it will take us quite some time to refer to it & build out the automated install sequence

also wanted to say that unlike many other vendors, you have spent considerable time & effort & done an awesome job in building out the infra as code & also containerizing the s/w deployment - the only thing thats missing is a single line install sequence someone can use or a script that does it

If you could spend some time on this area we are sure it will help us & many others like us save a lot of time in doing repeatable installs of SAS VIYA

[thpang]

HI @demorphica, glad the repo is helping. In order to do everything you need you can do the following:

1. Clone the viya4-iac-azure repo
2. From there either run terraform or build the Dockerfile and use docker to stand up your infrastructure.
3. Once that has been completed, clone the viya4-deployment repo
4. From there use the IAC Sample file and add modifications you need.
5. Once that has been adjusted run ansible-playbook or build the Dockerfile and use docker to run the baseline,install part of the deployment. This installs cert-manager, ingress-nginx, nfs-subdir-external-provisioner, metrics-server if needed, and cluster-autoscaler if needed. This will give your a kubernetes env. that's ready for Viya 4.
6. Next you run the same command in step 5 but now install viya. The task and actions are viya4,install

Now you'll have a cluster with Viya 4 installed and running. The following docs show you how to proceed in this repo from step 3 from above.

To run ansible look here To run docker look here

This info along with the docs should help move you along.

Thanks for the kind words and glad these and the IAC repos are helping.

[thpang]

Closing this one.