search

IBM-Cloud / observability-teams

Terraform Template for Monitoring with Teams
https://www.ibm.com/cloud/blog/terraform-template-for-monitoring-with-sysdig-teams
Other
3 stars 6 forks source link
go ibm-cloud ibm-cloud-monitoring ibm-cloud-solutions nodejs terraform

readme

IBM Cloud Monitoring, Logging and Activity Tracker with Teams

Use this template to:

What's in this repo

This repo has the following folder structure:

Costs

You must have a Pay-as-You-Go account in IBM Cloud™ to follow the steps in this repository to create resources. Since the costs for these resources will vary, use the Pricing Calculator to generate a cost estimate based on your projected usage.

Some of the services listed above offer a limited free tier, i.e. IBM Cloud Monitoring, IBM Cloud Log Analysis, and IBM Cloud Activity Trackler which you can use for testing and will work perfectly for our example application. Please note the implication of using the free services as some will be deleted automatically after 30 days.

If you deploy paid services, make sure to delete them when they are no longer required in order to not incur charges in your account.

Prerequisites

Before you start, make sure to have all the items completed below as the template requires them.

Determine which region you want to use. The value we will need is in the Region column and between the parentheses,i.e jp-tok, us-south, etc...

Invite users to your account

This template will configure monitoring Teams integrated with IAM, as the owner of the instance, you will have full visibility into all of the teams that you create, i.e. you can switch to them and see what other users would see if they were added to those teams. However, in order to get a more immersive experience, you need to have at minimum two additional users invited to the IBM Cloud account in which you will be creating these resources, follow the steps outlined below to invite users to your account:

  1. Go to Identity & Access > Users in the IBM Cloud console.
  2. Click the Invite Users button.
  3. Enter the e-mail address for two users and click on Invite

Getting started

  1. Clone this repository to your local computer.
  2. From a terminal window change to the observability-teams directory.
  3. Copy the config-template directory to a directory called config.

Build and push the container image for the applications

Build and push the Docker image to the IBM Cloud container registry.

  1. From a terminal window identify your IBM Cloud Container Registry hostname and save it for later use:

    a. Log in to the Container Registry service:

    ibmcloud cr login

    b. Obtain the hostname:

    ibmcloud cr region

  2. Pick one of your existing registry namespaces or create a new one. To list existing namespaces, use:

    ibmcloud cr namespaces

    To add a new namespace, use:

    ibmcloud cr namespace-add <your_registry_namespace>

Prepare the container image for the Go application

  1. From a terminal window, change to the samples/go-app directory.
  2. Build, tag (-t) to a image: 
    docker build -t <your_region_registry>/<your_registry_namespace>/metrics-go-app .
  3. Push the image to your container registry on IBM Cloud: 
    docker image push <your_region_registry>/<your_registry_namespace>/metrics-go-app

Prepare the container image for the Node.js application

  1. Open a terminal window and change to the samples/node-app directory.
  2. Build, tag (-t) to a image: 
    docker build -t <your_region_registry>/<your_registry_namespace>/metrics-node-app .
  3. Push the image to your container registry on IBM Cloud: 
    docker image push <your_region_registry>/<your_registry_namespace>/metrics-node-app

Deploy the application

  1. From a terminal window, change to the repository root directory.
  2. Enable tracing (optional): 
    export TF_LOG=TRACE
  3. Save all activities to a log file (optional): 
    export TF_LOG_PATH=./config/config.log
  4. Initialize the Terraform providers and modules: 
    terraform init

  5. Modify the config/config.tfvars to your own values.

    ibmcloud_api_key = "<your_ibm_cloud_api_key>"

resource_group = "<your_resource_group>"

activity_tracker_resource_group = "<resource_group_for_existing_activity_tracker_instance>"

cluster_resource_group = "<resource_group_for_kubernetes_cluster>"

region = "<your_region>"

resources_prefix = "<used_to_prefix_resources>"

cluster_id = "<your_kubernetes_cluster_id>"

go_image_repository = "<location_of_go_app_container_image>"

node_image_repository = "<location_of_node_app_container_image>"

team_go_members = ["<email_address_of_an_invited_user_to_your_ibm_cloud_account>"]

team_node_members = ["<email_address_of_an_invited_user_to_your_ibm_cloud_account>"]

activity_tracker_instance_name = "<name of existing activity tracker instance>"

activity_tracker_service_key = "Your service key can be generated or retrieved from the LogDNA web application. Navigate to Settings > Organization > API Keys"

logging_service_key = ""

Note: The template expects the Kubernetes cluster to have been created in a VPC Infrastructure.

Note: Both activity_tracker_resource_group and cluster_resource_group are empty values which will cause the template to use the value supplied for resource_group, if you require it specifiy a different value for each.

  1. Execute terraform plan by specifying the location of variable files, state and plan file: 
    terraform plan -var-file=config/config.tfvars -state=config/config.tfstate -out=config/config.plan

  2. Apply terraform plan by specifying the location of plan file:

    terraform apply -state-out=config/config.tfstate config/config.plan

    Note: If you plan on building for multiple environments or regions, you may want to maintain separate state files for each of these environments, you can use a different config directory for each environment or region. Another solution is to use Terraform workspaces which is discussed in our Plan, create and update deployment environments tutorial.

Create a service key for the Logging instance

  1. Log in to your IBM Cloud account from a browser.
  2. Navigate to Observability > Logging page.
  3. Click on the Open dashboard for your -logging instance.
  4. From the navigation panel, click on Settings (the gear icon) and expand Organization and click on API Keys.
  5. Click on Generate Service Key.
  6. Copy and paste the value provided for the service key into the logging_service_key variable of your config/config.tfvars file.

Create a service key for your existing Activity Tracker instance

  1. Log in to your IBM Cloud account from a browser.
  2. Navigate to Observability > Activity page.
  3. Click on the Open dashboard for the Activity Tracker instance in the region you are using.
  4. From the navigation panel, click on Settings (the gear icon) and expand Organization and click on API Keys.
  5. Click on Generate Service Key.
  6. Copy and paste the value provided for the service key into the activity_tracker_service_key variable of your config/config.tfvars file.

Run Terraform Plan and Apply

  1. Execute terraform plan by specifying the location of variable files, state and plan file: 
    terraform plan -var-file=config/config.tfvars -state=config/config.tfstate -out=config/config.plan
  2. Apply terraform plan by specifying the location of plan file: 
    terraform apply -state-out=config/config.tfstate config/config.plan

Verify metrics are visible to each designated team

  1. Log in to your IBM Cloud account from a browser.
  2. Navigate to Observability > Monitoring page.
  3. Click on the Open dashboard for your -monitoring instance.
  4. From the navigation panel, click on your initials, you should get a popup with a list of teams:
    • Monitor Operations
    • Team Go
    • Team Node
  5. Switch to each team and notice the different perspectives as seen by each team:
    • Monitor Operations: You can see metrics from all containers
    • Team Go: You can see metrics only from containers related to the Go application
    • Team Node: You can see metrics only from containers related to the Node.js application.

Verify metrics are visible to each account you added to a team

  1. Log in to your IBM Cloud account from a browser with an account that you added to the Go team.
  2. From the top nav bar, select to switch to the account this user was invited to.
  3. Navigate to Observability > Monitoring page.
  4. Click on the Open dashboard for your -monitoring instance.
  5. Notice the user only has access to the data related to the relevant application/containers.
  6. Repeat the above steps for the other accounts.

Verify logs and activities are visible to each account you added to a team

  1. Log in to your IBM Cloud account from a browser with an account that you added to the Go team.
  2. From the top nav bar, select to switch to the account this user was invited to.
  3. Navigate to Observability > Logging page.
  4. Click on the Open dashboard for your -logging instance.
  5. Content that is visible to you are specific to you only.
  6. Notice the user only has access to the data related to the Go application/containers.
  7. Repeat the above steps for your instance of Activity Tracker.
  8. Repeat the above steps for the other accounts.

Delete all resources

  1. Destroy resource when done by specifying the location of variable files, and state file:

    terraform destroy -var-file=config/config.tfvars -state=config/config.tfstate -refresh

    Note: This is not reversible all resources stored in the Terraform state will be removed.

  2. Terraform will not delete the group that was created in Activity Tracker, you will need to delete this group manualy. Follow the steps outlined in the documentation here: https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-group_data_access#groups_data_access_editing

Tips and Tricks

Importing an existing monitoring agent deployment into the Terraform State

Importing the namespace in which the monitoring agent was deployed:

  terraform import -var-file=config/config.tfvars -state-out=config/config.tfstate kubernetes_namespace.ibm_observe ibm-observe

Importing the service account for the monitoring agent:

  terraform import -var-file=config/config.tfvars -state-out=config/config.tfstate kubernetes_service_account.sysdig_agent ibm-observe/sysdig-agent

Importing the cluster role and cluster role binding:

  terraform import -var-file=config/config.tfvars -state-out=config/config.tfstate kubernetes_cluster_role.sysdig_agent sysdig-agent
  terraform import -var-file=config/config.tfvars -state-out=config/config.tfstate kubernetes_cluster_role_binding.sysdig_agent sysdig-agent

Related Content

Tutorial: Analyze logs and monitor application health.

Tutorial: Plan, create and update deployment environments

License

See License.txt for license information.