fancherblack / rqst

Splunk app to manage Splunk data onboarding requests and forecast license consumption.
MIT License
0 stars 0 forks source link

Overview

Check this page for a summary of rqst's capabilities.

The rqst app provides new structure and visibility to Splunk project teams, admins, and users. It takes the place of the "tracker" spreadsheet and streamlines data source-specific collaboration. It provides a customizable form inside of Splunk for end-users to submit and monitor requests for data ingest. From an admin standpoint, Splunk engineers can easily filter, view, and interact with these requests, to include adding private notes, tags, and other information. Managers can better understand user needs, resource requirements, and the budgetary impact of individual requests or requests from specific parts of the organization.

This document provides information relating to the installation and configuration of the rqst Splunk app.

Installation Quick Start

As a user with admin rights, perform the following on your search head:

kvkit Request Form Configuration

The kvkit application provides a tremendous amount of control over the request form configuration and layout. The options that follow are the recommended baseline for the request form, which is associated with the rqst app’s rqst_data collection (rqst > rqst_data > Config). Update the sort order, display, and descriptions as needed.

When a user opens the request form via the dashboard widget, information about the user are passed as tokens to the kvkit form. Thus submitted data will contain the requestor’s information even though it is not collected by the form.

Field-Specific Config

Field Name Recommended Attributes Recommended Field Type Recommended Field Value
admin_index Hidden Input
admin_sourcetype Hidden Input
admin_user Hidden Input

custom_field

This field can be used to collect any data. This example is for Business Justification.

Display: Business Justification Dropdown search: | inputlookup rqst_kv_options | makemv delim="," custom_field_values | mvexpand custom_field_values | table custom_field_values
data_description Display: Data Short Description
Required
Input
data_group_access Display: Stakeholder Access Multiselect search: | inputlookup rqst_kv_groups | table splunk_role
data_hostnames Display: System Inventory File .csv,.xlsx,.txt
data_id Hidden Input search: | makeresults | eval empty_id=0 | appendcols [| inputlookup rqst_kv_data | stats max(data_id) AS last_id | eval last_id=last_id + 1 | table last_id] | fillnull value=0 last_id | eval data_id=if(empty_id != last_id, last_id, 1) | table data_id
data_sample Display: Data Sample File .txt,.csv,.json,.log
data_tags Hidden Input
data_transport Display: Data Transport Dropdown search: | inputlookup rqst_kv_options | makemv delim="," data_transport | mvexpand data_transport | table data_transport
data_volume Display: Estimated Data Volume Input
due_date Display: Date Needed Date
notes Display: Notes Textarea
origin Hidden Input kvkit
priority Priority Dropdown Low|Medium|High
requestor_email Hidden Input
requestor_realname Hidden Input
requestor_user Hidden Input
status Hidden Input
timestamp Hidden Input search: | makeresults | eval timestamp = strftime(now(),"%Y-%m-%d %H:%M:%S") | table timestamp
use_case Display: Use Case Dropdown search: | inputlookup rqst_kv_options | rex mode=sed field=use_case "s/\s//g" | makemv delim="," use_case | mvexpand use_case | table use_case

Sharing

Set the form sharing to Open.

Public Template

Set the public template to rqst. This template controls the look and feel of the request form. Add your logo, update text, or completely change styling by editing <KVKIT_HOME>/views/custom/rqst.

Confirmation Page

Set the confirmation page to rqst-confirmation.

Post-Process Search

The following post-process search will log the request submission to the rqst_audit collection:

| inputlookup rqst_kv_data | search data_id=!{data_id} | eval action_detail="Created request", action_type="create", timestamp=now() | rename requestor_user AS user | table action_detail action_type data_id timestamp user | outputlookup append=true rqst_kv_audit

Alternate Request Form (without kvkit)

If you do not plan to use kvkit to serve the request form and would rather use the request form inside of Splunk, proceed with configuration as outlined under your preferred authentication option below.

Authentication Option 1: Hard-Coded

Authentication Option 2: passwords.conf**

Set the rqst_rest user password in passwords.conf at the command line with:

curl -k -u <your_admin_user>:<your_admin_password> https://&lt;your\_search\_head&gt;:8089/servicesNS/nobody/rqst/storage/passwords -d name=rqst_rest -d password=<the rqst_rest password you set>

This will look something like this:

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/rqst/storage/passwords -d name=rqst_rest -d password=restchangeme

Next, run the following:

curl -k -u <your_admin_user>:<your_admin_password> https://&lt;your\_search\_head&gt;:8089/servicesNS/nobody/rqst/storage/passwords/\_acl -d perms.read=* -d sharing=global

Using the same values as our other curl example, this will look something like this:

curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/rqst/storage/passwords/\_acl -d perms.read=* -d sharing=global

Dashboard Changes

To make use of the Splunk-native (non-kvkit) request form in the rqst app do the following:

Customization & Configuration Options

Certain aspects of rqst are driven by configurable options. The options shown below are set in the rqst_options collection.

The email notification and approval process related options are not used by the kvkit application, as kvkit exposes and provides this functionality by way of form configuration.

approval_process

Enable or disable the approval process. If set to "false", new requests will be set to a status of "New" and be immediately available to the admin team. If set to "true", new requests will be set to "Approval" status and require approval by a team member with approval authority before being available to the admin team.

Example value: true

priority

List of options for the priority field on rqst dashboards.

Example values: low, medium, high

status

List of options for the status field on rqst dashboards.

Required values: New, Approval, Hold, Rejected, Working, Complete

use_case

List of options for the use case field on rqst dashboards.

Example values: security, it ops, server, mission, voice

data_transport

List of options for the Data Transport field on rqst dashboards.

Example values: Universal Forwarder, Heavy Forwarder, Syslog, API

**indexer_daily_ingest_target

Your target daily index volume in GB per indexer based on your Splunk environment (hardware, I/O, etc.) and application mix.

Example value: 300

cost_per_license_gb

The cost of license (GB) to be used in dashboard calculations.

Example value: 5.50

cost_per_indexer

The cost of indexer resources to be used in dashboard calculations. Indexer resources, or Indexer Load (IDXL), is determined by requested license / indexer_daily_index_target.

Example value: 15.25

cost_per_tb_storage

The cost of storage (TB) to be used in dashboard calculations.

Example value: 2.50

help_url

The web address that the Help button links to in User Workspace. Consider setting this to an internal knowledgebase or intranet site containing information about your Splunk admin team and operations.

Example value: https://sharepoint.yourcompany.com/something/here/splunk-admin-team

email_notifications

Enable or disable email notifications. If set to "true", email notifications will be sent to the admin team and users on request creation and update. If set to "false", email notifications will not be sent.

Example value: false

email_server

Email server used for sending emails when requests are created/updated

Example value: smtp.gmail.com:587

email_new_request_user

Body of the email sent to the requestor upon request submission.

Example value: Hey there user, thanks for your request. We’re on it!

email_new_request_approver

If the approval process is enabled, the contents of the email sent to team members with approval authority for new requests.

Example value: Hi approvers! Please do your thing.

email_new_request_admin

If approval process is disabled, the contents of the email sent to admins once a request is created

Example value: Hey admins! You’ve got more work to do.

email_updated_request_user

Body of the email sent to a user when their request has been updated

Example value: Dearest user, your request has been updated! Woo!

kvkit_server

The address of your kvkit instance.

Example value: https://kvkit.yourcompany.xyz:8008

KV Store Collections

The rqst app leverages KV Store collections for all request operations. The table below lists the collections and their role:

Collection Name Description
rqst_data Contains main request information.
rqst_audit Contains log of activity on each request.
rqst_journal Contains admin notes made on requests.
rqst_groups Contains mapping of Splunk role to groups, which is used to provide organizational context.
rqst_users Contains Splunk users and email addresses used to populate the request form.
rqst_team Contains Splunk admin team members.
rqst_options Contains options for the rqst app.

KV Store Backup

Since all app data is stored in KV store collections and collections are susceptible to accidental deletion or overwrite (e.g., unintentional outputlookup by an admin), it’s very important that you frequently backup all rqst collections to prevent data loss.

If you would like to learn more on this topic, see Backup and Restore the KV store in the Splunk docs.

Appendix A – Open Source Software

The rqst app leverages the software identified in the table below.

Name URL License Type & Link
jQuery https://jquery.com MIT
Bootstrap https://getbootstrap.com MIT
Chosen.js https://harvesthq.github.io/chosen/ MIT
DOMPurify https://github.com/cure53/DOMPurify MPL v2