flexera-public / right_st

RightScale ServerTemplate and RightScript tool
MIT License
12 stars 1 forks source link

PSUP-1709_fix_alert_condition_validation #53

Closed gregcoit closed 6 years ago

gregcoit commented 6 years ago

Adding support for alert metrics like cpu-0.graph/cpu-idle.value

bryankaraffa commented 6 years ago

ST YAML:

Name: PSUP-1709 Test Template
Description: Example Template for WSOP poker casino instances for issue PSUP-1709
RightScripts:
  Boot:
  - RL10_Linux_Setup_Hostname.sh
  Decommission:
  - RL10_Linux_Shutdown_Reason.sh
Alerts:
- Name: Jetty Errors - Silent
  Clause: If GenericJMX-logs-METRICSAPPENDER.error/gauge-OneMinuteRate.value > 15
    for 1 minutes Then escalate Silent
- Name: Heap Gauge
  Clause: If GenericJMX-gauge-gauge_heap.usage/gauge.min > 0.6 for 60 minutes Then
    escalate default
- Name: rs high network tx activity
  Description: Raise an alert if the amount of network data transmitted is too high.
    This alert may be modified or cloned on an instance to match the actual network
    interfaces that are present. See the RL10 Linux Setup Alerts RightScript (rll/setup-alerts.sh)
    for more details.
  Clause: If interface-eth0/if_octets.tx > 10000000 for 10 minutes Then escalate PagerDuty-MTL
- Name: rs high network rx activity
  Description: Raise an alert if the amount of network data received is too high.
    This alert may be modified or cloned on an instance to match the actual network
    interfaces that are present. See the RL10 Linux Setup Alerts RightScript (rll/setup-alerts.sh)
    for more details.
  Clause: If interface-eth0/if_octets.rx > 50000000 for 30 minutes Then escalate PagerDuty-MTL
- Name: Config read error
  Description: Detect a config error
  Clause: If GenericJMX-gauge-gauge_config_read_error/gauge.value > 0 for 1 minutes
    Then escalate LiveOps

Upload output with this change:

# ../right_st st upload Director-embedded_jetty.yml -a 7954 -x `date +%s`
Validating Director-embedded_jetty.yml
Validation successful, uploading as '1519848256_Director - embedded jetty'
Creating ServerTemplate with HREF /api/server_templates/408170003
Updating MCIs:
  MCIs synced
Updating or Creating RightScripts:
  Creating a new RightScript named '1519848256_RL10 Linux Setup Hostname' from RL10_Linux_Setup_Hostname.sh
    RightScript created with HREF /api/right_scripts/619659003
  Creating a new RightScript named '1519848256_RL10 Linux Enable Managed Login' from RL10_Linux_Enable_Managed_Login.sh
    RightScript created with HREF /api/right_scripts/619660003
  Uploading attachment 'libnss_rightscale.tgz' with md5 63ee9fa129753d039eb25f30cf8af506
  Uploading attachment 'rs-ssh-keys.sh' with md5 ac21380e071c10514594fdc5380258dd
  Uploading attachment 'rightscale_login_policy.te' with md5 a15f735ad5c0d4d42c260b93dc35aeef
  Creating a new RightScript named '1519848256_RL10 Linux Setup NTP' from RL10_Linux_Setup_NTP.sh
    RightScript created with HREF /api/right_scripts/619661003
  Creating a new RightScript named '1519848256_RL10 Linux Enable Monitoring' from RL10_Linux_Enable_Monitoring.sh
    RightScript created with HREF /api/right_scripts/619662003
  Creating a new RightScript named '1519848256_RL10 Linux Setup Alerts' from RL10_Linux_Setup_Alerts.sh
    RightScript created with HREF /api/right_scripts/619663003
  Creating a new RightScript named '1519848256_RL10 Linux Shutdown Reason' from RL10_Linux_Shutdown_Reason.sh
    RightScript created with HREF /api/right_scripts/619664003
  RightScripts synced
Setting order of RightScripts:
  Adding /api/right_scripts/619659003 to boot bundle
  Adding /api/right_scripts/619660003 to boot bundle
  Adding /api/right_scripts/619661003 to boot bundle
  Adding /api/right_scripts/619662003 to boot bundle
  Adding /api/right_scripts/619663003 to boot bundle
  Adding /api/right_scripts/619664003 to decommission bundle
  RightScript order set
Setting Inputs
  No inputs to set
Synchronizing Alerts
  Adding Alert Jetty Errors - Silent
  Adding Alert Heap Gauge
  Adding Alert rs high network tx activity
  Adding Alert rs high network rx activity
  Adding Alert Config read error
Successfully uploaded ServerTemplate 1519848256_Director - embedded jetty with HREF /api/server_templates/408170003

No longer errors out on validation of alerts

douglaswth commented 6 years ago

I have a fix for the Travis glide issue that I'll bring in on another branch, I'm going to merge this one.