Open paki20 opened 3 years ago
Hi @paki20
I'm suspecting, that the alert_actions.conf is not replicated.
https://docs.splunk.com/Documentation/Splunk/8.1.0/DistSearch/HowconfrepoworksinSHC
You have to save the global settings on all SHC members manually if the file is not replicated.
Please let me know if it works out.
Thanks!
Hi my2ndhead,
I installed alert_manager 3.0.5 but my issue is not resolve.
When I watch alert_action.conf is replicated, when I make a health check I have:
Ok. it seems for some reasons the scripts that populate the kv store etc. have not been run.
Under Splunk's Settings -> Data Inptus -> Scripts you have to enable following alert_manager_migrate-vX.Y scripts:
2.0 2.1 3.0
After you have enabled the scripts do a rolling restart of the SHC.
Let me know if it helps.
Hi my2ndhead,
I have enable scripts 2.0 / 2.1 / 3.0
And I did a rolling restart of the SHC but not work again.
What does the health-check now say?
Hi m2ndhead,
After health-check is the same result:
Built-in alert statuses deployed Failed Default Email Templates deployed Failed Default Notification Schemes Deployed Failed
Can you check for any errors in $SPLUNK_HOME/var/log/alert_manager_migration.log and splunkd.log?
Hi my2ndhead,
I hope you are fine, happy new year !
When I checked, I have:
splunkd.log
01-04-2021 10:20:59.526 +0100 INFO sendmodalert - action=alert_manager - Alert action script completed in duration=1660 ms with exit code=0 01-04-2021 10:20:59.531 +0100 INFO sendmodalert - Invoking modular alert action=alert_manager for search="[TEST-01] Test" sid="scheduler__user1__search__RMD5rs9045078zc8w2b_at_1609832000_1907_6209994-GEEL-4LKP-Z5Y9-276CAE80FA7F" in app="search" owner="user1" type="saved" 01-04-2021 10:21:01.282 +0100 INFO sendmodalert - action=alert_manager - Alert action script completed in duration=1747 ms with exit code=0 01-04-2021 10:22:35.847 +0100 ERROR ExecProcessor - message from "/opt/splunk/etc/apps/alert_manager/bin/alert_manager_scheduler.sh" /bin/sh: /opt/splunk/etc/apps/alert_manager/bin/alert_manager_scheduler.sh: Permission not granted 01-04-2021 10:22:35.865 +0100 INFO sendmodalert - action=alert_manager - Alert action script completed in duration=1489 ms with exit code=0 01-04-2021 10:22:35.869 +0100 INFO sendmodalert - Invoking modular alert action=alert_manager for search="[TEST-01] Test" sid="scheduler__user1__search__RMD5rs9045078zc8w2b_at_1609832000_1907_6209994-GEEL-4LKP-Z5Y9-276CAE80FA7F" in app="search" owner="user1" type="saved"
alert_manager.log
2021-01-04 10:20:45,010 INFO pid="23145" logger="alert_manager" message="Firing incident_created event for incident=f0f12944-b5d6-4c72-8316-3cf85db07192" (alert_manager.py:745) 2021-01-04 10:20:45,093 INFO pid="23145" logger="alert_manager" message="Alert handler finished. duration=0.676s" (alert_manager.py:791) 2021-01-04 10:20:46,100 INFO pid="23152" logger="alert_manager" message="Found job for alert '[TEST-01] Test' with title '[TEST-01] Test'. Context is 'search' with 39 results." (alert_manager.py:566) 2021-01-04 10:20:46,295 INFO pid="23152" logger="alert_manager" message="Incident status after suppresion check: new" (alert_manager.py:669) 2021-01-04 10:20:46,369 INFO pid="23152" logger="alert_manager" message="Incident initial state added to collection for job_id=scheduler__user1__search__RMD5rs9045078zc8w2b_at_1609832000_1907_6209994-GEEL-4LKP-Z5Y9-276CAE80FA7F with incident_id=f0f12944-b5d6-4c72-8316-3cf85db07192 key=8ff2ggmm0964g3098098q221" (alert_manager.py:688) 2021-01-04 10:20:46,437 INFO pid="23152" logger="alert_manager" message="Results for incident_id=f0f12944-b5d6-4c72-8316-3cf85db07192 written to collection." (alert_manager.py:706) 2021-01-04 10:20:46,448 INFO pid="23152" logger="alert_manager" message="Alert metadata written to index=alerts" (alert_manager.py:255)
Hi,
Could you help me?
Hi,
I installed alert manager 3.0.4 in my SH standalone (splunk test) is working in my splunk enterprise 8.1
I installled alert manager 3.0.4 in my SHC in cluster (splunk prod) is not working in my splunk enterprise 8.1
Could you tell me why because I don't understand. Alert Manager 3.0.4 not compatible with SHC cluster?
Thanks for your help.