search

alertmanager / alert_manager

Splunk Alert Manager with advanced reporting on alerts, workflows (modify assignee, status, severity) and auto-resolve features
Other
81 stars 44 forks source link

Incident shows no results for Incident Posture dashboard #259

Open floppyza opened 4 years ago

floppyza commented 4 years ago

Hi

Running Splunk 8.0.4.1 - with Alert manager 3.0.3 (Linux). All the health checks are good and the alerts are being created.

However when I click on the Incident on the Incident Posture dashboard it shows now results: image

I check the alert manager index and I do see the alert info. If I click on the search button - it opens the search in a different window and returns all the results/table. I also tried adding the space seperated list of fields in the incident settings but that has not worked either.

Regards Brandon

floppyza commented 4 years ago

Tried running the | loadincidentresults incident_id=fc137b74-d730-48b8-8afe-190f80a0239c

Message: RuntimeWarning at "/opt/search_head/splunk/etc/apps/alert_manager/bin/loadincidentresults.py", line 29 : Specified setting ""collect_data_results" in "alert_manager.conf" does not exist.

my2ndhead commented 4 years ago

Have you restarted Splunk after saving Global Settings?

Can you check if $SPLUNK_HOME/etc/apps/alert_manager/default/alert_manager.conf resp. $SPLUNK_HOME/etc/apps/alert_manager/local/alert_manager.conf exists?

On Mon, Aug 3, 2020 at 1:06 PM floppyza notifications@github.com wrote:

Tried running the | loadincidentresults incident_id=fc137b74-d730-48b8-8afe-190f80a0239c

Message: RuntimeWarning at "/opt/search_head/splunk/etc/apps/alert_manager/bin/loadincidentresults.py", line 29 : Specified setting ""collect_data_results" in "alert_manager.conf" does not exist.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/alertmanager/alert_manager/issues/259#issuecomment-667960520, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNC4F6HEDCQSKSRDQONLOTR62KZTANCNFSM4PTGUQEQ .

floppyza commented 4 years ago

Yes - it is a SHC with 3 nodes. Configured on the 1 node and did a rolling restart.

Both files exist in the local and default directories.

Have you restarted Splunk after saving Global Settings? Can you check if $SPLUNK_HOME/etc/apps/alert_manager/default/alert_manager.conf resp. $SPLUNK_HOME/etc/apps/alert_manager/local/alert_manager.conf exists? … On Mon, Aug 3, 2020 at 1:06 PM floppyza @.***> wrote: Tried running the | loadincidentresults incident_id=fc137b74-d730-48b8-8afe-190f80a0239c Message: RuntimeWarning at "/opt/search_head/splunk/etc/apps/alert_manager/bin/loadincidentresults.py", line 29 : Specified setting ""collect_data_results" in "alert_manager.conf" does not exist. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#259 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNC4F6HEDCQSKSRDQONLOTR62KZTANCNFSM4PTGUQEQ .

my2ndhead commented 4 years ago

Can you hit the rest endpoint for the settings https://:8089/servicesNS/nobody/alert_manager/configs/alert_manager/settings https://95.216.220.64:8089/servicesNS/nobody/alert_manager/configs/alert_manager/settings to see if the setting is there?

On Mon, Aug 3, 2020 at 1:33 PM floppyza notifications@github.com wrote:

Yes - it is a SHC with 3 nodes. Configured on the 1 node and did a rolling restart.

Both files exist in the local and default directories.

Have you restarted Splunk after saving Global Settings? Can you check if $SPLUNK_HOME/etc/apps/alert_manager/default/alert_manager.conf resp. $SPLUNK_HOME/etc/apps/alert_manager/local/alertmanager.conf exists? … <#m-4484360733227131319_> On Mon, Aug 3, 2020 at 1:06 PM floppyza @.***> wrote: Tried running the | loadincidentresults incident_id=fc137b74-d730-48b8-8afe-190f80a0239c Message: RuntimeWarning at "/opt/search_head/splunk/etc/apps/alert_manager/bin/loadincidentresults.py", line 29 : Specified setting ""collect_data_results" in "alert_manager.conf" does not exist. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#259 (comment) https://github.com/alertmanager/alert_manager/issues/259#issuecomment-667960520>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNC4F6HEDCQSKSRDQONLOTR62KZTANCNFSM4PTGUQEQ .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alertmanager/alert_manager/issues/259#issuecomment-667972062, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNC4FZCNQ3XFATTUHHAS3DR62OB7ANCNFSM4PTGUQEQ .

floppyza commented 4 years ago

Yes - here are the results:

<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?> <feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/"> <title></title> <id>https://localhost:9000/servicesNS/nobody/alert_manager/configs/alert_manager</id> <updated>2020-08-24T08:41:24+02:00</updated> <generator build="ab7a85abaa98" version="8.0.4.1"/> <author> <name>Splunk</name> </author> <opensearch:totalResults>1</opensearch:totalResults> <opensearch:itemsPerPage>30</opensearch:itemsPerPage> <opensearch:startIndex>0</opensearch:startIndex> <s:messages/> <entry> <title>settings</title> <id>https://localhost:9000/servicesNS/nobody/alert_manager/configs/alert_manager/settings</id> <updated>1970-01-01T02:00:00+02:00</updated> <link href="/servicesNS/nobody/alert_manager/configs/alert_manager/settings" rel="alternate"/> <author> <name>system</name> </author> <link href="/servicesNS/nobody/alert_manager/configs/alert_manager/settings" rel="list"/> <link href="/servicesNS/nobody/alert_manager/configs/alert_manager/settings" rel="edit"/> <content type="text/xml"> <s:dict> <s:key name="append_ignore_status">(resolved|closed)</s:key> <s:key name="auto_close_info">false</s:key> <s:key name="auto_close_info_status">auto_info_resolved</s:key> <s:key name="collect_data_results">true</s:key> <s:key name="default_impact">low</s:key> <s:key name="default_owner">unassigned</s:key> <s:key name="default_priority">low</s:key> <s:key name="default_urgency">low</s:key> <s:key name="eai:acl"> <s:dict> <s:key name="app"></s:key> <s:key name="can_list">1</s:key> <s:key name="can_write">1</s:key> <s:key name="modifiable">0</s:key> <s:key name="owner">system</s:key> <s:key name="perms"> <s:dict> <s:key name="read"> <s:list> <s:item>*</s:item> </s:list> </s:key> <s:key name="write"> <s:list> <s:item>*</s:item> </s:list> </s:key> </s:dict> </s:key> <s:key name="removable">0</s:key> <s:key name="sharing">system</s:key> </s:dict> </s:key> <s:key name="eai:attributes"> <s:dict> <s:key name="optionalFields"> <s:list> <s:item>auto_close_info</s:item> <s:item>auto_close_info_status</s:item> <s:item>collect_data_results</s:item> <s:item>default_impact</s:item> <s:item>default_owner</s:item> <s:item>default_priority</s:item> <s:item>default_urgency</s:item> <s:item>incident_list_length</s:item> <s:item>index</s:item> <s:item>index_data_results</s:item> <s:item>user_directories</s:item> </s:list> </s:key> <s:key name="requiredFields"> <s:list/> </s:key> <s:key name="wildcardFields"> <s:list/> </s:key> </s:dict> </s:key> <s:key name="incident_list_length">20</s:key> <s:key name="index">security_soc_alerts</s:key> <s:key name="index_data_results">true</s:key> <s:key name="user_directories">both</s:key> </s:dict> </content> </entry> </feed>

@my2ndhead