[Security Solution] Sending Entire Detection Alert JSON to JIRA Service Desk Issue Output Doesn't Work

[austinsonger]

Describe the bug:

Sending Entire Detection Alert JSON to JIRA Service Desk Output Doesn't work

It probably has to deal with JIRA Wiki Rendering I really wish I could send the JSON into a code block in JIRA.

Kibana/Elasticsearch Stack version: 7.12.0

Server OS version: Elastic Cloud

Browser and Browser OS versions: Brave,Chrome

                "subActionParams": {
                    "comments": [],
                    "incident": {
                        "issueType": "10001",
                        "summary": "{{alertName}}",
                        "description": "{code:json}\n{{#context.alerts}}{{{.}}}{{/context.alerts}}\n{code}",
                        "priority": "Highest"
                    }

It looks like this when it creates a ticket in JIRA.

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[austinsonger]

It has never done this to me before. Most of the other ones it has came through whole.

Took me here

[jonathan-buttner]

Just wanted to give an update, @cnasikas, @michaelolo24 and I will be pairing on this next week

[jonathan-buttner]

@austinsonger I think we were able to track down why the {code} block isn't formatting correctly. The alerts contain the action parameters that are being sent to Jira. So we have code blocks nested inside of code blocks. You can find the action parameters here:

Unfortunately this means you can't include the entire alert in the description. Instead you could define the specific objects like this:

{code:json}\n{{#context.alerts}}{{agent}},{{host}},{{event}}{{/context.alerts}}\n{code}

Are you still encountering the URL issue?

[austinsonger]

That is going to be a tedious things to get what I want in the ticket. Like it would be nice sometime to be able to get all of it.

Like here is example of what we get from AlienVault

URL

Yeah Probably 1/3 tickets of the tickets being generated are basically replacing elastic url like the following example shows.

[jonathan-buttner]

Thanks for the example @austinsonger

That is going to be a tedious things to get what I want in the ticket. Like it would be nice sometime to be able to get all of it.

Yeah I can't seem to find anyway in the mustache docs to exclude a specific field or object. Ideally we could just ignore signals.rule.actions and still use the {{.}} notation so we don't have to write every single object in the alert.

I'll see if I can figure out the URL issue.

@pmuellr @gmmorris @mikecote We'd like to format the alerts that came from a rule in {code} tags for Jira but avoid the nesting of the tags because of the signals.rule.actions, do you know of a way to include all of an alert using something like {{.}} but exclude specific objects or fields in the mustache syntax?

[jonathan-buttner]

@austinsonger could you share the mustache template you're using with the Jira connector? I'm having some trouble reproducing the URL issue. Does it seem to be consistent with any particular rule? Or could you share the rule that is providing the invalid URL so I can try it locally?

Thanks!

[austinsonger]

Here you go. So sometimes it produces the URL I want, but sometimes it errors like above.

- *Number of Alerts*: {{state.signals_count}}\n- *Risk score*: {{context.rule.risk_score}}\n- *Severity*: {{context.rule.severity}}\n\n h2. Rule Details\n [View Detection Alert|{{{context.results_link}}}]\n- *Rule Description*: {quote}{{context.rule.description}}{quote}\n- *Rule Query*: {quote}{{context.rule.query}}{quote}\n\n h2. Source\n {{#context.alerts}}\n- *Source IP Address*: {noformat}{{source.ip}}{noformat}\n- *Source Port*: {noformat}{{source.port}}{noformat}\n {{/context.alerts}}\n\n h2. Destination\n {{#context.alerts}}\n- *Destination IP Address*: {noformat}{{destination.ip}}{noformat}\n- *Destination Port*: {noformat}{{destination.port}}{noformat}\n {{/context.alerts}}\n

[gmmorris]

@pmuellr @gmmorris @mikecote We'd like to format the alerts that came from a rule in {code} tags for Jira but avoid the nesting of the tags because of the signals.rule.actions, do you know of a way to include all of an alert using something like {{.}} but exclude specific objects or fields in the mustache syntax?

I don't think there's a way to do that in Mustache TBH. The only thing I can think of is the use of custom functions, but that's buried deep in our backlog, and given the current set of priorities I don't see how we could fit this into the 7.x timeframe.

[jonathan-buttner]

Thanks for the response @gmmorris, any thoughts on the context.results_link not producing the correct URL? Seems like sometimes it doesn't include the correct basepath to kibana but instead is just a relative path for the alert.

@dhurley14 @marshallmain is the context.results_link populated in the detection engine?

[cnasikas]

@austinsonger Could you please inspect if the results_link is a relative one or an absolute one?

[gmmorris]

Thanks for the response @gmmorris, any thoughts on the context.results_link not producing the correct URL? Seems like sometimes it doesn't include the correct basepath to kibana but instead is just a relative path for the alert.

@dhurley14 @marshallmain is the context.results_link populated in the detection engine?

Looks like that's set by detections: https://github.com/elastic/kibana/blob/ca17d931deaff7217dc7f75ae1ad6ff3be680543/x-pack/plugins/security_solution/server/lib/detection_engine/notifications/schedule_notification_actions.ts#L38

[austinsonger]

Sometimes it takes the relative path and places it behind the Jira instance and then sometimes it places behind the kibana instance.

[pmuellr]

There are at least some potential cases where the results_link being produced by the alert is relative.

See: https://github.com/elastic/kibana/blob/master/x-pack/plugins/security_solution/server/lib/detection_engine/notifications/utils.ts

APP_PATH is a const of /app/security, so if kibanaSiemAppUrl is not set when the function is called, the result will be a relative URL.

I'd imagine it's always a relative URL, but not sure.

As of of 7.12, there is a top-level kibanaBaseUrl variable defined, which you should be able to use in your action to make the URL absolute: see PR https://github.com/elastic/kibana/pull/90525

[pmuellr]

I should have been more specific; I think you should be able to replace

[View Detection Alert|{{{context.results_link}}}]

with

[View Detection Alert|{{{kibanaBaseUrl}}}{{{context.results_link}}}]

[austinsonger]

I will try it and see what happens.

[austinsonger]

@gmmorris @pmuellr @cnasikas

This is what happens

after I followed this:

[View Detection Alert|{{{kibanaBaseUrl}}}{{{context.results_link}}}]

[jonathan-buttner]

@spong @FrankHassanabad any ideas why the context.results_link would be undefined? it looks like it's populated here: https://github.com/elastic/kibana/blob/master/x-pack/plugins/security_solution/public/detections/components/rules/step_rule_actions/index.tsx#L83

during the rule creation/edit flow. I checked on our dev instance and in the default .kibana alias it seems like there are a lot of rules that do not have it populated.

I can't seem to reproduce the issue locally though 🤔

[marshallmain]

It appears that rules created through the UI have meta.kibana_siem_app_url populated with an absolute path to the Kibana instance, whereas prepackaged rules and rules created through the API do not have that field populated. meta.kibana_siem_app_url, if provided, is used as the base path for results_link here as Patrick mentioned. When meta.kibana_siem_app_url is not provided, results_link defaults to a relative path during rule execution. I suspect JIRA is respecting the absolute paths, but when provided with a relative path it automatically assumes that it's a link within JIRA and adds its own base path.

@austinsonger can you verify if the rules with correct URLs have the meta.kibana_siem_app_url field defined on the rule?

If this is the problem, we should have the detection engine default to using the full absolute path rather than a relative path.

[gmmorris]

Looks like this is being picked up by Security, so removing the Actions labeling to reduce noise on that feature :)

[austinsonger]

Weird... Now all of sudden it started to add my custom friendly URL right in front of the Elastic generated URL. It used to only generate the Elastic generated URL.

https://<Custom>.kb.eastus2.azure.elastic-cloud.com:9243https://######################.eastus2.azure.elastic-cloud.com:9243/app/security/detections/rules/id/b03e2910-97e9-11eb-ab85-dfc1f6da9d3f?timerange=(global:(linkTo:!(timeline),timerange:(from:1627330301543,kind:absolute,to:1627330601543)),timeline:(linkTo:!(global),timerange:(from:1627330301543,kind:absolute,to:1627330601543)))

[austinsonger]

FYI: Just updated to 7.13.4 on Friday.

When I actually click the link the I get this.

[admlko]

I'm seeing the same behavior.

I am using the API to load and enable prebuilt rules. After that, the context.results_link contains only relative path (https://hostname:port is missing).

If I open the rule in Kibana for editing and save it, the context.results_link is absolute path including https://hostname:port and works correctly.

[chadnormanimpact]

Also seeing this still...

[chadnormanimpact]

@sophiec20 has this been completed?

[pmuellr]

This issue is a blast from the past I guess!

Also seeing this still...

OK, so this issue starts off with an issue about including Jira markdown bits in an action parameter template, and dumping the alerts doesn't work because it includes the action templates which include the Jira markdown, which screw up Jira.

Then the issue seems to change to issues with context.results_link.

Which of these are you still seeing?

[pmuellr]

I think I figured out how to make the Jira markdown things work - at least better - when the action parameter templates are in the JSON being rendered.

In my action template, I specified the following:

{c{{X}}ode:json} "42" {co{{X}}de}

The {{X}} bits will be rendered to empty strings as there is no variable X available, so the end result is:

{code:json} "42" {code}

The action templates embedded deep in the JSON would still have the {{X}} in them, and presumably wouldn't get matched by Jira's markup parser.

It's not clear to me how elaborate the Jira parser is, maybe just the {code:json} and {code} parts need the artificial embedded empty string variable? Or will things like {quote} as well? Dunno.

[chadnormanimpact]

This issue is a blast from the past I guess!

Also seeing this still...

OK, so this issue starts off with an issue about including Jira markdown bits in an action parameter template, and dumping the alerts doesn't work because it includes the action templates which include the Jira markdown, which screw up Jira.

Then the issue seems to change to issues with context.results_link.

Which of these are you still seeing?

Still seeing issues with context.results_link. in some alerts not all.

[pmuellr]

You'll need to provide the version of the Elastic Stack you're using, the rule types with the issues, and some examples of what you're seeing and what you expect.

But could you open a new issue please? This already has a long thread, will be pinging lots of people, and seems to have changed topic over time.