[SentinelOne] add `agent.id` to all agent related data collected from SentinelOne

[paul-tavares]

Description

Two requests with regards to the SentinelOne Integration:

1. For every document streamed to ES by the integration, if the data is associated with a SentinelOne agent, include the agent's id in the ES document (sentinel_one.[source_type].agent.id)
2. Change the default intervals for the data pulling to 30s for all data streams

Background

Opened as a result of discussion here: https://github.com/elastic/integrations/issues/9313#issuecomment-2110722080

Changes requested are in support of Security Solution's Bi-Directional Response Actions feature which enables our SIEM users to send actions to SentinelOne Agents directly from Kibana.

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[paul-tavares]

HI @chrisberkhout 👋

Just wanted to let you know - we recently lost access to the S1 environment and Jamie has been working with S1 to get it back. Looks like they re-enabled our env. today but we still don't yet know for how long we'll have access. Just wanted you to be aware case you want to login and grab data/info. for this issue case we lose access again (we're doing the same on my team for other bi-directional response actions).

cc/ @dasansol92 , @tomsonpl , @ashokaditya

[chrisberkhout]

@paul-tavares Thanks. I've opened PRs for these changes. I logged in to confirm details in the S1 documentation but I think we can confidently merge this without running against a live endpoint first.

[paul-tavares]

Thanks @chrisberkhout - thats great. and I agree - these are additions and relatively low risk.

[paul-tavares]

Hey @chrisberkhout , I just realized I have my local dev setup connected to S1 and so I went ahead and upgraded the S1 integration package to v1.22..0. Is the expectation that once I do that, any new data streamed to ES will have the field?

I just tested it and the alert even does not have the field populated (expand below to see data in index)

S1 Alert event

```jsonp { "_index": ".ds-logs-sentinel_one.alert-default-2024.06.04-000001", "_id": "J9pfTfpN9iJnTlCi3LBd6Ek4oEQ=", "_score": 1, "_source": { "agent": { "name": "ptavares-agentless-integrations-6800", "id": "4a8ec82f-8cb2-4954-9e7f-a12d70ab82fe", "ephemeral_id": "2b703b52-c771-47a8-9020-37217afd2ad1", "type": "filebeat", "version": "8.15.0" }, "process": { "parent": { "name": "bash", "start": "2024-06-10T12:43:07.170Z", "pid": 77078, "entity_id": "d72b329e-021a-98d6-1cc5-264139a7d587", "user": { "name": "Effective: ubuntu, Real: ubuntu, Login: ubuntu" }, "command_line": "-bash", "hash": { "sha1": "9ee6ab357c616aeeffc9831900b9a1424e201ffb" }, "executable": "/bin/bash" }, "name": "nslookup", "start": "2024-06-10T12:52:56.540Z", "pid": 77171, "entity_id": "d7b46f35-27d6-084c-103b-e660412eba34", "user": { "name": "Effective: ubuntu, Real: ubuntu, Login: ubuntu" }, "command_line": "nslookup elastic.co", "hash": { "sha1": "1c1bffe31dda5d5923efb31cf071b86bc607a5cf" }, "executable": "/usr/bin/nslookup" }, "elastic_agent": { "id": "4a8ec82f-8cb2-4954-9e7f-a12d70ab82fe", "version": "8.15.0", "snapshot": true }, "dns": { "question": { "name": "type: 1 elastic.co" } }, "rule": { "name": "test", "id": "1412065033578525161" }, "tags": [ "forwarded", "sentinel_one-alert" ], "input": { "type": "httpjson" }, "observer": { "serial_number": "d5b3aff3-1fbd-d1be-7875-d702c2850211", "version": "24.1.2.6" }, "sentinel_one": { "alert": { "analyst_verdict": "Undefined", "agent": { "site_id": "1392053568582758390" }, "process": { "integrity_level": "unknown", "parent": { "integrity_level": "unknown", "storyline": "d72b329d-9057-6c87-0167-29c7f50d3a35", "subsystem": "unknown" }, "storyline": "d72b329d-9057-6c87-0167-29c7f50d3a35", "subsystem": "unknown" }, "rule": { "severity": "High", "treat_as_threat": "Suspicious", "scope_level": "site" }, "dv_event": { "id": "01J013B57QBNAQNA4DGKW7MB6F_3" }, "info": { "hit": { "type": "Events" }, "event_type": "DNS", "updated_at": "2024-06-10T12:53:26.901Z", "reported_at": "2024-06-10T12:53:26.901Z", "dns": { "response": "34.107.161.234;" }, "source": "STAR", "status": "Unresolved" }, "target": { "process": { "start_time": "1970-01-01T00:00:00.000Z", "proc": { "integrity_level": "unknown" }, "file": { "is_signed": "unsigned" } } } } }, "@timestamp": "2024-06-10T12:53:10.608Z", "file": { "created": "1970-01-01T00:00:00.000Z", "mtime": "1970-01-01T00:00:00.000Z" }, "ecs": { "version": "8.11.0" }, "related": { "hosts": [ "ptavares-sentinelone-9160" ], "hash": [ "9ee6ab357c616aeeffc9831900b9a1424e201ffb", "1c1bffe31dda5d5923efb31cf071b86bc607a5cf" ] }, "data_stream": { "namespace": "default", "type": "logs", "dataset": "sentinel_one.alert" }, "host": { "os": { "name": "Linux", "family": "linux", "version": "Ubuntu 24.04 LTS 6.8.0-31-generic" }, "name": "ptavares-sentinelone-9160", "type": "server" }, "event": { "agent_id_status": "verified", "ingested": "2024-06-10T12:53:57Z", "created": "2024-06-10T12:53:47.707Z", "kind": "event", "id": "1969604950985996380", "category": [ "malware" ], "type": [ "info" ], "dataset": "sentinel_one.alert" } } }, ```

I also took a quick look at index schema and the new field does not seem to be defined for it.

Looking at the PR that was merged - I see this in several places: target_field: sentinel_one.agent.agent.id - note the double usage of the word agent. Is that correct?

Another question: Although I have never worked with it, I know S1 has another integration for Cloud Funnel. would these changes applicable to that data as well?

[paul-tavares]

@chrisberkhout - Did you see my comment above? ☝️

[chrisberkhout]

Hi @paul-tavares,

Is the expectation that once I do that, any new data streamed to ES will have the field?

The mappings should be updated, which you should see under Stack Management > Index Management > Component Templates, but you might need to wait for the data stream to roll over to a new backing index to see it in an index.

The example event you shared is for the alert data stream, which is one for which no agent ID data is available. Here's a summary of what was done for each data stream in the PR:

Data Stream	Agent ID field after PR	Source of Agent ID
agent	sentinel_one.agent.agent.id	Was already stored in host.id.
threat	sentinel_one.threat.agent.id	Was already stored in host.id.
activity	sentinel_one.activity.agent.id	Was already stored in sentinel_one.activity.agent.id.
alert	N/A	No agent ID available, but agentDetectionInfo.uuid was already stored in observer.serial_number.
group	N/A	No agent ID available, and no information about any specific agent.

Looking at the https://github.com/elastic/integrations/pull/10102 - I see this in several places: target_field: sentinel_one.agent.agent.id - note the double usage of the word agent. Is that correct?

Yes, that was intended. The field is sentinel_one.<source_type>.agent.id, so for the agent data stream (data stream name = source type) you'll see agent twice in the field name.

Although I have never worked with it, I know S1 has another integration for Cloud Funnel. would these changes applicable to that data as well?

Taking a quick look, the data or at least the structure of the data ingested with Sentinel One Cloud Funnel seems quite different from what we get from the API for the sentinal_one integration. I do see a sentinel_one_cloud_funnel.event.agent.uuid field, but the Agent UUID seems to be a different value from the Agent ID, which isn't available, so I don't think there's a corresponding change to make in the Sentinel One Cloud Funnel integration.

[chrisberkhout]

@chrisberkhout - Did you see my comment above? ☝️

Yes, sorry. PR auto-closed while I was writing the above comment.

[paul-tavares]

Thanks @chrisberkhout

Re: "...for the alert data stream, which is one for which no agent ID data is available..."

In these cases, were we have observer.serial_number - would it be possible to query the agents API with this uuid and get the agent.id that way?

Also, thanks for the explanation on the mappings/index. completely forgot that i should probably have triggered a manual index rollover 🤦

[chrisberkhout]

Hi @paul-tavares,

In these cases, were we have observer.serial_number - would it be possible to query the agents API with this uuid and get the agent.id that way?

It is possible, shall I open an issue for that?

It may require an extra request for every received alert event.

It's tricky enough that I think we'd need a test account with some data in it to make sure it works. Is that something you already have or could help with?

---

Notes for myself: the /web/api/v2.1/agents endpoint does allow filtering for particular UUIDs. We could use the request chaining functionality of the HTTP JSON input to make an additional request for the agent related to each alert, then produce an event from the parent response (use .parent_last_response) that includes the ID from the chained response. It would be easier to do this enrichment in CEL, but I don't think we have a smooth migration path for that yet.

[paul-tavares]

Re: "It is possible, shall I open an issue for that?"

Yes please :)

Yes, we do have access to the S1 test environment again - here are the creds:

https://p.elstc.co/paste/iRBKfAHF#vNjqZkS+2khFmcqYPVsqDJ1CAg7LCSW3gr5E+GDmxTo