[SecuritySolution] Service Entity Store

[machadoum]

Summary

Service Definition:

https://github.com/elastic/kibana/pull/202344/files#diff-42c7dd345e0500c97f85824904a70a11162827ea8f8df6982082a9047ca04ff1

Acceptance Criteria

• [x] Upon installation of the entity store, the Service entity definition should be created by default
• [x] The Service definition will be installed in the exact same way as the User and Host definitions
• [x] The unique identifier for service entities will be service.name
• [x] The fields captured for service entities should match the field mapping spreadsheet (see Implementation Notes below)

Stored Entity

{
          "@timestamp": "2024-12-02T10:43:13.856Z",
          "event": {
            "ingested": "2024-12-02T10:51:28.987428Z"
          },
          "entity": {
            "name": "test123 name",
            "id": "test123 name",
            "source": "logs-blito",
            "type": "service"
          },
          "service": {
            "node": {
              "roles": [
                "test123 node roles"
              ],
              "name": [
                "test123 node name"
              ]
            },
            "environment": [
              "test123 environment"
            ],
            "address": [
              "test123 address"
            ],
            "name": "test123 name",
            "id": [
              "test123 id"
            ],
            "state": [
              "test123 state"
            ],
            "ephemeral_id": [
              "test123 ephemeral_id"
            ],
            "type": [
              "test123 type"
            ],
            "version": [
              "test123 version"
            ]
          }
}

How to test it?

• Start Kibana
  Create mappings

PUT /logs-test
{
  "mappings": {
    "properties": {      
      "service.name": {
        "type": "keyword"
      },
      "service.address": {
        "type": "keyword"
      },
      "service.environment": {
        "type": "keyword"
      },
      "service.ephemeral_id": {
        "type": "keyword"
      },
      "service.id": {
        "type": "keyword"
      },
      "service.node.name": {
        "type": "keyword"
      },
      "service.node.roles": {
        "type": "keyword"
      },
      "service.state": {
        "type": "keyword"
      },
      "service.type": {
        "type": "keyword"
      },
      "service.version": {
        "type": "keyword"
      },
      "@timestamp": {
        "type": "date"
      }
    }
  }
}

Create document

``` PUT /logs-test POST logs-test/_doc { "service": { "name": "test123 name", "address": "test123 address", "environment": "test123 environment", "ephemeral_id": "test123 ephemeral_id", "id": "test123 id", "node.roles": "test123 node roles", "node.name": "test123 node name", "state": "test123 state", "type": "test123 type", "version": "test123 version" }, "@timestamp": "2024-12-02T10:43:13.856Z" } ````

• Init the entity store
• Wait...
• Query the service index GET .entities.v1.latest.security_service_default/_search

Open Questions

• Can we merge this PR without first updating all other features that will use service entities?
  • If we merge it, the service engine will be installed together with other entities, but it won't provide any functionality
• Do we need an experimental flag?

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

[jcoyne-elastic]

@jcoyne-elastic Hi Jordyn. This PR captures the new Service entity fields we are targeting for the next release. Does customer 0 include or use service.name values in your events? Are these service.* fields satisfactory?

We do use service.name in some autditbeat logs, @aarju do you know when service.* is specifically used and what fields we would look to add to a service entity store if relevant?

[machadoum]

@joedatlive @jaredburgettelastic On this PR, I am collecting all values for all service fields. We might want to store only the most recent value for some fields, as we do for risk score and asset criticality. If so, let me know which field should have this behaviour.

Some candidates: service.state and service.version

[kibanamachine]

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12240705761

[elasticmachine]

:yellow_heart: Build succeeded, but was flaky

• Buildkite Build
• Commit: 281681d6c209e5516ec2463b9e22b5f30893f212

Failed CI Steps

• Jest Tests #2

Test Failures

• [job] [logs] Jest Tests #2 / StepDefinePackagePolicy default API response should display vars coming from package policy

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	14.6MB	14.6MB	+136.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	87.8KB	87.9KB	+29.0B

History

• :green_heart: Build #258218 succeeded 085ab8892fb32d6ef9261cde5ff4d265529f7845
• :green_heart: Build #257023 succeeded e48bca437b8280771a658cad8bd737c88234f87f
• :broken_heart: Build #256955 failed 50dd854a694d96b5459b04e70ab64515c5ceb32d
• :broken_heart: Build #256906 failed 6943a53c05c50863c21f285c3ab06fc61548f3bf
• :broken_heart: Build #256456 failed 9e68c9b9bbe5880aa1ea002324c723c05a63b889
• :green_heart: Build #255996 succeeded 33279efca100434584982381fb5a03ea6894b109

cc @machadoum

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 202344

Questions ?

Please refer to the Backport tool documentation

[machadoum]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation