Closed approksiu closed 5 months ago
The related integrations field has the schema:
RelatedIntegrationArray:
type: array
items:
$ref: '#/components/schemas/RelatedIntegration'
RelatedIntegration:
type: object
properties:
package:
$ref: '#/components/schemas/NonEmptyString'
version:
$ref: '#/components/schemas/NonEmptyString'
integration:
$ref: '#/components/schemas/NonEmptyString'
required:
- package
- version
for example, for existing rules:
[
{
"name": "Linux Restricted Shell Breakout via Linux Binary(s)",
"ri": [{ "package": "endpoint", "version": "^8.2.0" }]
},
{
"name": "Potential Credential Access via DCSync",
"ri": [
{ "package": "system", "version": "^1.6.4" },
{ "package": "windows", "version": "^1.5.0" }
]
},
{
"name": "Microsoft Build Engine Using an Alternate Name",
"ri": [
{ "package": "endpoint", "version": "^8.2.0" },
{ "package": "windows", "version": "^1.5.0" }
]
},
{
"name": "Unusual AWS Command for a User",
"ri": [{ "package": "aws", "version": "^2.0.0" }]
},
{
"name": "First Time Seen AWS Secret Value Accessed in Secrets Manager",
"ri": [
{ "package": "aws", "version": "^2.0.0", "integration": "cloudtrail" }
]
},
{
"name": "AWS S3 Bucket Configuration Deletion",
"ri": [
{ "package": "aws", "version": "^2.0.0", "integration": "cloudtrail" }
]
}
]
Notice that the integration
field is optional, and indeed some have it and some not. The integration
field supercedes the package
field what we show in the UI. For example, for the related integration:
{ "package": "aws", "version": "^2.0.0" }
We link to the page: /app/integrations/detail/aws-2.0.0/overview
while for the integration:
{ "package": "aws", "version": "^2.0.0", "integration": "cloudtrail" }
we link to: /app/integrations/detail/aws-2.0.0/overview?integration=cloudtrail
We can fetch all packages via the Fleet endpoint GET /kbn/api/fleet/epm/packages
, and the response looks like:
{
"items": [
{
"name": "aws",
"title": "AWS",
"version": "2.11.1",
"release": "ga",
"description": "Collect logs and metrics from Amazon Web Services (AWS) with Elastic Agent.",
"type": "integration",
"download": "/epr/aws/aws-2.11.1.zip",
"path": "/package/aws/2.11.1",
"icons": [
{
"src": "/img/logo_aws.svg",
"path": "/package/aws/2.11.1/img/logo_aws.svg",
"title": "logo aws",
"size": "32x32",
"type": "image/svg+xml"
}
],
"policy_templates": [
// more ingration policies
{
"name": "cloudtrail",
"title": "AWS CloudTrail",
"description": "Collect AWS CloudTrail logs with Elastic Agent",
"icons": [
{
"src": "/img/logo_cloudtrail.svg",
"path": "/package/aws/2.11.1/img/logo_cloudtrail.svg",
"title": "AWS Cloudtrail logo",
"size": "32x32",
"type": "image/svg+xml"
}
],
"categories": [
"security"
]
},
{
"name": "cloudwatch",
"title": "AWS CloudWatch",
"description": "Use this integration to collect logs and metrics from Amazon CloudWatch with Elastic Agent, where no out of the box integration is available.",
"icons": [
{
"src": "/img/logo_cloudwatch.svg",
"path": "/package/aws/2.11.1/img/logo_cloudwatch.svg",
"title": "AWS CloudWatch logo",
"size": "32x32",
"type": "image/svg+xml"
}
],
"categories": [
"observability",
"monitoring"
]
},
// more ingration policies
],
"conditions": {
"kibana": {
"version": "^8.10.2"
},
"elastic": {
"subscription": "basic"
}
},
"owner": {
"type": "elastic",
"github": "elastic/obs-ds-hosted-services"
},
"categories": [
"aws"
],
"signature_path": "/epr/aws/aws-2.11.1.zip.sig",
"id": "aws",
"status": "not_installed"
},
// more integration packages
{
"name": "windows",
"title": "Windows",
"version": "1.43.0",
"release": "ga",
"description": "Collect logs and metrics from Windows OS and services with Elastic Agent.",
"type": "integration",
"download": "/epr/windows/windows-1.43.0.zip",
"path": "/package/windows/1.43.0",
"icons": [
{
"src": "/img/logo_windows.svg",
"path": "/package/windows/1.43.0/img/logo_windows.svg",
"title": "logo windows",
"size": "32x32",
"type": "image/svg+xml"
}
],
"policy_templates": [
{
"name": "windows",
"title": "Windows logs and metrics",
"description": "Collect logs and metrics from Windows instances"
}
],
"conditions": {
"kibana": {
"version": "^8.8.0"
}
},
"owner": {
"github": "elastic/elastic-agent-data-plane"
},
"categories": [
"os_system",
"security"
],
"signature_path": "/epr/windows/windows-1.43.0.zip.sig",
"id": "windows",
"status": "not_installed"
},
]
}
So:
package
s, which have info that we might need such as version
and description
.policy_templates
array whose elements correspond to integrations
in our schema. For example, AWS Cloudtrail
.version
? Do we just automatically select the previous minor version, major version? Do we allow the user to type in what they want? (This is important for the Related Integration component, where we show if the installation is installed or not)For installed integrations - select current version, for non-installed - latest version. Do we show installed and non-installed integrations, and do we show only security category or others too?
Consider showing packages and integrations in the "flat" drop-down.
Do we show installed and non-installed integrations, and do we show only security category or others too?
We should show all security integrations. Would be nice to indicate which ones are installed/not installed - maybe as a follow up? cc @ARWNightingale It should be possible to search by name
Great @approksiu .
So we just met with @ARWNightingale and discussed this.
I think it makes sense to give the user a dropdown of all possible related integrations, pulling the list from Fleet. At the same time, we should show the user which of those integrations are installed, with some kind of label on each integration's row, if it is actually installed. Maybe Alex you can think an example of how this should look like.
Also, integrations that are installed should be pushed up to the top of the list, because we can presume that the user will be selecting one of those with much higher probability.
Makes sense to you @approksiu ?
@jpdjere totally, thanks!
Pinging @elastic/security-detections-response (Team:Detections and Resp)
Pinging @elastic/security-solution (Team: SecuritySolution)
@maximpn my preference is Design 2 option from the figma file. Let's pre-fill version value if possible for the user to accept or modify. Thanks!
@approksiu thank you for an update đ
I attach the current Design 2 Figma snapshot for quick reference:
@maximpn Since https://github.com/elastic/kibana/pull/178295 has been approved by reviewers (I'd like to take a look, but it shouldn't block the work on the ticket), I think we should notify the stakeholders that work on acceptance testing, exploratory testing, and documentation can start.
Once you have the PR deployed to Cloud for testing, please ping @approksiu, @ARWNightingale, and @vgomez-el in this thread and share credentials with them.
Please also open a documentation ticket for related_integrations similar to https://github.com/elastic/security-docs/issues/4917 and https://github.com/elastic/security-docs/issues/5029 and ping @joepeeples here in this thread.
It would be great if you could also try to align with the stakeholders on timing and plan a release date in Serverless.
@approksiu, @ARWNightingale, and @vgomez-el could you perform testing of https://github.com/elastic/kibana/pull/178295?
@banderror there is a problem with automatic cloud deploy via tags. I had to use another approach to deploy my PR. It's only possible to deploy in ESS env atm.
UPDATE: Serverless cloud deploy works well. I updated the information above.
UPDATE 2: It seems Serverless cloud deploy stoped working.
Great work Maxim, looks good to me!
Looks good to me!! Just a note for future enhancements maybe we should look at disabling the add + option when we have an empty input so the user can not add more than 1 empty inputs, but this is across all similar interactions elsewhere too.
@approksiu @banderror According to the issue title, user Story and @maximpn's PR description, I understand that this feature is only available for custom rules, but If I select another rule type rather than Custom rules, the related integrations option is displayed as well: New Terms:
Indicator Match:
ES|QL:
Is that intended? In case it is, it should be reflected on any of the existing issues, because the current issues description can lead to confusion.
Hi @vgomez-el!
Sorry for the confusion. Custom rules mean NOT Prebuilt rules since it's a prerequisite for Milestone 3 prebuilt rules customization. It's applicable to all rule types. Word custom
is to only highlight we don't touch prebuilt rules.
Sorry for not understanding it correctly, @maximpn. I have performed exploratory testing on your environment and the feature looks nice to me. I will perform the Kibana Upgrade scenarios and the CCS scenarios once I have the first build candidate of the 8.15 version. Please, Ping me if someone needs more details on the exploratory testing performed.
@maximpn and all, a docs PR is now open for serverless docs: https://github.com/elastic/staging-serverless-security-docs/pull/337
The feature went live in Serverless today đ Thanks to everyone for your efforts!
Epics: https://github.com/elastic/security-team/issues/1974 (internal), https://github.com/elastic/kibana/issues/174168
Summary
We need to expose the
related_integrations
field in the rule edit page and allow editing it. We should provide a list of valid security integrations for the users to choose from. It is an optional field.Background
We want to allow users to easily specify the prerequisites for their custom rules.
User story
Acceptance criteria
Designs
Figma file
Release progress
Planned release date in Serverless: Week of May 6, 2024. Planned release date in ESS: June 18 2024 (
v8.15.0
).