[DOCS] improve airgapped customers missing documentation for security artifacts

111andre111 commented 1 year ago

Description

At the moment there is only a documentation for airgapped customers how to deal with the beats binary artifacts here for artifacts.elastic.co: https://www.elastic.co/guide/en/fleet/8.4/air-gapped.html#host-artifact-registry as well as for getting integrations from a local epr for epr.elastic.co: https://www.elastic.co/guide/en/fleet/8.4/air-gapped.html#air-gapped-diy-epr But the security artifacts for endpoint security for an airgapped customer are missing: The endpoints need contact to follow 2 additional domains: artifacts.security.elastic.co cloud.security.elastic.co

Acceptance Test Criteria

Make clear how the contact to domains: artifacts.security.elastic.co cloud.security.elastic.co works and how to deal with the same for airgapped customers

clarify advanced settings for the integration

clarify the contact to cloud.security.elastic.co

Notes

It would be good if we could clarify the following:

In security integration there exist several settings: windows.advanced.artifacts.global.base_url, linux.advanced.artifacts.global.base_url, mac.advanced.artifacts.global.base_url windows.advanced.artifacts.global.manifest_relative_url, linux.advanced.artifacts.global.manifest_relative_url, mac.advanced.artifacts.global.manifest_relative_url

As an example for linux looks like that: linux.advanced.artifacts.global.base_url default https://artifacts.security.elastic.co/ linux.advanced.artifacts.global.manifest_relative_url default /downloads/endpoint/manifest/artifacts-.zip

So for downloading a manifest for version 8.4.0 the complete url lookup looks like that: https://artifacts.security.elastic.co/downloads/endpoint/manifest/artifacts-8.4.0.zip Inside the manifest are a sig file and a json file.

Looking to the json file it has again a relative_url field which goes back to the base_url config field as reference:

# cat manifest.json | grep  relative_url | awk '{print "https://artifacts.security.elastic.co "$2}' | sed 's/ "//g' | sed 's/"$//g'
https://artifacts.security.elastic.co/downloads/endpoint/diagnostic-configuration-v1/11b4c72e47d4b2651915cb741305e775b3a54bf0a247e0acc3021bdc49d0e636
https://artifacts.security.elastic.co/downloads/endpoint/diagnostic-endpointelf-v1-blocklist/5093bb6536c76a5c6d90c2588e552a74845710fe8af4b8adb2967320dd771a19
https://artifacts.security.elastic.co/downloads/endpoint/diagnostic-endpointelf-v1-exceptionlist/30aeab4ad131ea70bd21e9c2e9528b4d178279aa3c8fe6de57c5645e49a185a9
https://artifacts.security.elastic.co/downloads/endpoint/diagnostic-endpointelf-v1-model/98611a30e1ce18f374d6e3d81934581f475cf644cf56434120915de660ca6a6b
https://artifacts.security.elastic.co/downloads/endpoint/diagnostic-endpointmacho-v1-blocklist/e2637d1c5dd39719423ce67b7ca8ad3ca1286b508f1336e5479f85b5a9abb228
...
...

So if the base_url setting would be changed to another main server like e.g. localserver, the manifest lookup would change to https://localserver/downloads/endpoint/manifest/artifacts-8.4.0.zip and the follow-up artifacts to e.g. https://localserver/downloads/endpoint/diagnostic-configuration-v1/11b4c72e47d4b2651915cb741305e775b3a54bf0a247e0acc3021bdc49d0e636

That way the whole endpoint infrastructure could ask an internal server. However it's still the duty of the customer to download the artifacts from internet to the local server similar to point 1.a outlined in this artifacts documentation: https://www.elastic.co/guide/en/fleet/8.4/air-gapped.html#host-artifact-registry And it's important to point out that the endpoint security binary on the endpoint initiates this connection individually, not kibana.

It might be important to point out, that all manifests get actualized under the hood with a newer manifest version possibly:
```
# cat manifest.json | jq | grep manifest_version
"manifest_version": "1.0.385",
```
and we update if a newer manifest version comes out all version artifacts to a newer manifest version. So my conclusion would be customers would need to do a script or something like that to investigate if a new manifest version came out and new artifacts would be needed to be downloaded.
Additionally it is unfortunately missing and in depth description of what all the advanced settings in the endpoint security integration are for. https://docs.elastic.co/integrations/endpoint There are some hinting blobs that give some guidance, but don't help for an in depth insight and it would be good if we set a more clear documentation here:
Finally in case an alert gets raised the endpoint initiates a connection to this domain as a first sha lookup: https://cloud.security.elastic.co/v1/lookup/ And it's important to point out again for this one, that the endpoint security binary on the endpoint initiates this connection individually, not kibana. I not even sure here, if there is any good way how to replicate this one for airgapped customers. It could be clarified that the lookup happens only on Windows and MacOS endpoints as far as I can see: https://github.com/elastic/endpoint-dev/blob/main/Plugins/Policy/Iface/plugins/Policy_Alerts.h#L21

@elastic/threat-data-services @jmikell821

111andre111 commented 1 year ago

Seems like this is ongoing here: https://github.com/elastic/security-docs/pull/2791

111andre111 commented 9 months ago

cloud.security.elastic.co I think this is needed for feature Reputation service

intxgo commented 6 months ago

there's already a dedicated doc for airgapped cases https://www.elastic.co/guide/en/security/current/offline-endpoint.html

111andre111 commented 6 months ago

Yes, but we don't nowhere outline that this endpoint mentioned is used for Reputation Service.

So partially it's still not there what has been outlined here regarding the endpoint.

intxgo commented 6 months ago

Yes, but we don't nowhere outline that this endpoint mentioned is used for Reputation Service.

So partially it's still not there what has been outlined here regarding the endpoint.

Quite the contrary, reputation services are available only in cloud subscriptions. Endpoint won't ever try to reach those URL endpoints in non-cloud deployment.

This page already clarifies the availability of the feature https://www.elastic.co/guide/en/security/current/configure-endpoint-integration-policy.html

Reputation service requires an active Platinum or Enterprise subscription and is available on cloud deployments only.

elastic / security-docs