[Bug] [auditbeat] [winlogbeat] Running `setup` with `setup.ilm.check_exists: false` creates invalid mappings (`text` for `keyword` fields), effecting the Security app

andrew-goldstein commented 3 years ago

Running `setup` with `setup.ilm.check_exists: false` creates invalid mappings (`text` for `keyword` fields), effecting the Security app

Version: 7.9, 7.10 BC1
Operating System: Windows 10, macOS Catalina 10.15.7
Discuss Forum URL: https://discuss.elastic.co/t/elastic-7-9-1-security-siem-your-visualization-has-error-s-illegal-argument-exception/250265

As reported by a user in Discuss against 7.9.1 via winlogbeat, and reproducable in 7.10 BC1 with both winlogbeat and auditbeat, running:

winlogbeat setup

(as reported in 7.9.1, and reproducible in 7.10 BC1)

or

auditbeat setup

in 7.10 BC1 with the following configuration in winlogbeat.yml or auditbeat.yml:

setup.ilm.check_exists: false

creates incorrect mappings with type text for some keyword fields. These incorrect mappings effect the Security app in the following ways:

Errors in `7.9`

As reported in Discuss against 7.9.1:

We have a brand new cluster install to test 7.9.1 and the security features in particular.

When navigating to the Overview page we get the following error.

[illegal_argument_exception] Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [event.module] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

text-fields-are-not-optimised-for-operations

No hosts in the `Hosts` page in `7.10 BC1`

Per the screenshot below, in 7.10 BC1, the Hosts page will not list any hosts, even though searching for host.name : * in a Timeline returns results for the same period (Last 24 hours):

Why users may add this setting

In 7.10 BC1 when, for example, auditbeat setup is run, the following output is displayed:

➜  auditbeat-7.10.0-darwin-x86_64 sudo ./auditbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

Some users may see

Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

in the output above and explicitly negate the setup.ilm.check_exist setting by adding

setup.ilm.check_exists: false

to their config.

Mapping Errors

When setup is run without including

setup.ilm.check_exists: false

in the configuration, the following query run from Dev Tools:

GET /auditbeat-7.10.0/_mapping

returns the expected mapping for event.module:

        ...
        "event" : {
          "properties" : {
            "action" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },
            ...
            "module" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },
            ...
            }
          }
        },

When auditbeat setup is run with setup.ilm.check_exists: false in auditbeat.yml in 7.10 BC1, the mapping for event.module returned by GET /auditbeat-7.10.0/_mapping is:

        ...
        "event" : {
          "properties" : {
            "action" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            ...
            "module" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            ...
          }
        },

Note the extra (incorrect):

 "type" : "text",

in the definition of event.module.

Reproduction steps

1) Download the latest version of auditbeat for 7.10 BC1

2) Spin up a new 7.10 BC1 instance on Elastic Cloud

3) Login to Kibana, and navigate to the Security app

4) Click the Add data button

5) Download the version of auditbeat built for 7.10 BC1

6) After extracting the archive, edit auditbeat.yml

7) In addition to your Elastic Cloud credentials, add the following setting to auditbeat.yml:

setup.ilm.check_exists: false

8) Run

auditbeat setup

9) Ingest data by running

auditbeat -e

10) Click the Check data button on the Auditbeat add data page to confirm that Data was successfully received.

11) From the main Kibana menu, Navigate to the Security > Hosts page

Expected result

The host you ingested Auditbeat data from is displayed

Actual result

Per the screenshot below, the Hosts page shows 0 hosts:

12) Click the Timeline flyout to open a Timeline

13) Enter the following KQL:

host.name : *

Expected result

For the same date range (Last 24 hours), data from the host where you ran auditbeat appears.

14) In Kibana Dev Tools, execute the following command:

GET /auditbeat-7.10.0/_mapping

Expected result

GET /auditbeat-7.10.0/_mapping returns the expected mapping for event.module:

        ...
        "event" : {
          "properties" : {
            "action" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },
            ...
            "module" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },
            ...
            }
          }
        },

Actual result

The mapping returned by GET /auditbeat-7.10.0/_mapping:

        ...
        "event" : {
          "properties" : {
            "action" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            ...
            "module" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            ...
          }
        },

contains an extra (incorrect):

 "type" : "text",

in the definition of event.module.

CC @polyfractal

elasticmachine commented 3 years ago

Pinging @elastic/siem (Team:SIEM)

andrewkroh commented 3 years ago

@andrew-goldstein Did the index template get installed? Does GET _template/winlogbeat-7.10.0 return a template?

I suspect that having setup.ilm.check_exists: false causes critical parts of ILM and index template setup to not occur when running the setup command. This setting is typically used to operate a Beat (after setup) without read_ilm cluster privileges. You'd do this when you have the cluster locked-down as much as possible. But when you run the setup operations you normally run them under a role that has elevated privileges (including manage_ilm).

andrew-goldstein commented 3 years ago

Thanks for your reply @andrewkroh!

The output from GET _template/winlogbeat-7.10.0 is:

Output

```json { "winlogbeat-7.10.0" : { "order" : 1, "index_patterns" : [ "winlogbeat-7.10.0-*" ], "settings" : { "index" : { "lifecycle" : { "name" : "winlogbeat", "rollover_alias" : "winlogbeat-7.10.0" }, "mapping" : { "total_fields" : { "limit" : "10000" } }, "refresh_interval" : "5s", "number_of_shards" : "1", "max_docvalue_fields_search" : "200", "query" : { "default_field" : [ "message", "tags", "agent.ephemeral_id", "agent.id", "agent.name", "agent.type", "agent.version", "as.organization.name", "client.address", "client.as.organization.name", "client.domain", "client.geo.city_name", "client.geo.continent_name", "client.geo.country_iso_code", "client.geo.country_name", "client.geo.name", "client.geo.region_iso_code", "client.geo.region_name", "client.mac", "client.registered_domain", "client.top_level_domain", "client.user.domain", "client.user.email", "client.user.full_name", "client.user.group.domain", "client.user.group.id", "client.user.group.name", "client.user.hash", "client.user.id", "client.user.name", "cloud.account.id", "cloud.availability_zone", "cloud.instance.id", "cloud.instance.name", "cloud.machine.type", "cloud.provider", "cloud.region", "container.id", "container.image.name", "container.image.tag", "container.name", "container.runtime", "destination.address", "destination.as.organization.name", "destination.domain", "destination.geo.city_name", "destination.geo.continent_name", "destination.geo.country_iso_code", "destination.geo.country_name", "destination.geo.name", "destination.geo.region_iso_code", "destination.geo.region_name", "destination.mac", "destination.registered_domain", "destination.top_level_domain", "destination.user.domain", "destination.user.email", "destination.user.full_name", "destination.user.group.domain", "destination.user.group.id", "destination.user.group.name", "destination.user.hash", "destination.user.id", "destination.user.name", "dns.answers.class", "dns.answers.data", "dns.answers.name", "dns.answers.type", "dns.header_flags", "dns.id", "dns.op_code", "dns.question.class", "dns.question.name", "dns.question.registered_domain", "dns.question.subdomain", "dns.question.top_level_domain", "dns.question.type", "dns.response_code", "dns.type", "ecs.version", "error.code", "error.id", "error.message", "error.type", "event.action", "event.category", "event.code", "event.dataset", "event.hash", "event.id", "event.kind", "event.module", "event.outcome", "event.provider", "event.timezone", "event.type", "file.device", "file.directory", "file.extension", "file.gid", "file.group", "file.hash.md5", "file.hash.sha1", "file.hash.sha256", "file.hash.sha512", "file.inode", "file.mode", "file.name", "file.owner", "file.path", "file.target_path", "file.type", "file.uid", "geo.city_name", "geo.continent_name", "geo.country_iso_code", "geo.country_name", "geo.name", "geo.region_iso_code", "geo.region_name", "group.domain", "group.id", "group.name", "hash.md5", "hash.sha1", "hash.sha256", "hash.sha512", "host.architecture", "host.geo.city_name", "host.geo.continent_name", "host.geo.country_iso_code", "host.geo.country_name", "host.geo.name", "host.geo.region_iso_code", "host.geo.region_name", "host.hostname", "host.id", "host.mac", "host.name", "host.os.family", "host.os.full", "host.os.kernel", "host.os.name", "host.os.platform", "host.os.version", "host.type", "host.user.domain", "host.user.email", "host.user.full_name", "host.user.group.domain", "host.user.group.id", "host.user.group.name", "host.user.hash", "host.user.id", "host.user.name", "http.request.body.content", "http.request.method", "http.request.referrer", "http.response.body.content", "http.version", "log.level", "log.logger", "log.origin.file.name", "log.origin.function", "log.syslog.facility.name", "log.syslog.severity.name", "network.application", "network.community_id", "network.direction", "network.iana_number", "network.name", "network.protocol", "network.transport", "network.type", "observer.geo.city_name", "observer.geo.continent_name", "observer.geo.country_iso_code", "observer.geo.country_name", "observer.geo.name", "observer.geo.region_iso_code", "observer.geo.region_name", "observer.hostname", "observer.mac", "observer.name", "observer.os.family", "observer.os.full", "observer.os.kernel", "observer.os.name", "observer.os.platform", "observer.os.version", "observer.product", "observer.serial_number", "observer.type", "observer.vendor", "observer.version", "organization.id", "organization.name", "os.family", "os.full", "os.kernel", "os.name", "os.platform", "os.version", "package.architecture", "package.checksum", "package.description", "package.install_scope", "package.license", "package.name", "package.path", "package.version", "process.args", "text", "process.executable", "process.hash.md5", "process.hash.sha1", "process.hash.sha256", "process.hash.sha512", "process.name", "text", "text", "text", "text", "text", "process.thread.name", "process.title", "process.working_directory", "server.address", "server.as.organization.name", "server.domain", "server.geo.city_name", "server.geo.continent_name", "server.geo.country_iso_code", "server.geo.country_name", "server.geo.name", "server.geo.region_iso_code", "server.geo.region_name", "server.mac", "server.registered_domain", "server.top_level_domain", "server.user.domain", "server.user.email", "server.user.full_name", "server.user.group.domain", "server.user.group.id", "server.user.group.name", "server.user.hash", "server.user.id", "server.user.name", "service.ephemeral_id", "service.id", "service.name", "service.node.name", "service.state", "service.type", "service.version", "source.address", "source.as.organization.name", "source.domain", "source.geo.city_name", "source.geo.continent_name", "source.geo.country_iso_code", "source.geo.country_name", "source.geo.name", "source.geo.region_iso_code", "source.geo.region_name", "source.mac", "source.registered_domain", "source.top_level_domain", "source.user.domain", "source.user.email", "source.user.full_name", "source.user.group.domain", "source.user.group.id", "source.user.group.name", "source.user.hash", "source.user.id", "source.user.name", "threat.framework", "threat.tactic.id", "threat.tactic.name", "threat.tactic.reference", "threat.technique.id", "threat.technique.name", "threat.technique.reference", "tracing.trace.id", "tracing.transaction.id", "url.domain", "url.extension", "url.fragment", "url.full", "url.original", "url.password", "url.path", "url.query", "url.registered_domain", "url.scheme", "url.top_level_domain", "url.username", "user.domain", "user.email", "user.full_name", "user.group.domain", "user.group.id", "user.group.name", "user.hash", "user.id", "user.name", "user_agent.device.name", "user_agent.name", "text", "user_agent.original", "user_agent.os.family", "user_agent.os.full", "user_agent.os.kernel", "user_agent.os.name", "user_agent.os.platform", "user_agent.os.version", "user_agent.version", "text", "agent.hostname", "timeseries.instance", "cloud.image.id", "host.os.build", "host.os.codename", "kubernetes.pod.name", "kubernetes.pod.uid", "kubernetes.namespace", "kubernetes.node.name", "kubernetes.replicaset.name", "kubernetes.deployment.name", "kubernetes.statefulset.name", "kubernetes.container.name", "kubernetes.container.image", "jolokia.agent.version", "jolokia.agent.id", "jolokia.server.product", "jolokia.server.version", "jolokia.server.vendor", "jolokia.url", "event.original", "winlog.api", "winlog.activity_id", "winlog.computer_name", "winlog.event_data.AuthenticationPackageName", "winlog.event_data.Binary", "winlog.event_data.BitlockerUserInputTime", "winlog.event_data.BootMode", "winlog.event_data.BootType", "winlog.event_data.BuildVersion", "winlog.event_data.Company", "winlog.event_data.CorruptionActionState", "winlog.event_data.CreationUtcTime", "winlog.event_data.Description", "winlog.event_data.Detail", "winlog.event_data.DeviceName", "winlog.event_data.DeviceNameLength", "winlog.event_data.DeviceTime", "winlog.event_data.DeviceVersionMajor", "winlog.event_data.DeviceVersionMinor", "winlog.event_data.DriveName", "winlog.event_data.DriverName", "winlog.event_data.DriverNameLength", "winlog.event_data.DwordVal", "winlog.event_data.EntryCount", "winlog.event_data.ExtraInfo", "winlog.event_data.FailureName", "winlog.event_data.FailureNameLength", "winlog.event_data.FileVersion", "winlog.event_data.FinalStatus", "winlog.event_data.Group", "winlog.event_data.IdleImplementation", "winlog.event_data.IdleStateCount", "winlog.event_data.ImpersonationLevel", "winlog.event_data.IntegrityLevel", "winlog.event_data.IpAddress", "winlog.event_data.IpPort", "winlog.event_data.KeyLength", "winlog.event_data.LastBootGood", "winlog.event_data.LastShutdownGood", "winlog.event_data.LmPackageName", "winlog.event_data.LogonGuid", "winlog.event_data.LogonId", "winlog.event_data.LogonProcessName", "winlog.event_data.LogonType", "winlog.event_data.MajorVersion", "winlog.event_data.MaximumPerformancePercent", "winlog.event_data.MemberName", "winlog.event_data.MemberSid", "winlog.event_data.MinimumPerformancePercent", "winlog.event_data.MinimumThrottlePercent", "winlog.event_data.MinorVersion", "winlog.event_data.NewProcessId", "winlog.event_data.NewProcessName", "winlog.event_data.NewSchemeGuid", "winlog.event_data.NewTime", "winlog.event_data.NominalFrequency", "winlog.event_data.Number", "winlog.event_data.OldSchemeGuid", "winlog.event_data.OldTime", "winlog.event_data.OriginalFileName", "winlog.event_data.Path", "winlog.event_data.PerformanceImplementation", "winlog.event_data.PreviousCreationUtcTime", "winlog.event_data.PreviousTime", "winlog.event_data.PrivilegeList", "winlog.event_data.ProcessId", "winlog.event_data.ProcessName", "winlog.event_data.ProcessPath", "winlog.event_data.ProcessPid", "winlog.event_data.Product", "winlog.event_data.PuaCount", "winlog.event_data.PuaPolicyId", "winlog.event_data.QfeVersion", "winlog.event_data.Reason", "winlog.event_data.SchemaVersion", "winlog.event_data.ScriptBlockText", "winlog.event_data.ServiceName", "winlog.event_data.ServiceVersion", "winlog.event_data.ShutdownActionType", "winlog.event_data.ShutdownEventCode", "winlog.event_data.ShutdownReason", "winlog.event_data.Signature", "winlog.event_data.SignatureStatus", "winlog.event_data.Signed", "winlog.event_data.StartTime", "winlog.event_data.State", "winlog.event_data.Status", "winlog.event_data.StopTime", "winlog.event_data.SubjectDomainName", "winlog.event_data.SubjectLogonId", "winlog.event_data.SubjectUserName", "winlog.event_data.SubjectUserSid", "winlog.event_data.TSId", "winlog.event_data.TargetDomainName", "winlog.event_data.TargetInfo", "winlog.event_data.TargetLogonGuid", "winlog.event_data.TargetLogonId", "winlog.event_data.TargetServerName", "winlog.event_data.TargetUserName", "winlog.event_data.TargetUserSid", "winlog.event_data.TerminalSessionId", "winlog.event_data.TokenElevationType", "winlog.event_data.TransmittedServices", "winlog.event_data.UserSid", "winlog.event_data.Version", "winlog.event_data.Workstation", "winlog.event_data.param1", "winlog.event_data.param2", "winlog.event_data.param3", "winlog.event_data.param4", "winlog.event_data.param5", "winlog.event_data.param6", "winlog.event_data.param7", "winlog.event_data.param8", "winlog.event_id", "winlog.keywords", "winlog.channel", "winlog.record_id", "winlog.related_activity_id", "winlog.opcode", "winlog.provider_guid", "winlog.provider_name", "winlog.task", "winlog.user.identifier", "winlog.user.name", "winlog.user.domain", "winlog.user.type", "fields.*" ] } } }, "mappings" : { "_meta" : { "beat" : "winlogbeat", "version" : "7.10.0" }, "dynamic_templates" : [ { "labels" : { "path_match" : "labels.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "string" } }, { "container.labels" : { "path_match" : "container.labels.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "string" } }, { "fields" : { "path_match" : "fields.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "string" } }, { "docker.container.labels" : { "path_match" : "docker.container.labels.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "string" } }, { "kubernetes.labels.*" : { "path_match" : "kubernetes.labels.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "*" } }, { "kubernetes.annotations.*" : { "path_match" : "kubernetes.annotations.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "*" } }, { "winlog.event_data" : { "path_match" : "winlog.event_data.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "string" } }, { "winlog.user_data" : { "path_match" : "winlog.user_data.*", "mapping" : { "type" : "keyword" }, "match_mapping_type" : "string" } }, { "strings_as_keyword" : { "mapping" : { "ignore_above" : 1024, "type" : "keyword" }, "match_mapping_type" : "string" } } ], "date_detection" : false, "properties" : { "container" : { "properties" : { "image" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "tag" : { "ignore_above" : 1024, "type" : "keyword" } } }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "runtime" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "labels" : { "type" : "object" } } }, "kubernetes" : { "properties" : { "container" : { "properties" : { "image" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "node" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "pod" : { "properties" : { "uid" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "statefulset" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "namespace" : { "ignore_above" : 1024, "type" : "keyword" }, "annotations" : { "properties" : { "*" : { "type" : "object" } } }, "replicaset" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "deployment" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "labels" : { "properties" : { "*" : { "type" : "object" } } } } }, "agent" : { "properties" : { "hostname" : { "ignore_above" : 1024, "type" : "keyword" }, "build" : { "properties" : { "original" : { "ignore_above" : 1024, "type" : "keyword" } } }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "ephemeral_id" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" } } }, "source" : { "properties" : { "nat" : { "properties" : { "port" : { "type" : "long" }, "ip" : { "type" : "ip" } } }, "address" : { "ignore_above" : 1024, "type" : "keyword" }, "top_level_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "ip" : { "type" : "ip" }, "mac" : { "ignore_above" : 1024, "type" : "keyword" }, "packets" : { "type" : "long" }, "geo" : { "properties" : { "region_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "continent_name" : { "ignore_above" : 1024, "type" : "keyword" }, "city_name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "country_name" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "location" : { "type" : "geo_point" }, "region_name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "as" : { "properties" : { "number" : { "type" : "long" }, "organization" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } } } }, "registered_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "port" : { "type" : "long" }, "bytes" : { "type" : "long" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "user" : { "properties" : { "full_name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "roles" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" }, "email" : { "ignore_above" : 1024, "type" : "keyword" }, "group" : { "properties" : { "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } } } }, "cloud" : { "properties" : { "image" : { "properties" : { "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "availability_zone" : { "ignore_above" : 1024, "type" : "keyword" }, "instance" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "provider" : { "ignore_above" : 1024, "type" : "keyword" }, "machine" : { "properties" : { "type" : { "ignore_above" : 1024, "type" : "keyword" } } }, "project" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "region" : { "ignore_above" : 1024, "type" : "keyword" }, "account" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "observer" : { "properties" : { "product" : { "ignore_above" : 1024, "type" : "keyword" }, "os" : { "properties" : { "kernel" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "family" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "platform" : { "ignore_above" : 1024, "type" : "keyword" }, "full" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } }, "ip" : { "type" : "ip" }, "serial_number" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "mac" : { "ignore_above" : 1024, "type" : "keyword" }, "egress" : { "type" : "object", "properties" : { "vlan" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "zone" : { "ignore_above" : 1024, "type" : "keyword" }, "interface" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "alias" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "geo" : { "properties" : { "region_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "continent_name" : { "ignore_above" : 1024, "type" : "keyword" }, "city_name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_name" : { "ignore_above" : 1024, "type" : "keyword" }, "location" : { "type" : "geo_point" }, "region_name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "ingress" : { "type" : "object", "properties" : { "vlan" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "zone" : { "ignore_above" : 1024, "type" : "keyword" }, "interface" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "alias" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "hostname" : { "ignore_above" : 1024, "type" : "keyword" }, "vendor" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "ecs" : { "properties" : { "version" : { "ignore_above" : 1024, "type" : "keyword" } } }, "timeseries" : { "properties" : { "instance" : { "ignore_above" : 1024, "type" : "keyword" } } }, "code_signature" : { "properties" : { "valid" : { "type" : "boolean" }, "trusted" : { "type" : "boolean" }, "subject_name" : { "ignore_above" : 1024, "type" : "keyword" }, "exists" : { "type" : "boolean" }, "status" : { "ignore_above" : 1024, "type" : "keyword" } } }, "host" : { "properties" : { "os" : { "properties" : { "build" : { "ignore_above" : 1024, "type" : "keyword" }, "kernel" : { "ignore_above" : 1024, "type" : "keyword" }, "codename" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "family" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "platform" : { "ignore_above" : 1024, "type" : "keyword" }, "full" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } }, "ip" : { "type" : "ip" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "mac" : { "ignore_above" : 1024, "type" : "keyword" }, "uptime" : { "type" : "long" }, "geo" : { "properties" : { "region_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "continent_name" : { "ignore_above" : 1024, "type" : "keyword" }, "city_name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "country_name" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "location" : { "type" : "geo_point" }, "region_name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "hostname" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "containerized" : { "type" : "boolean" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "user" : { "properties" : { "full_name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "roles" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" }, "email" : { "ignore_above" : 1024, "type" : "keyword" }, "group" : { "properties" : { "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "architecture" : { "ignore_above" : 1024, "type" : "keyword" } } }, "group" : { "properties" : { "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "tracing" : { "properties" : { "trace" : { "properties" : { "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "transaction" : { "properties" : { "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "span" : { "properties" : { "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "package" : { "properties" : { "installed" : { "type" : "date" }, "build_version" : { "ignore_above" : 1024, "type" : "keyword" }, "description" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "reference" : { "ignore_above" : 1024, "type" : "keyword" }, "path" : { "ignore_above" : 1024, "type" : "keyword" }, "license" : { "ignore_above" : 1024, "type" : "keyword" }, "install_scope" : { "ignore_above" : 1024, "type" : "keyword" }, "size" : { "type" : "long" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "checksum" : { "ignore_above" : 1024, "type" : "keyword" }, "architecture" : { "ignore_above" : 1024, "type" : "keyword" } } }, "dns" : { "properties" : { "op_code" : { "ignore_above" : 1024, "type" : "keyword" }, "resolved_ip" : { "type" : "ip" }, "response_code" : { "ignore_above" : 1024, "type" : "keyword" }, "question" : { "properties" : { "registered_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "top_level_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "subdomain" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "class" : { "ignore_above" : 1024, "type" : "keyword" } } }, "answers" : { "type" : "object", "properties" : { "data" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "class" : { "ignore_above" : 1024, "type" : "keyword" }, "ttl" : { "type" : "long" } } }, "header_flags" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" } } }, "vulnerability" : { "properties" : { "reference" : { "ignore_above" : 1024, "type" : "keyword" }, "severity" : { "ignore_above" : 1024, "type" : "keyword" }, "score" : { "properties" : { "environmental" : { "type" : "float" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "temporal" : { "type" : "float" }, "base" : { "type" : "float" } } }, "report_id" : { "ignore_above" : 1024, "type" : "keyword" }, "scanner" : { "properties" : { "vendor" : { "ignore_above" : 1024, "type" : "keyword" } } }, "description" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "enumeration" : { "ignore_above" : 1024, "type" : "keyword" }, "category" : { "ignore_above" : 1024, "type" : "keyword" }, "classification" : { "ignore_above" : 1024, "type" : "keyword" } } }, "tags" : { "ignore_above" : 1024, "type" : "keyword" }, "labels" : { "type" : "object" }, "x509" : { "properties" : { "not_after" : { "type" : "date" }, "public_key_exponent" : { "index" : false, "type" : "long" }, "not_before" : { "type" : "date" }, "subject" : { "properties" : { "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "country" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } }, "public_key_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_curve" : { "ignore_above" : 1024, "type" : "keyword" }, "signature_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_size" : { "type" : "long" }, "serial_number" : { "ignore_above" : 1024, "type" : "keyword" }, "version_number" : { "ignore_above" : 1024, "type" : "keyword" }, "alternative_names" : { "ignore_above" : 1024, "type" : "keyword" }, "issuer" : { "properties" : { "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "country" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "as" : { "properties" : { "number" : { "type" : "long" }, "organization" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } } } }, "http" : { "properties" : { "request" : { "properties" : { "referrer" : { "ignore_above" : 1024, "type" : "keyword" }, "method" : { "ignore_above" : 1024, "type" : "keyword" }, "bytes" : { "type" : "long" }, "body" : { "properties" : { "bytes" : { "type" : "long" }, "content" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } } } }, "response" : { "properties" : { "status_code" : { "type" : "long" }, "bytes" : { "type" : "long" }, "body" : { "properties" : { "bytes" : { "type" : "long" }, "content" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } } } }, "version" : { "ignore_above" : 1024, "type" : "keyword" } } }, "fields" : { "type" : "object" }, "hash" : { "properties" : { "sha1" : { "ignore_above" : 1024, "type" : "keyword" }, "sha256" : { "ignore_above" : 1024, "type" : "keyword" }, "sha512" : { "ignore_above" : 1024, "type" : "keyword" }, "md5" : { "ignore_above" : 1024, "type" : "keyword" } } }, "server" : { "properties" : { "nat" : { "properties" : { "port" : { "type" : "long" }, "ip" : { "type" : "ip" } } }, "address" : { "ignore_above" : 1024, "type" : "keyword" }, "top_level_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "ip" : { "type" : "ip" }, "mac" : { "ignore_above" : 1024, "type" : "keyword" }, "packets" : { "type" : "long" }, "geo" : { "properties" : { "region_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "continent_name" : { "ignore_above" : 1024, "type" : "keyword" }, "city_name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_name" : { "ignore_above" : 1024, "type" : "keyword" }, "region_name" : { "ignore_above" : 1024, "type" : "keyword" }, "location" : { "type" : "geo_point" } } }, "as" : { "properties" : { "number" : { "type" : "long" }, "organization" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } } } }, "registered_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "port" : { "type" : "long" }, "bytes" : { "type" : "long" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "user" : { "properties" : { "full_name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "roles" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" }, "email" : { "ignore_above" : 1024, "type" : "keyword" }, "group" : { "properties" : { "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } } } }, "log" : { "properties" : { "original" : { "ignore_above" : 1024, "index" : false, "type" : "keyword" }, "file" : { "properties" : { "path" : { "ignore_above" : 1024, "type" : "keyword" } } }, "level" : { "ignore_above" : 1024, "type" : "keyword" }, "logger" : { "ignore_above" : 1024, "type" : "keyword" }, "origin" : { "properties" : { "file" : { "properties" : { "line" : { "type" : "long" }, "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "function" : { "ignore_above" : 1024, "type" : "keyword" } } }, "syslog" : { "type" : "object", "properties" : { "severity" : { "properties" : { "code" : { "type" : "long" }, "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "priority" : { "type" : "long" }, "facility" : { "properties" : { "code" : { "type" : "long" }, "name" : { "ignore_above" : 1024, "type" : "keyword" } } } } } } }, "destination" : { "properties" : { "nat" : { "properties" : { "port" : { "type" : "long" }, "ip" : { "type" : "ip" } } }, "address" : { "ignore_above" : 1024, "type" : "keyword" }, "top_level_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "ip" : { "type" : "ip" }, "mac" : { "ignore_above" : 1024, "type" : "keyword" }, "packets" : { "type" : "long" }, "geo" : { "properties" : { "region_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "continent_name" : { "ignore_above" : 1024, "type" : "keyword" }, "city_name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "country_name" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "location" : { "type" : "geo_point" }, "region_name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "registered_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "as" : { "properties" : { "number" : { "type" : "long" }, "organization" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } } } }, "port" : { "type" : "long" }, "bytes" : { "type" : "long" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "user" : { "properties" : { "full_name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "roles" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "email" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" }, "group" : { "properties" : { "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } } } }, "rule" : { "properties" : { "reference" : { "ignore_above" : 1024, "type" : "keyword" }, "license" : { "ignore_above" : 1024, "type" : "keyword" }, "author" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "ruleset" : { "ignore_above" : 1024, "type" : "keyword" }, "description" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "category" : { "ignore_above" : 1024, "type" : "keyword" }, "uuid" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" } } }, "error" : { "properties" : { "code" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "stack_trace" : { "ignore_above" : 1024, "index" : false, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "message" : { "norms" : false, "type" : "text" }, "type" : { "ignore_above" : 1024, "type" : "keyword" } } }, "interface" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "alias" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "network" : { "properties" : { "transport" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "inner" : { "type" : "object", "properties" : { "vlan" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "packets" : { "type" : "long" }, "protocol" : { "ignore_above" : 1024, "type" : "keyword" }, "community_id" : { "ignore_above" : 1024, "type" : "keyword" }, "forwarded_ip" : { "type" : "ip" }, "vlan" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "application" : { "ignore_above" : 1024, "type" : "keyword" }, "bytes" : { "type" : "long" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "iana_number" : { "ignore_above" : 1024, "type" : "keyword" }, "direction" : { "ignore_above" : 1024, "type" : "keyword" } } }, "docker" : { "properties" : { "container" : { "properties" : { "labels" : { "type" : "object" } } } } }, "geo" : { "properties" : { "continent_name" : { "ignore_above" : 1024, "type" : "keyword" }, "region_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "city_name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "country_name" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "location" : { "type" : "geo_point" }, "region_name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "file" : { "properties" : { "extension" : { "ignore_above" : 1024, "type" : "keyword" }, "gid" : { "ignore_above" : 1024, "type" : "keyword" }, "drive_letter" : { "ignore_above" : 1, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "accessed" : { "type" : "date" }, "mtime" : { "type" : "date" }, "directory" : { "ignore_above" : 1024, "type" : "keyword" }, "mode" : { "ignore_above" : 1024, "type" : "keyword" }, "inode" : { "ignore_above" : 1024, "type" : "keyword" }, "path" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "uid" : { "ignore_above" : 1024, "type" : "keyword" }, "code_signature" : { "properties" : { "valid" : { "type" : "boolean" }, "trusted" : { "type" : "boolean" }, "subject_name" : { "ignore_above" : 1024, "type" : "keyword" }, "exists" : { "type" : "boolean" }, "status" : { "ignore_above" : 1024, "type" : "keyword" } } }, "ctime" : { "type" : "date" }, "group" : { "ignore_above" : 1024, "type" : "keyword" }, "owner" : { "ignore_above" : 1024, "type" : "keyword" }, "created" : { "type" : "date" }, "target_path" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "x509" : { "properties" : { "not_after" : { "type" : "date" }, "public_key_exponent" : { "index" : false, "type" : "long" }, "subject" : { "properties" : { "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "country" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } }, "not_before" : { "type" : "date" }, "public_key_curve" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "signature_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "version_number" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_size" : { "type" : "long" }, "serial_number" : { "ignore_above" : 1024, "type" : "keyword" }, "alternative_names" : { "ignore_above" : 1024, "type" : "keyword" }, "issuer" : { "properties" : { "country" : { "ignore_above" : 1024, "type" : "keyword" }, "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "size" : { "type" : "long" }, "pe" : { "properties" : { "file_version" : { "ignore_above" : 1024, "type" : "keyword" }, "product" : { "ignore_above" : 1024, "type" : "keyword" }, "imphash" : { "ignore_above" : 1024, "type" : "keyword" }, "description" : { "ignore_above" : 1024, "type" : "keyword" }, "original_file_name" : { "ignore_above" : 1024, "type" : "keyword" }, "company" : { "ignore_above" : 1024, "type" : "keyword" }, "architecture" : { "ignore_above" : 1024, "type" : "keyword" } } }, "mime_type" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "attributes" : { "ignore_above" : 1024, "type" : "keyword" }, "device" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "properties" : { "sha1" : { "ignore_above" : 1024, "type" : "keyword" }, "sha256" : { "ignore_above" : 1024, "type" : "keyword" }, "sha512" : { "ignore_above" : 1024, "type" : "keyword" }, "md5" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "vlan" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "related" : { "properties" : { "hosts" : { "ignore_above" : 1024, "type" : "keyword" }, "ip" : { "type" : "ip" }, "user" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" } } }, "client" : { "properties" : { "nat" : { "properties" : { "port" : { "type" : "long" }, "ip" : { "type" : "ip" } } }, "address" : { "ignore_above" : 1024, "type" : "keyword" }, "top_level_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "ip" : { "type" : "ip" }, "mac" : { "ignore_above" : 1024, "type" : "keyword" }, "packets" : { "type" : "long" }, "geo" : { "properties" : { "region_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "continent_name" : { "ignore_above" : 1024, "type" : "keyword" }, "city_name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_iso_code" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "country_name" : { "ignore_above" : 1024, "type" : "keyword" }, "region_name" : { "ignore_above" : 1024, "type" : "keyword" }, "location" : { "type" : "geo_point" } } }, "as" : { "properties" : { "number" : { "type" : "long" }, "organization" : { "properties" : { "name" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" } } } } }, "registered_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "port" : { "type" : "long" }, "bytes" : { "type" : "long" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "user" : { "properties" : { "full_name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "roles" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "email" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" }, "group" : { "properties" : { "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } } } }, "event" : { "properties" : { "reason" : { "ignore_above" : 1024, "type" : "keyword" }, "code" : { "ignore_above" : 1024, "type" : "keyword" }, "timezone" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "duration" : { "type" : "long" }, "reference" : { "ignore_above" : 1024, "type" : "keyword" }, "ingested" : { "type" : "date" }, "provider" : { "ignore_above" : 1024, "type" : "keyword" }, "action" : { "ignore_above" : 1024, "type" : "keyword" }, "end" : { "type" : "date" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "outcome" : { "ignore_above" : 1024, "type" : "keyword" }, "severity" : { "type" : "long" }, "original" : { "ignore_above" : 1024, "type" : "keyword" }, "risk_score" : { "type" : "float" }, "kind" : { "ignore_above" : 1024, "type" : "keyword" }, "created" : { "type" : "date" }, "module" : { "ignore_above" : 1024, "type" : "keyword" }, "start" : { "type" : "date" }, "url" : { "ignore_above" : 1024, "type" : "keyword" }, "sequence" : { "type" : "long" }, "risk_score_norm" : { "type" : "float" }, "category" : { "ignore_above" : 1024, "type" : "keyword" }, "dataset" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" } } }, "user_agent" : { "properties" : { "original" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "os" : { "properties" : { "kernel" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "family" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "platform" : { "ignore_above" : 1024, "type" : "keyword" }, "full" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "device" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "jolokia" : { "properties" : { "server" : { "properties" : { "product" : { "ignore_above" : 1024, "type" : "keyword" }, "vendor" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" } } }, "agent" : { "properties" : { "id" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" } } }, "secured" : { "type" : "boolean" }, "url" : { "ignore_above" : 1024, "type" : "keyword" } } }, "registry" : { "properties" : { "hive" : { "ignore_above" : 1024, "type" : "keyword" }, "path" : { "ignore_above" : 1024, "type" : "keyword" }, "data" : { "properties" : { "strings" : { "ignore_above" : 1024, "type" : "keyword" }, "bytes" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" } } }, "value" : { "ignore_above" : 1024, "type" : "keyword" }, "key" : { "ignore_above" : 1024, "type" : "keyword" } } }, "process" : { "properties" : { "parent" : { "properties" : { "pgid" : { "type" : "long" }, "start" : { "type" : "date" }, "working_directory" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "pid" : { "type" : "long" }, "thread" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "type" : "long" } } }, "title" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "entity_id" : { "ignore_above" : 1024, "type" : "keyword" }, "executable" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "ppid" : { "type" : "long" }, "uptime" : { "type" : "long" }, "args" : { "ignore_above" : 1024, "type" : "keyword" }, "code_signature" : { "properties" : { "valid" : { "type" : "boolean" }, "trusted" : { "type" : "boolean" }, "subject_name" : { "ignore_above" : 1024, "type" : "keyword" }, "exists" : { "type" : "boolean" }, "status" : { "ignore_above" : 1024, "type" : "keyword" } } }, "pe" : { "properties" : { "file_version" : { "ignore_above" : 1024, "type" : "keyword" }, "product" : { "ignore_above" : 1024, "type" : "keyword" }, "imphash" : { "ignore_above" : 1024, "type" : "keyword" }, "description" : { "ignore_above" : 1024, "type" : "keyword" }, "original_file_name" : { "ignore_above" : 1024, "type" : "keyword" }, "company" : { "ignore_above" : 1024, "type" : "keyword" }, "architecture" : { "ignore_above" : 1024, "type" : "keyword" } } }, "name" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "exit_code" : { "type" : "long" }, "args_count" : { "type" : "long" }, "hash" : { "properties" : { "sha1" : { "ignore_above" : 1024, "type" : "keyword" }, "sha256" : { "ignore_above" : 1024, "type" : "keyword" }, "sha512" : { "ignore_above" : 1024, "type" : "keyword" }, "md5" : { "ignore_above" : 1024, "type" : "keyword" } } }, "command_line" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" } } }, "pgid" : { "type" : "long" }, "start" : { "type" : "date" }, "pid" : { "type" : "long" }, "working_directory" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "thread" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "type" : "long" } } }, "entity_id" : { "ignore_above" : 1024, "type" : "keyword" }, "title" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "executable" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "uptime" : { "type" : "long" }, "ppid" : { "type" : "long" }, "args" : { "ignore_above" : 1024, "type" : "keyword" }, "code_signature" : { "properties" : { "valid" : { "type" : "boolean" }, "trusted" : { "type" : "boolean" }, "subject_name" : { "ignore_above" : 1024, "type" : "keyword" }, "exists" : { "type" : "boolean" }, "status" : { "ignore_above" : 1024, "type" : "keyword" } } }, "pe" : { "properties" : { "file_version" : { "ignore_above" : 1024, "type" : "keyword" }, "product" : { "ignore_above" : 1024, "type" : "keyword" }, "imphash" : { "ignore_above" : 1024, "type" : "keyword" }, "description" : { "ignore_above" : 1024, "type" : "keyword" }, "original_file_name" : { "ignore_above" : 1024, "type" : "keyword" }, "company" : { "ignore_above" : 1024, "type" : "keyword" }, "architecture" : { "ignore_above" : 1024, "type" : "keyword" } } }, "exit_code" : { "type" : "long" }, "name" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "args_count" : { "type" : "long" }, "command_line" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "hash" : { "properties" : { "sha1" : { "ignore_above" : 1024, "type" : "keyword" }, "sha256" : { "ignore_above" : 1024, "type" : "keyword" }, "sha512" : { "ignore_above" : 1024, "type" : "keyword" }, "md5" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "winlog" : { "properties" : { "related_activity_id" : { "ignore_above" : 1024, "type" : "keyword" }, "computer_name" : { "ignore_above" : 1024, "type" : "keyword" }, "process" : { "properties" : { "pid" : { "type" : "long" }, "thread" : { "properties" : { "id" : { "type" : "long" } } } } }, "keywords" : { "ignore_above" : 1024, "type" : "keyword" }, "channel" : { "ignore_above" : 1024, "type" : "keyword" }, "event_data" : { "properties" : { "SignatureStatus" : { "ignore_above" : 1024, "type" : "keyword" }, "DeviceTime" : { "ignore_above" : 1024, "type" : "keyword" }, "ProcessName" : { "ignore_above" : 1024, "type" : "keyword" }, "LogonGuid" : { "ignore_above" : 1024, "type" : "keyword" }, "OriginalFileName" : { "ignore_above" : 1024, "type" : "keyword" }, "BootMode" : { "ignore_above" : 1024, "type" : "keyword" }, "Product" : { "ignore_above" : 1024, "type" : "keyword" }, "TargetLogonGuid" : { "ignore_above" : 1024, "type" : "keyword" }, "FileVersion" : { "ignore_above" : 1024, "type" : "keyword" }, "StopTime" : { "ignore_above" : 1024, "type" : "keyword" }, "Status" : { "ignore_above" : 1024, "type" : "keyword" }, "KeyLength" : { "ignore_above" : 1024, "type" : "keyword" }, "CorruptionActionState" : { "ignore_above" : 1024, "type" : "keyword" }, "TargetInfo" : { "ignore_above" : 1024, "type" : "keyword" }, "PreviousCreationUtcTime" : { "ignore_above" : 1024, "type" : "keyword" }, "ServiceVersion" : { "ignore_above" : 1024, "type" : "keyword" }, "SubjectUserSid" : { "ignore_above" : 1024, "type" : "keyword" }, "PerformanceImplementation" : { "ignore_above" : 1024, "type" : "keyword" }, "TargetUserSid" : { "ignore_above" : 1024, "type" : "keyword" }, "Group" : { "ignore_above" : 1024, "type" : "keyword" }, "Description" : { "ignore_above" : 1024, "type" : "keyword" }, "ShutdownActionType" : { "ignore_above" : 1024, "type" : "keyword" }, "DwordVal" : { "ignore_above" : 1024, "type" : "keyword" }, "ProcessPid" : { "ignore_above" : 1024, "type" : "keyword" }, "DeviceVersionMajor" : { "ignore_above" : 1024, "type" : "keyword" }, "ScriptBlockText" : { "ignore_above" : 1024, "type" : "keyword" }, "TransmittedServices" : { "ignore_above" : 1024, "type" : "keyword" }, "MaximumPerformancePercent" : { "ignore_above" : 1024, "type" : "keyword" }, "NewTime" : { "ignore_above" : 1024, "type" : "keyword" }, "FinalStatus" : { "ignore_above" : 1024, "type" : "keyword" }, "IdleStateCount" : { "ignore_above" : 1024, "type" : "keyword" }, "Path" : { "ignore_above" : 1024, "type" : "keyword" }, "MajorVersion" : { "ignore_above" : 1024, "type" : "keyword" }, "SchemaVersion" : { "ignore_above" : 1024, "type" : "keyword" }, "TokenElevationType" : { "ignore_above" : 1024, "type" : "keyword" }, "MinorVersion" : { "ignore_above" : 1024, "type" : "keyword" }, "SubjectLogonId" : { "ignore_above" : 1024, "type" : "keyword" }, "IdleImplementation" : { "ignore_above" : 1024, "type" : "keyword" }, "ProcessPath" : { "ignore_above" : 1024, "type" : "keyword" }, "QfeVersion" : { "ignore_above" : 1024, "type" : "keyword" }, "DeviceVersionMinor" : { "ignore_above" : 1024, "type" : "keyword" }, "OldTime" : { "ignore_above" : 1024, "type" : "keyword" }, "IpAddress" : { "ignore_above" : 1024, "type" : "keyword" }, "DeviceName" : { "ignore_above" : 1024, "type" : "keyword" }, "Company" : { "ignore_above" : 1024, "type" : "keyword" }, "PuaPolicyId" : { "ignore_above" : 1024, "type" : "keyword" }, "IntegrityLevel" : { "ignore_above" : 1024, "type" : "keyword" }, "LastShutdownGood" : { "ignore_above" : 1024, "type" : "keyword" }, "IpPort" : { "ignore_above" : 1024, "type" : "keyword" }, "DriverNameLength" : { "ignore_above" : 1024, "type" : "keyword" }, "LmPackageName" : { "ignore_above" : 1024, "type" : "keyword" }, "UserSid" : { "ignore_above" : 1024, "type" : "keyword" }, "LastBootGood" : { "ignore_above" : 1024, "type" : "keyword" }, "PuaCount" : { "ignore_above" : 1024, "type" : "keyword" }, "Version" : { "ignore_above" : 1024, "type" : "keyword" }, "Signed" : { "ignore_above" : 1024, "type" : "keyword" }, "StartTime" : { "ignore_above" : 1024, "type" : "keyword" }, "ShutdownEventCode" : { "ignore_above" : 1024, "type" : "keyword" }, "NewProcessName" : { "ignore_above" : 1024, "type" : "keyword" }, "FailureNameLength" : { "ignore_above" : 1024, "type" : "keyword" }, "ServiceName" : { "ignore_above" : 1024, "type" : "keyword" }, "State" : { "ignore_above" : 1024, "type" : "keyword" }, "PreviousTime" : { "ignore_above" : 1024, "type" : "keyword" }, "BootType" : { "ignore_above" : 1024, "type" : "keyword" }, "Binary" : { "ignore_above" : 1024, "type" : "keyword" }, "ImpersonationLevel" : { "ignore_above" : 1024, "type" : "keyword" }, "TargetUserName" : { "ignore_above" : 1024, "type" : "keyword" }, "MemberName" : { "ignore_above" : 1024, "type" : "keyword" }, "Detail" : { "ignore_above" : 1024, "type" : "keyword" }, "TerminalSessionId" : { "ignore_above" : 1024, "type" : "keyword" }, "MemberSid" : { "ignore_above" : 1024, "type" : "keyword" }, "DriverName" : { "ignore_above" : 1024, "type" : "keyword" }, "DeviceNameLength" : { "ignore_above" : 1024, "type" : "keyword" }, "OldSchemeGuid" : { "ignore_above" : 1024, "type" : "keyword" }, "CreationUtcTime" : { "ignore_above" : 1024, "type" : "keyword" }, "Reason" : { "ignore_above" : 1024, "type" : "keyword" }, "ShutdownReason" : { "ignore_above" : 1024, "type" : "keyword" }, "TargetServerName" : { "ignore_above" : 1024, "type" : "keyword" }, "Number" : { "ignore_above" : 1024, "type" : "keyword" }, "BuildVersion" : { "ignore_above" : 1024, "type" : "keyword" }, "SubjectDomainName" : { "ignore_above" : 1024, "type" : "keyword" }, "MinimumPerformancePercent" : { "ignore_above" : 1024, "type" : "keyword" }, "TargetDomainName" : { "ignore_above" : 1024, "type" : "keyword" }, "LogonId" : { "ignore_above" : 1024, "type" : "keyword" }, "TSId" : { "ignore_above" : 1024, "type" : "keyword" }, "LogonProcessName" : { "ignore_above" : 1024, "type" : "keyword" }, "PrivilegeList" : { "ignore_above" : 1024, "type" : "keyword" }, "param7" : { "ignore_above" : 1024, "type" : "keyword" }, "param8" : { "ignore_above" : 1024, "type" : "keyword" }, "param5" : { "ignore_above" : 1024, "type" : "keyword" }, "param6" : { "ignore_above" : 1024, "type" : "keyword" }, "DriveName" : { "ignore_above" : 1024, "type" : "keyword" }, "NewProcessId" : { "ignore_above" : 1024, "type" : "keyword" }, "LogonType" : { "ignore_above" : 1024, "type" : "keyword" }, "ExtraInfo" : { "ignore_above" : 1024, "type" : "keyword" }, "param3" : { "ignore_above" : 1024, "type" : "keyword" }, "param4" : { "ignore_above" : 1024, "type" : "keyword" }, "param1" : { "ignore_above" : 1024, "type" : "keyword" }, "param2" : { "ignore_above" : 1024, "type" : "keyword" }, "TargetLogonId" : { "ignore_above" : 1024, "type" : "keyword" }, "Workstation" : { "ignore_above" : 1024, "type" : "keyword" }, "SubjectUserName" : { "ignore_above" : 1024, "type" : "keyword" }, "FailureName" : { "ignore_above" : 1024, "type" : "keyword" }, "Signature" : { "ignore_above" : 1024, "type" : "keyword" }, "NewSchemeGuid" : { "ignore_above" : 1024, "type" : "keyword" }, "MinimumThrottlePercent" : { "ignore_above" : 1024, "type" : "keyword" }, "ProcessId" : { "ignore_above" : 1024, "type" : "keyword" }, "EntryCount" : { "ignore_above" : 1024, "type" : "keyword" }, "BitlockerUserInputTime" : { "ignore_above" : 1024, "type" : "keyword" }, "AuthenticationPackageName" : { "ignore_above" : 1024, "type" : "keyword" }, "NominalFrequency" : { "ignore_above" : 1024, "type" : "keyword" } } }, "user_data" : { "type" : "object" }, "opcode" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "type" : "long" }, "record_id" : { "ignore_above" : 1024, "type" : "keyword" }, "event_id" : { "ignore_above" : 1024, "type" : "keyword" }, "task" : { "ignore_above" : 1024, "type" : "keyword" }, "provider_guid" : { "ignore_above" : 1024, "type" : "keyword" }, "activity_id" : { "ignore_above" : 1024, "type" : "keyword" }, "api" : { "ignore_above" : 1024, "type" : "keyword" }, "provider_name" : { "ignore_above" : 1024, "type" : "keyword" }, "user" : { "properties" : { "identifier" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "os" : { "properties" : { "kernel" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "family" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "platform" : { "ignore_above" : 1024, "type" : "keyword" }, "full" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } } } }, "dll" : { "properties" : { "path" : { "ignore_above" : 1024, "type" : "keyword" }, "code_signature" : { "properties" : { "valid" : { "type" : "boolean" }, "trusted" : { "type" : "boolean" }, "subject_name" : { "ignore_above" : 1024, "type" : "keyword" }, "exists" : { "type" : "boolean" }, "status" : { "ignore_above" : 1024, "type" : "keyword" } } }, "pe" : { "properties" : { "file_version" : { "ignore_above" : 1024, "type" : "keyword" }, "product" : { "ignore_above" : 1024, "type" : "keyword" }, "imphash" : { "ignore_above" : 1024, "type" : "keyword" }, "description" : { "ignore_above" : 1024, "type" : "keyword" }, "original_file_name" : { "ignore_above" : 1024, "type" : "keyword" }, "company" : { "ignore_above" : 1024, "type" : "keyword" }, "architecture" : { "ignore_above" : 1024, "type" : "keyword" } } }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "properties" : { "sha1" : { "ignore_above" : 1024, "type" : "keyword" }, "sha256" : { "ignore_above" : 1024, "type" : "keyword" }, "sha512" : { "ignore_above" : 1024, "type" : "keyword" }, "md5" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "message" : { "norms" : false, "type" : "text" }, "url" : { "properties" : { "extension" : { "ignore_above" : 1024, "type" : "keyword" }, "original" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "scheme" : { "ignore_above" : 1024, "type" : "keyword" }, "top_level_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "query" : { "ignore_above" : 1024, "type" : "keyword" }, "path" : { "ignore_above" : 1024, "type" : "keyword" }, "password" : { "ignore_above" : 1024, "type" : "keyword" }, "registered_domain" : { "ignore_above" : 1024, "type" : "keyword" }, "fragment" : { "ignore_above" : 1024, "type" : "keyword" }, "port" : { "type" : "long" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "full" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "username" : { "ignore_above" : 1024, "type" : "keyword" } } }, "@timestamp" : { "type" : "date" }, "pe" : { "properties" : { "file_version" : { "ignore_above" : 1024, "type" : "keyword" }, "product" : { "ignore_above" : 1024, "type" : "keyword" }, "imphash" : { "ignore_above" : 1024, "type" : "keyword" }, "description" : { "ignore_above" : 1024, "type" : "keyword" }, "original_file_name" : { "ignore_above" : 1024, "type" : "keyword" }, "company" : { "ignore_above" : 1024, "type" : "keyword" }, "architecture" : { "ignore_above" : 1024, "type" : "keyword" } } }, "service" : { "properties" : { "node" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword" } } }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "state" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "type" : { "ignore_above" : 1024, "type" : "keyword" }, "ephemeral_id" : { "ignore_above" : 1024, "type" : "keyword" }, "version" : { "ignore_above" : 1024, "type" : "keyword" } } }, "organization" : { "properties" : { "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "tls" : { "properties" : { "cipher" : { "ignore_above" : 1024, "type" : "keyword" }, "established" : { "type" : "boolean" }, "server" : { "properties" : { "not_after" : { "type" : "date" }, "x509" : { "properties" : { "not_after" : { "type" : "date" }, "public_key_exponent" : { "index" : false, "type" : "long" }, "not_before" : { "type" : "date" }, "subject" : { "properties" : { "country" : { "ignore_above" : 1024, "type" : "keyword" }, "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } }, "public_key_curve" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "signature_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_size" : { "type" : "long" }, "serial_number" : { "ignore_above" : 1024, "type" : "keyword" }, "version_number" : { "ignore_above" : 1024, "type" : "keyword" }, "alternative_names" : { "ignore_above" : 1024, "type" : "keyword" }, "issuer" : { "properties" : { "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "country" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "subject" : { "ignore_above" : 1024, "type" : "keyword" }, "not_before" : { "type" : "date" }, "ja3s" : { "ignore_above" : 1024, "type" : "keyword" }, "certificate" : { "ignore_above" : 1024, "type" : "keyword" }, "issuer" : { "ignore_above" : 1024, "type" : "keyword" }, "certificate_chain" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "properties" : { "sha1" : { "ignore_above" : 1024, "type" : "keyword" }, "sha256" : { "ignore_above" : 1024, "type" : "keyword" }, "md5" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "curve" : { "ignore_above" : 1024, "type" : "keyword" }, "next_protocol" : { "ignore_above" : 1024, "type" : "keyword" }, "client" : { "properties" : { "not_after" : { "type" : "date" }, "server_name" : { "ignore_above" : 1024, "type" : "keyword" }, "x509" : { "properties" : { "not_after" : { "type" : "date" }, "public_key_exponent" : { "index" : false, "type" : "long" }, "not_before" : { "type" : "date" }, "subject" : { "properties" : { "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "country" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } }, "public_key_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_curve" : { "ignore_above" : 1024, "type" : "keyword" }, "signature_algorithm" : { "ignore_above" : 1024, "type" : "keyword" }, "serial_number" : { "ignore_above" : 1024, "type" : "keyword" }, "version_number" : { "ignore_above" : 1024, "type" : "keyword" }, "public_key_size" : { "type" : "long" }, "alternative_names" : { "ignore_above" : 1024, "type" : "keyword" }, "issuer" : { "properties" : { "state_or_province" : { "ignore_above" : 1024, "type" : "keyword" }, "country" : { "ignore_above" : 1024, "type" : "keyword" }, "organization" : { "ignore_above" : 1024, "type" : "keyword" }, "distinguished_name" : { "ignore_above" : 1024, "type" : "keyword" }, "locality" : { "ignore_above" : 1024, "type" : "keyword" }, "common_name" : { "ignore_above" : 1024, "type" : "keyword" }, "organizational_unit" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "supported_ciphers" : { "ignore_above" : 1024, "type" : "keyword" }, "not_before" : { "type" : "date" }, "subject" : { "ignore_above" : 1024, "type" : "keyword" }, "certificate" : { "ignore_above" : 1024, "type" : "keyword" }, "ja3" : { "ignore_above" : 1024, "type" : "keyword" }, "issuer" : { "ignore_above" : 1024, "type" : "keyword" }, "certificate_chain" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "properties" : { "sha1" : { "ignore_above" : 1024, "type" : "keyword" }, "sha256" : { "ignore_above" : 1024, "type" : "keyword" }, "md5" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "resumed" : { "type" : "boolean" }, "version" : { "ignore_above" : 1024, "type" : "keyword" }, "version_protocol" : { "ignore_above" : 1024, "type" : "keyword" } } }, "threat" : { "properties" : { "framework" : { "ignore_above" : 1024, "type" : "keyword" }, "technique" : { "properties" : { "reference" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } }, "tactic" : { "properties" : { "reference" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } }, "user" : { "properties" : { "full_name" : { "ignore_above" : 1024, "type" : "keyword", "fields" : { "text" : { "norms" : false, "type" : "text" } } }, "roles" : { "ignore_above" : 1024, "type" : "keyword" }, "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "fields" : { "text" : { "norms" : false, "type" : "text" } }, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" }, "email" : { "ignore_above" : 1024, "type" : "keyword" }, "hash" : { "ignore_above" : 1024, "type" : "keyword" }, "group" : { "properties" : { "domain" : { "ignore_above" : 1024, "type" : "keyword" }, "name" : { "ignore_above" : 1024, "type" : "keyword" }, "id" : { "ignore_above" : 1024, "type" : "keyword" } } } } } } }, "aliases" : { } } } ```

andrewkroh commented 3 years ago

@andrew-goldstein So you do have the index template installed. Note that it applies to indices matching

    "index_patterns" : [
      "winlogbeat-7.10.0-*"
    ],

When data was indexed where did it go? Can you share at the output of these?

GET _cat/aliases/winlogbeat*?v
GET _cat/indices/winlogbeat*?v

My guess is that your data is contained in winlogbeat-7.10.0 which is supposed to be an alias for ILM managed indices like winlogbeat-7.10.0-2020.10.13-000001. This would happen if the beat starts writing under the assumption that winlogbeat-7.10.0 is an alias.

andrew-goldstein commented 3 years ago

`GET _cat/aliases/winlogbeat*?v`

The output from running:

GET _cat/aliases/winlogbeat*?v

is

alias index filter routing.index routing.search is_write_index

`GET _cat/indices/winlogbeat*?v`

The output of:

GET _cat/indices/winlogbeat*?v

is

health status index             uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   winlogbeat-7.10.0 dlc5Ng18Q1i8pViK0BOTpA   1   1       7795            0     13.5mb          6.6mb

andrewkroh commented 3 years ago

@andrew-goldstein Did you try running setup with setup.ilm.overwrite: true? I think that log message from setup was trying to warn you (but wasn't very clear).

Overwriting ILM policy is disabled. Set setup.ilm.overwrite: true for enabling.

The documentation for check_exists also has a similar message:

If you set this option to false, set setup.ilm.overwrite: true so the lifecycle policy can be installed.

If that fixes the setup problem then I'd say the warning message should be stronger and more clear (or become a hard error). And the documentation should be more clear about the impact of the check_exists on the beat setup --index-management command.

andrewkroh commented 3 years ago

I'm thinking that when the beat setup command is executed it should always setup the policy and rollover alias assuming setup.ilm.enabled==true (the default). This would make the docs for the command true (specifically the part about " ilm policy and rollover alias").

 ./filebeat setup -h
This command does initial setup of the environment:

 * Index mapping template in Elasticsearch to ensure fields are mapped.
 * Kibana dashboards (where available).
 * Ingest pipelines (where available).
 * ILM policy (for Elasticsearch 6.5 and newer).

Usage:
  filebeat setup [flags]

Flags:
      --dashboards         Setup dashboards
  -h, --help               help for setup
      --index-management   Setup all components related to Elasticsearch index management, including template, ilm policy and rollover alias
      --modules string     List of enabled modules (comma separated)
      --pipelines          Setup Ingest pipelines

I'm pretty sure setup.ilm.check_exists was created to allow the beat to send events with least privileges. And was intended to affect the normal running state (but not setup).

andrew-goldstein commented 3 years ago

I created another instance in Elastic Cloud, and ran auditbeat setup with

setup.ilm.check_exists: true

which produces the following output on the command line:

➜  auditbeat-7.10.0-darwin-x86_64 sudo ./auditbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

and produces the correct mappings.

Summary

🟢 Running setup using defaults (without specifying a value for the setup.ilm.check_exists setting) results in correct mappings

🟢 Running setup with setup.ilm.check_exists: true results in correct mappings

🔴 Running setup with setup.ilm.check_exists: false results in incorrect mappings

andrewkroh commented 3 years ago

How about setup.ilm.check_exists: false and setup.ilm.overwrite: true like the docs mention?

andrew-goldstein commented 3 years ago

How about setup.ilm.check_exists: false and setup.ilm.overwrite: true like the docs mention?

I created another instance in Elastic Cloud, and ran auditbeat setup with

setup.ilm.check_exists: false
setup.ilm.overwrite: true

which produces the following output on the command line:

➜  auditbeat-7.10.0-darwin-x86_64 sudo ./auditbeat setup
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

and produces incorrect mappings.

Summary

🔴 Running setup with setup.ilm.check_exists: false and setup.ilm.overwrite: true results in incorrect mappings

ManuelFFF commented 3 years ago

Hi all,

I am experiencing a very similar issue with Winlogbeat v7.9.2 and v7.10. I went through the issue described on this thread some time ago, when I saw for the 1st time the PowerShell output mentioning configs setup.ilm.overwrite: true and setup.ilm.check_exists: false. I added both lines to my config and that caused wrong mappings. After I removed the lines and re-created the index, the issue was gone. Since then, I avoid adding any specific ILM setting to the config file, so the default ILM settings take place and take care of everything ILM related. Sadly something is broken now and I am having wrong mappings on new Winlogbeat deployments for v7.9.2 and v7.10. Of course I have removed all indexes, Index Template, Index Pattern, ILM and deployed the beat with fresh download, but the issue persist. After posting for help on ELK discuss, they advised to open a bug report, but I found this one ongoing, which is very close to what I have, so I though in avoid opening duplicates.

Can I have some help?

Thank you in advance

ManuelFFF commented 3 years ago

Any comment on this?

elasticmachine commented 6 months ago

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

elastic / beats