search

deviantony / docker-elk

The Elastic stack (ELK) powered by Docker and Compose.
MIT License
17.34k stars 6.8k forks source link

Kibana fails to authenticate with security_exception when the trial license expires #895

Closed grizzlycode closed 1 year ago

grizzlycode commented 1 year ago

Problem description

The trial license has expired, so I shutdown the cluster and changed the license to "basic" in elasticsearch config, however this has broken a few components of the cluster since I used the TLS version.

Specifically Kibana and Logstash are throwing errors (see logs). Elasticsearch log states, "All data operations (read and write) continue to work." even though the license is expired. But, of course its blocking paid features which is fine.

It looks like I need to make config changes to restore access. I'm thinking remove TLS since its a paid feature. If that's the case what components/configs do I need to modify to remove it and get access to my data?

My main goal is to at least get Kibana functional again to see my data. But if I can restore the cluster to HTTP instead to make it a fully functional http cluster with "basic" license that would be great as well.

I really hope elastic makes TLS the default and allows basic license to use this in the future...

Extra information

Curl results for server health: 

{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "current license is non-compliant for [security]",
        "license.expired.feature" : "security"
      }
    ],
    "type" : "security_exception",
    "reason" : "current license is non-compliant for [security]",
    "license.expired.feature" : "security"
  },
  "status" : 403
}
* Connection #0 to host localhost left intact

Stack configuration

The only change I made was to elasticsearch.yml. I changed from "trial" to "basic" and restarted the cluster.

Docker setup

$ docker version

Client: Docker Engine - Community
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:35:18 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.5
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.6
  Git commit:       a61e2b4
  Built:            Fri Jul 21 20:35:18 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker-compose version

Docker Compose version v2.20.2

Container logs

$ docker-compose logs

KIBANA:

[2023-08-07T17:39:08.343+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. security_exception
    Root causes:
        security_exception: unable to authenticate user [kibana_system] for REST request [/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip]
[2023-08-07T17:39:13.849+00:00][INFO ][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
[2023-08-07T17:39:13.849+00:00][INFO ][savedobjects-service] Starting saved objects migrations
[2023-08-07T17:39:14.277+00:00][INFO ][savedobjects-service] [.kibana_analytics] INIT -> WAIT_FOR_YELLOW_SOURCE. took: 222ms.
[2023-08-07T17:39:14.307+00:00][INFO ][savedobjects-service] [.kibana_alerting_cases] INIT -> WAIT_FOR_YELLOW_SOURCE. took: 222ms.
[2023-08-07T17:39:14.322+00:00][ERROR][savedobjects-service] [.kibana_analytics] Action failed with 'security_exception

[2023-08-07T19:00:53.841+00:00][ERROR][savedobjects-service] [.kibana] Action failed with 'security_exception
    Root causes:
        security_exception: current license is non-compliant for [security]'. Retrying attempt 10 in 64 seconds.
[2023-08-07T19:00:53.841+00:00][INFO ][savedobjects-service] [.kibana] WAIT_FOR_YELLOW_SOURCE -> WAIT_FOR_YELLOW_SOURCE. took: 64006ms.
[2023-08-07T19:00:53.846+00:00][ERROR][savedobjects-service] [.kibana_ingest] Action failed with 'security_exception

LOGSTASH:

[2023-08-07T17:39:00,782][INFO ][logstash.outputs.elasticsearch][main] Failed to perform request {:message=>"Connect to elasticsearch:9200 [elasticsearch/172.18.0.3] failed: Connection refused", :exception=>Manticore::SocketException, :cause=>#<Java::OrgApacheHttpConn::HttpHostConnectException: Connect to elasticsearch:9200 [elasticsearch/172.18.0.3] failed: Connection refused>}
[2023-08-07T17:39:00,784][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://logstash_internal:xxxxxx@elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://elasticsearch:9200/][Manticore::SocketException] Connect to elasticsearch:9200 [elasticsearch/172.18.0.3] failed: Connection refused"}
[2023-08-07T17:39:05,793][INFO ][logstash.outputs.elasticsearch][main] Failed to perform request {:message=>"Connect to elasticsearch:9200 [elasticsearch/172.18.0.3] failed: Connection refused", :exception=>Manticore::SocketException, :cause=>#<Java::OrgApacheHttpConn::HttpHostConnectException: Connect to elasticsearch:9200 [elasticsearch/172.18.0.3] failed: Connection refused>}
[2023-08-07T17:39:05,795][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://logstash_internal:xxxxxx@elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://elasticsearch:9200/][Manticore::SocketException] Connect to elasticsearch:9200 [elasticsearch/172.18.0.3] failed: Connection refused"}
[2023-08-07T17:39:12,512][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https://logstash_internal:xxxxxx@elasticsearch:9200/"}
[2023-08-07T17:39:12,607][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (8.8.2) {:es_version=>8}
[2023-08-07T17:39:12,613][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2023-08-07T17:39:13,059][WARN ][logstash.outputs.elasticsearch][main] Elasticsearch license is not active, please check Elasticsearch’s licensing information {:url=>"https://logstash_internal:xxxxxx@elasticsearch:9200/", :license=>{"status"=>"expired", "uid"=>"298da51e-59ba-4c61-9333-11c7dd11798c", "type"=>"trial", "issue_date"=>"2023-07-05T17:31:14.438Z", "issue_date_in_millis"=>1688578274438, "expiry_date"=>"2023-08-04T17:31:14.438Z", "expiry_date_in_millis"=>1691170274438, "max_nodes"=>1000, "max_resource_units"=>nil, "issued_to"=>"docker-cluster", "issuer"=>"elasticsearch", "start_date_in_millis"=>-1}}

Edit: Added additoinal Kibana log showing "security_exception" error.

antoineco commented 1 year ago

I doubt enabling security or TLS is the issue here. The page https://www.elastic.co/subscriptions lists both "Encrypted communications" and "File and native authentication" as free and open-basic features. Only the more "enterprisey" authentication features, such as Active Directory, Single Sign-On, etc. require a premium license.

The logs of your Elasticsearch instance should contain a message pointing at the root cause, but you didn't share them so I can't tell what's going on.

grizzlycode commented 1 year ago

I didn't see anything that specifically points to the root cause in the Elasticsearch logs, but here they are below. It repeats the licensing issue throughout the log.

Elasticsearch log:

Created elasticsearch keystore in /usr/share/elasticsearch/config/elasticsearch.keystore
{"@timestamp":"2023-08-07T17:54:13.964Z", "log.level": "INFO", "message":"version[8.8.2], pid[174], build[docker/98e1271edf932a480e4262a471281f1ee295ce6b/2023-06-26T05:16:16.196344851Z], OS[Linux/5.15.0-78-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/20.0.1/20.0.1+9-29]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:14.003Z", "log.level": "INFO", "message":"JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:14.004Z", "log.level": "INFO", "message":"JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -Des.cgroups.hierarchy.override=/, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-13956753009332259900, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms512m, -Xmx512m, -XX:MaxDirectMemorySize=268435456, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=docker, --module-path=/usr/share/elasticsearch/lib, --add-modules=jdk.net, --add-modules=org.elasticsearch.preallocate, -Djdk.module.main=org.elasticsearch.server]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:21.984Z", "log.level": "WARN", "message":"SLF4J: No SLF4J providers were found.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:21.988Z", "log.level": "WARN", "message":"SLF4J: Defaulting to no-operation (NOP) logger implementation", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:21.988Z", "log.level": "WARN", "message":"SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:21.998Z", "log.level": "WARN", "message":"SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:22.010Z", "log.level": "WARN", "message":"SLF4J: Ignoring binding found at [jar:file:///usr/share/elasticsearch/modules/repository-azure/log4j-slf4j-impl-2.19.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:22.010Z", "log.level": "WARN", "message":"SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.842Z", "log.level": "INFO", "message":"loaded module [repository-url]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.843Z", "log.level": "INFO", "message":"loaded module [x-pack-core]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.843Z", "log.level": "INFO", "message":"loaded module [x-pack-redact]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.843Z", "log.level": "INFO", "message":"loaded module [dlm]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.843Z", "log.level": "INFO", "message":"loaded module [ingest-user-agent]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.843Z", "log.level": "INFO", "message":"loaded module [x-pack-async-search]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.844Z", "log.level": "INFO", "message":"loaded module [x-pack-monitoring]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.844Z", "log.level": "INFO", "message":"loaded module [repository-s3]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.844Z", "log.level": "INFO", "message":"loaded module [x-pack-analytics]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.844Z", "log.level": "INFO", "message":"loaded module [x-pack-ent-search]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.844Z", "log.level": "INFO", "message":"loaded module [x-pack-autoscaling]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.844Z", "log.level": "INFO", "message":"loaded module [lang-painless]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.845Z", "log.level": "INFO", "message":"loaded module [x-pack-ml]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.845Z", "log.level": "INFO", "message":"loaded module [legacy-geo]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.845Z", "log.level": "INFO", "message":"loaded module [lang-mustache]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.845Z", "log.level": "INFO", "message":"loaded module [x-pack-ql]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.845Z", "log.level": "INFO", "message":"loaded module [rank-rrf]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.845Z", "log.level": "INFO", "message":"loaded module [analysis-common]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.869Z", "log.level": "INFO", "message":"loaded module [transport-netty4]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.870Z", "log.level": "INFO", "message":"loaded module [aggregations]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.870Z", "log.level": "INFO", "message":"loaded module [ingest-common]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.870Z", "log.level": "INFO", "message":"loaded module [frozen-indices]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.870Z", "log.level": "INFO", "message":"loaded module [x-pack-identity-provider]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.871Z", "log.level": "INFO", "message":"loaded module [x-pack-shutdown]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.871Z", "log.level": "INFO", "message":"loaded module [x-pack-text-structure]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.871Z", "log.level": "INFO", "message":"loaded module [snapshot-repo-test-kit]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.871Z", "log.level": "INFO", "message":"loaded module [ml-package-loader]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.871Z", "log.level": "INFO", "message":"loaded module [kibana]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.871Z", "log.level": "INFO", "message":"loaded module [constant-keyword]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.872Z", "log.level": "INFO", "message":"loaded module [x-pack-logstash]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.872Z", "log.level": "INFO", "message":"loaded module [x-pack-ccr]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.872Z", "log.level": "INFO", "message":"loaded module [x-pack-graph]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.872Z", "log.level": "INFO", "message":"loaded module [parent-join]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.872Z", "log.level": "INFO", "message":"loaded module [x-pack-enrich]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.872Z", "log.level": "INFO", "message":"loaded module [repositories-metering-api]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.872Z", "log.level": "INFO", "message":"loaded module [transform]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.873Z", "log.level": "INFO", "message":"loaded module [repository-azure]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.873Z", "log.level": "INFO", "message":"loaded module [repository-gcs]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.873Z", "log.level": "INFO", "message":"loaded module [spatial]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.873Z", "log.level": "INFO", "message":"loaded module [mapper-version]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.873Z", "log.level": "INFO", "message":"loaded module [mapper-extras]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.873Z", "log.level": "INFO", "message":"loaded module [apm]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.873Z", "log.level": "INFO", "message":"loaded module [x-pack-aggregate-metric]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.874Z", "log.level": "INFO", "message":"loaded module [x-pack-rollup]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.874Z", "log.level": "INFO", "message":"loaded module [percolator]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.874Z", "log.level": "INFO", "message":"loaded module [x-pack-stack]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.874Z", "log.level": "INFO", "message":"loaded module [data-streams]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.874Z", "log.level": "INFO", "message":"loaded module [reindex]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.874Z", "log.level": "INFO", "message":"loaded module [rank-eval]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.874Z", "log.level": "INFO", "message":"loaded module [x-pack-security]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.875Z", "log.level": "INFO", "message":"loaded module [blob-cache]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.890Z", "log.level": "INFO", "message":"loaded module [searchable-snapshots]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.890Z", "log.level": "INFO", "message":"loaded module [snapshot-based-recoveries]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.890Z", "log.level": "INFO", "message":"loaded module [x-pack-watcher]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.890Z", "log.level": "INFO", "message":"loaded module [old-lucene-versions]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.891Z", "log.level": "INFO", "message":"loaded module [x-pack-ilm]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.891Z", "log.level": "INFO", "message":"loaded module [x-pack-voting-only-node]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.891Z", "log.level": "INFO", "message":"loaded module [x-pack-deprecation]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.891Z", "log.level": "INFO", "message":"loaded module [x-pack-fleet]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.891Z", "log.level": "INFO", "message":"loaded module [x-pack-profiling]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.891Z", "log.level": "INFO", "message":"loaded module [ingest-geoip]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.892Z", "log.level": "INFO", "message":"loaded module [x-pack-write-load-forecaster]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.892Z", "log.level": "INFO", "message":"loaded module [search-business-rules]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.892Z", "log.level": "INFO", "message":"loaded module [ingest-attachment]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.892Z", "log.level": "INFO", "message":"loaded module [wildcard]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.892Z", "log.level": "INFO", "message":"loaded module [x-pack-sql]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.892Z", "log.level": "INFO", "message":"loaded module [unsigned-long]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.893Z", "log.level": "INFO", "message":"loaded module [x-pack-async]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.893Z", "log.level": "INFO", "message":"loaded module [runtime-fields-common]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.893Z", "log.level": "INFO", "message":"loaded module [vector-tile]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.893Z", "log.level": "INFO", "message":"loaded module [lang-expression]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:25.893Z", "log.level": "INFO", "message":"loaded module [x-pack-eql]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.plugins.PluginsService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:35.894Z", "log.level": "WARN", "message":"Aug 07, 2023 5:54:35 PM org.apache.lucene.store.MemorySegmentIndexInputProvider <init>", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:35.918Z", "log.level": "INFO", "message":"using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/ubuntu--vg-ubuntu--lv)]], net usable_space [13.6gb], net total_space [47.9gb], types [ext4]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.env.NodeEnvironment","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:35.921Z", "log.level": "INFO", "message":"heap size [512mb], compressed ordinary object pointers [true]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.env.NodeEnvironment","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:36.232Z", "log.level": "INFO", "message":"node name [elasticsearch], node ID [bBfsNJ7HRvWzDriRKUv7UQ], cluster name [docker-cluster], roles [transform, data_content, data_warm, master, remote_cluster_client, data, data_cold, ingest, data_frozen, ml, data_hot]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:44.548Z", "log.level": "INFO", "message":"[controller/199] [Main.cc@123] controller (64 bit): Version 8.8.2 (Build 12b72d8b77ab68) Copyright (c) 2023 Elasticsearch BV", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"ml-cpp-log-tail-thread","log.logger":"org.elasticsearch.xpack.ml.process.logging.CppLogMessageHandler","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:45.063Z", "log.level": "INFO", "message":"Security is enabled", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.xpack.security.Security","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:46.608Z", "log.level": "INFO", "message":"parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.xpack.security.authz.store.FileRolesStore","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:48.659Z", "log.level": "INFO", "message":"Profiling is enabled", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.xpack.profiler.ProfilingPlugin","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.168Z", "log.level": "WARN", "message":"SLF4J: No SLF4J providers were found.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.169Z", "log.level": "WARN", "message":"SLF4J: Defaulting to no-operation (NOP) logger implementation", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.169Z", "log.level": "WARN", "message":"SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.171Z", "log.level": "WARN", "message":"SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.172Z", "log.level": "WARN", "message":"SLF4J: Ignoring binding found at [jar:file:///usr/share/elasticsearch/modules/x-pack-security/log4j-slf4j-impl-2.19.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.173Z", "log.level": "WARN", "message":"SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"stderr","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.219Z", "log.level": "INFO", "message":"creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=4mb, heap_size=512mb}]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.transport.netty4.NettyAllocator","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.254Z", "log.level": "INFO", "message":"using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.indices.recovery.RecoverySettings","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:50.321Z", "log.level": "INFO", "message":"using discovery type [multi-node] and seed hosts providers [settings]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.discovery.DiscoveryModule","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:53.650Z", "log.level": "INFO", "message":"initialized", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:53.662Z", "log.level": "INFO", "message":"starting ...", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:53.711Z", "log.level": "INFO", "message":"persistent cache index loaded", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:53.712Z", "log.level": "INFO", "message":"deprecation component started", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.xpack.deprecation.logging.DeprecationIndexingComponent","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:53.856Z", "log.level": "INFO", "message":"publish_address {172.22.0.2:9300}, bound_addresses {0.0.0.0:9300}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.transport.TransportService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:56.110Z", "log.level": "INFO", "message":"bound or publishing to a non-loopback address, enforcing bootstrap checks", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.BootstrapChecks","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:56.229Z", "log.level": "WARN", "message":"this node is locked into cluster UUID [uwiDV2ptSKueoEkV9_QPew] but [cluster.initial_master_nodes] is set to [elasticsearch, es02, es03]; remove this setting to avoid possible data loss caused by subsequent cluster bootstrap attempts; for further information see https://www.elastic.co/guide/en/elasticsearch/reference/8.8/important-settings.html#initial_master_nodes", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.cluster.coordination.ClusterBootstrapService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:54:59.273Z", "log.level": "INFO", "message":"master node changed {previous [], current [{es02}{XYJs5HCFReyQKPFfArOqOw}{_npfGvHgSiGJtKOmzLDyUg}{es02}{172.22.0.4}{172.22.0.4:9300}{cdfhilmrstw}{8.8.2}]}, added {{es02}{XYJs5HCFReyQKPFfArOqOw}{_npfGvHgSiGJtKOmzLDyUg}{es02}{172.22.0.4}{172.22.0.4:9300}{cdfhilmrstw}{8.8.2}, {es03}{VL9n926cRDeYOVZ_edUcEg}{RHg5NmK7QD6FhBoM1xhnQA}{es03}{172.22.0.3}{172.22.0.3:9300}{cdfhilmrstw}{8.8.2}}, term: 23, version: 1110, reason: ApplyCommitRequest{term=23, version=1110, sourceNode={es02}{XYJs5HCFReyQKPFfArOqOw}{_npfGvHgSiGJtKOmzLDyUg}{es02}{172.22.0.4}{172.22.0.4:9300}{cdfhilmrstw}{8.8.2}{xpack.installed=true, ml.machine_memory=16766164992, ml.allocated_processors=4, ml.max_jvm_size=536870912, ml.allocated_processors_double=4.0}}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.service.ClusterApplierService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.642Z", "log.level": "INFO", "message":"refresh keys", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.xpack.security.authc.TokenService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.905Z", "log.level": "INFO", "message":"refreshed keys", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.xpack.security.authc.TokenService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.961Z", "log.level": "INFO", "message":"license [298da51e-59ba-4c61-9333-11c7dd11798c] mode [trial] - valid", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.license.ClusterStateLicenseService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.969Z", "log.level": "WARN", "message":"LICENSE [EXPIRED] ON [FRIDAY, AUGUST 04, 2023].\n# IF YOU HAVE A NEW LICENSE, PLEASE UPDATE IT. OTHERWISE, PLEASE REACH OUT TO\n# YOUR SUPPORT CONTACT.\n# \n# COMMERCIAL PLUGINS OPERATING WITH REDUCED FUNCTIONALITY\n# - security\n#  - Cluster health, cluster stats and indices stats operations are blocked\n#  - All data operations (read and write) continue to work\n# - watcher\n#  - PUT / GET watch APIs are disabled, DELETE watch API continues to work\n#  - Watches execute and write to the history\n#  - The actions of the watches don't execute\n# - monitoring\n#  - The agent will stop collecting cluster and indices metrics\n# - graph\n#  - Graph explore APIs are disabled\n# - ml\n#  - Machine learning APIs are disabled\n# - logstash\n#  - Logstash will continue to poll centrally-managed pipelines\n# - beats\n#  - Beats will continue to poll centrally-managed configuration\n# - deprecation\n#  - Deprecation APIs are disabled\n# - upgrade\n#  - Upgrade API is disabled\n# - sql\n#  - SQL support is disabled\n# - enterprise_search\n#  - Search Applications and behavioral analytics will be disabled\n# - rollup\n#  - Creating and Starting rollup jobs will no longer be allowed.\n#  - Stopping/Deleting existing jobs, RollupCaps API and RollupSearch continue to function.\n# - transform\n#  - Creating, starting, updating transforms will no longer be allowed.\n#  - Stopping/Deleting existing transforms continue to function.\n# - analytics\n#  - Aggregations provided by Analytics plugin are no longer usable.\n# - ccr\n#  - Creating new follower indices will be blocked\n#  - Configuring auto-follow patterns will be blocked\n#  - Auto-follow patterns will no longer discover new leader indices\n#  - The CCR monitoring endpoint will be blocked\n#  - Existing follower indices will continue to replicate data\n# - redact_processor\n#  - Executing a redact processor in an ingest pipeline will fail.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][trigger_engine_scheduler][T#1]","log.logger":"org.elasticsearch.license.ClusterStateLicenseService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.971Z", "log.level": "INFO", "message":"license mode is [trial], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.xpack.security.authc.Realms","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.972Z", "log.level": "WARN", "message":"license [298da51e-59ba-4c61-9333-11c7dd11798c] - expired", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.license.ClusterStateLicenseService","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.980Z", "log.level": "INFO", "message":"publish_address {172.22.0.2:9200}, bound_addresses {0.0.0.0:9200}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.http.AbstractHttpServerTransport","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:00.982Z", "log.level": "INFO", "message":"started {elasticsearch}{bBfsNJ7HRvWzDriRKUv7UQ}{upW1eXTuTxCQZLq-N_4GDA}{elasticsearch}{172.22.0.2}{172.22.0.2:9300}{cdfhilmrstw}{8.8.2}{xpack.installed=true, ml.machine_memory=16766164992, ml.allocated_processors=4, ml.max_jvm_size=536870912, ml.allocated_processors_double=4.0}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.node.Node","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:03.892Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-Country.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][generic][T#3]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:04.740Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-ASN.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][generic][T#4]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:05.646Z", "log.level": "INFO", "message":"Node [{elasticsearch}{bBfsNJ7HRvWzDriRKUv7UQ}] is selected as the current health node.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][management][T#1]","log.logger":"org.elasticsearch.health.node.selection.HealthNodeTaskExecutor","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:06.619Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#2]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:06.812Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#2]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:07.016Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#1]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:07.195Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#3]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:07.416Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:07.464Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:08.564Z", "log.level": "INFO", "message":"successfully loaded geoip database file [GeoLite2-City.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][generic][T#2]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:08.637Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:08.825Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:09.022Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:09.372Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:09.427Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:09.488Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:12.659Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:12.833Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:13.028Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:13.391Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:13.433Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:13.494Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:20.665Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:20.846Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:21.034Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:21.397Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:21.444Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:21.500Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:36.674Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:36.852Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:37.040Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:37.403Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:37.449Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:55:37.506Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:56:08.681Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:56:08.859Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:56:09.045Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:56:09.409Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:56:09.455Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:56:09.513Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:57:12.687Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:57:12.864Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:57:13.052Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:57:13.416Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:57:13.460Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:57:13.519Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:58:16.693Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:58:16.869Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:58:17.057Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:58:17.421Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:58:17.465Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:58:17.525Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:59:20.699Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:59:20.875Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:59:21.063Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:59:21.430Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:59:21.473Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T17:59:21.531Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T18:00:24.706Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T18:00:24.880Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T18:00:25.069Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T18:00:25.436Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T18:00:25.478Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch][transport_worker][T#4]","log.logger":"org.elasticsearch.xpack.security.action.filter.SecurityActionFilter","trace.id":"4aa38c71fc08936da1b8f675ec1b9154","elasticsearch.cluster.uuid":"uwiDV2ptSKueoEkV9_QPew","elasticsearch.node.id":"bBfsNJ7HRvWzDriRKUv7UQ","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2023-08-07T18:00:25.536Z", "log.level":"ERROR", "message":"blocking [cluster:monitor/health] operation due to expired license. Cluster health, cluster stats and indices stats \noperations are blocked on license expiration. All data operations (read and write) continue to work. \nIf you have a new license, please update it. Otherwise, please reach out to your support contact.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch
antoineco commented 1 year ago

Unwrapping the license warning for readability:

LICENSE [EXPIRED] ON [FRIDAY, AUGUST 04, 2023]. # IF YOU HAVE A NEW LICENSE, PLEASE UPDATE IT. OTHERWISE, PLEASE REACH OUT TO # YOUR SUPPORT CONTACT. # # COMMERCIAL PLUGINS OPERATING WITH REDUCED FUNCTIONALITY # - security # - Cluster health, cluster stats and indices stats operations are blocked # - All data operations (read and write) continue to work # - watcher # - PUT / GET watch APIs are disabled, DELETE watch API continues to work # - Watches execute and write to the history # - The actions of the watches don't execute # - monitoring # - The agent will stop collecting cluster and indices metrics # - graph # - Graph explore APIs are disabled # - ml # - Machine learning APIs are disabled # - logstash # - Logstash will continue to poll centrally-managed pipelines # - beats # - Beats will continue to poll centrally-managed configuration # - deprecation # - Deprecation APIs are disabled # - upgrade # - Upgrade API is disabled # - sql # - SQL support is disabled # - enterprise_search # - Search Applications and behavioral analytics will be disabled # - rollup # - Creating and Starting rollup jobs will no longer be allowed. # - Stopping/Deleting existing jobs, RollupCaps API and RollupSearch continue to function. # - transform # - Creating, starting, updating transforms will no longer be allowed. # - Stopping/Deleting existing transforms continue to function. # - analytics # - Aggregations provided by Analytics plugin are no longer usable. # - ccr # - Creating new follower indices will be blocked # - Configuring auto-follow patterns will be blocked # - Auto-follow patterns will no longer discover new leader indices # - The CCR monitoring endpoint will be blocked # - Existing follower indices will continue to replicate data # - redact_processor # - Executing a redact processor in an ingest pipeline will fail.

The two issues that seem to affect your installation are:

Everything else seems to indicate that your cluster is actually healthy.

Do you maybe have Metricbeat, or some Elastic Agent, generating the requests which are shown as blocked in the logs?

grizzlycode commented 1 year ago

Yes, I did have Metricbeat enabled to test it. During troubleshooting I have metricbeat down.

I wonder why I'm getting this then when I execute curl for server health? 

"reason" : "current license is non-compliant for [security]",
        "license.expired.feature" : "security"
"type" : "security_exception"

This security exception is also referenced in Kibana logs: 

Root causes:
        security_exception: current license is non-compliant for [security]'. Retrying attempt 10 in 64 seconds.
[2023-08-07T19:00:53.841+00:00][INFO ][savedobjects-service] [.kibana] WAIT_FOR_YELLOW_SOURCE -> WAIT_FOR_YELLOW_SOURCE. took: 64006ms.
[2023-08-07T19:00:53.846+00:00][ERROR][savedobjects-service] [.kibana_ingest] Action failed with 'security_exception
antoineco commented 1 year ago

You mentioned that you tried to change the license type in the Elasticsearch config. However, the logs still show trial instead of basic. I wonder whether the issue could be related to that change of license.

This issue report look very similar to something similar reported almost 2 years ago, which I wasn't able to reproduce:

I would try two things if I were you:

  1. Revert the license setting to trial inside your Elasticsearch config (in case you really switched it to basic). If that works and you can now log into Kibana, try to change the license via Kibana.
  2. If you still cannot log into Kibana, execute curl -D- -X POST 'https://localhost:9200/_license/start_basic?acknowledge=true' -u elastic:changeme while Elasticsearch is running (notice the acknowledgment parameter).

Ref. https://www.elastic.co/guide/en/elasticsearch/reference/8.9/start-basic.html

grizzlycode commented 1 year ago

I did notice it still said, "trial" even though I changed it to "basic" not sure why that was. Anyways I changed it back to "trial" and restarted the stack. I'm still unable to get to Kibana interface.

I tried the command you gave me. Here is the output. I can now get into Kibana! 

HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 46

{"acknowledged":true,"basic_was_started":true}

I test if the elasticsearch.yml "trial" config was going to be an impact if I restarted service, I restarted and I was still able to access the interface and it still says "basic" license. So I think were good now.

Maybe there is an issue with changing the license type in the elasticsearch.yml config that breaks it? Anyways I'm able to access Kibana and my data now.

So that resloves the Kibana issue! But, I'm still having erros on Logstash. Its getting "refused" for some reason. 

[2023-08-07T19:43:32,325][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://logstash_internal:xxxxxx@elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://elasticsearch:9200/][Manticore::SocketException] Connect to elasticsearch:9200 [elasticsearch/172.26.0.2] failed: Connection refused"}
[2023-08-07T19:43:38,637][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https://logstash_internal:xxxxxx@elasticsearch:9200/"}
[2023-08-07T19:43:38,748][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (8.8.2) {:es_version=>8}
[2023-08-07T19:43:38,749][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
antoineco commented 1 year ago

Great! It means that the license should not be changed via the Elasticsearch config but rather through its API to be able to provide the acknowledgment. I'm making a mental note about this issue and will rephrase the ambiguous first paragraph at https://github.com/deviantony/docker-elk/tree/8.2305.1#how-to-disable-paid-features.

(Regarding the other "issue" with Logstash, I see nothing abnormal. Logstash throws "connection refused" while Elasticsearch is still starting, then "Restored connection to ES instance" when it finally manages to connect. It's being resilient and is working as designed.)

grizzlycode commented 1 year ago

Thanks for the quick turn and finding a working soution! Its one of the reasons I stick with this repo!