search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.28k stars 24.86k forks source link

Failure accessing class MacOsPreallocator #95125

Closed davidkyle closed 1 year ago

davidkyle commented 1 year ago

Elasticsearch Version

8.8.0-SNAPSHOT

Installed Plugins

No response

Java Version

bundled

OS Version

macOS 13.3

Problem Description

When running the Yaml rest tests on macOS the Elasticsearch node fails to start.

Caused by: java.lang.IllegalAccessException: class com.sun.jna.Structure (in module com.sun.jna) cannot access class org.elasticsearch.preallocate.MacOsPreallocator$Natives$Fcntl$FStore (in module org.elasticsearch.preallocate) because module org.elasticsearch.preallocate does not export org.elasticsearch.preallocate to module com.sun.jna

And the last 2 log messages.

» [2023-04-11T09:40:55,474][INFO ][o.e.p.Preallocate        ] [yamlRestTest-0] pre-allocating cache file [/Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/data/shared_snapshot_cache] (16777216 bytes) using native methods
» [2023-04-11T08:40:56.184025Z] [BUILD] Stopping node

If Elasticsearch is started with ./gradlew run then the error does not occur - because the pre-allocating cache file event is not triggered

Possibly related to #94884

Steps to Reproduce

Run a test:

./gradlew ':x-pack:plugin:yamlRestTest' --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=ml/3rd_party_deployment/*}"

Logs (if relevant)

When running the Yaml rest test

[2023-04-11T09:17:45.069118Z] [BUILD] Setting up 3 additional config files
[2023-04-11T09:17:45.070657Z] [BUILD] Copying additional config files from distro [/Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/jvm.options.d, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/users_roles, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/role_mapping.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/users, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/elasticsearch.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/roles.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/log4j2.properties, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/elasticsearch-plugins.example.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/jvm.options]
[2023-04-11T09:17:45.072416Z] [BUILD] Creating elasticsearch keystore with password set to []
[2023-04-11T09:17:45.714354Z] [BUILD] Adding 2 keystore settings and 0 keystore files
[2023-04-11T09:17:47.119935Z] [BUILD] Installing 0 modules
[2023-04-11T09:17:47.120161Z] [BUILD] Setting up 1 users
[2023-04-11T09:17:47.903670Z] [BUILD] Setting up roles.yml
[2023-04-11T09:17:47.903975Z] [BUILD] Starting Elasticsearch process
[2023-04-11T10:17:49,686][INFO ][o.e.c.u.FeatureFlag      ] [yamlRestTest-0] The current build is a snapshot, feature flag [untrusted_remote_cluster] is enabled
[2023-04-11T10:17:50,081][INFO ][o.e.n.Node               ] [yamlRestTest-0] version[8.8.0-SNAPSHOT], pid[63135], build[tar/67ccbd8aabd0047327322920168dde2106b1f37e/2023-04-11T08:32:27.730052Z], OS[Mac OS X/13.3/aarch64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/20/20+36-2344]
[2023-04-11T10:17:50,081][INFO ][o.e.n.Node               ] [yamlRestTest-0] JVM home [/Users/davidkyle/.gradle/caches/transforms-3/9724627d5cf99175aa30a9bab0366ce2/transformed/macos-20-aarch64.tar.gz/Contents/Home], using bundled JDK [false]
[2023-04-11T10:17:50,082][INFO ][o.e.n.Node               ] [yamlRestTest-0] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=/Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/tmp, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=/Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/logs, -XX:ErrorFile=/Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms512m, -Xmx512m, -ea, -esa, -Dingest.geoip.downloader.enabled.default=false, -Dio.netty.leakDetection.level=paranoid, -XX:MaxDirectMemorySize=268435456, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/lib, --add-modules=jdk.net, --add-modules=org.elasticsearch.preallocate, -Djdk.module.main=org.elasticsearch.server]
[2023-04-11T10:17:50,082][WARN ][o.e.n.Node               ] [yamlRestTest-0] version [8.8.0-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production
[2023-04-11T10:17:50,995][WARN ][stderr                   ] [yamlRestTest-0] SLF4J: No SLF4J providers were found.
[2023-04-11T10:17:50,995][WARN ][stderr                   ] [yamlRestTest-0] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2023-04-11T10:17:50,995][WARN ][stderr                   ] [yamlRestTest-0] SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
[2023-04-11T10:17:50,996][WARN ][stderr                   ] [yamlRestTest-0] SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
[2023-04-11T10:17:50,996][WARN ][stderr                   ] [yamlRestTest-0] SLF4J: Ignoring binding found at [jar:file:///Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/modules/repository-azure/log4j-slf4j-impl-2.19.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[2023-04-11T10:17:50,996][WARN ][stderr                   ] [yamlRestTest-0] SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
[2023-04-11T10:17:51,639][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [repository-url]
[2023-04-11T10:17:51,639][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [dlm]
[2023-04-11T10:17:51,639][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [ingest-user-agent]
[2023-04-11T10:17:51,639][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-core]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-async-search]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [test-error-query]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-monitoring]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [repository-s3]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-analytics]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-ent-search]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-autoscaling]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [lang-painless]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-ml]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [test-die-with-dignity]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [legacy-geo]
[2023-04-11T10:17:51,640][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [lang-mustache]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-ql]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [analysis-common]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [test-seek-tracking-directory]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [transport-netty4]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [aggregations]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [ingest-common]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-identity-provider]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [frozen-indices]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-text-structure]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-shutdown]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [snapshot-repo-test-kit]
[2023-04-11T10:17:51,641][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [test-delayed-aggs]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [kibana]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [constant-keyword]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-logstash]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-graph]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-ccr]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [parent-join]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-enrich]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [repositories-metering-api]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [transform]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [repository-azure]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [repository-gcs]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [spatial]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [apm]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [mapper-extras]
[2023-04-11T10:17:51,642][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [mapper-version]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-aggregate-metric]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-rollup]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [percolator]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [data-streams]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-stack]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [rank-eval]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [reindex]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-security]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [blob-cache]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [searchable-snapshots]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [snapshot-based-recoveries]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-watcher]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [old-lucene-versions]
[2023-04-11T10:17:51,643][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-ilm]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-voting-only-node]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-deprecation]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-fleet]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-profiling]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [ingest-geoip]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-write-load-forecaster]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [search-business-rules]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [wildcard]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [ingest-attachment]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [unsigned-long]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-sql]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [runtime-fields-common]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-async]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [vector-tile]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [lang-expression]
[2023-04-11T10:17:51,644][INFO ][o.e.p.PluginsService     ] [yamlRestTest-0] loaded module [x-pack-eql]
[2023-04-11T10:17:51,720][INFO ][o.e.c.u.FeatureFlag      ] [yamlRestTest-0] The current build is a snapshot, feature flag [dlm] is enabled
[2023-04-11T10:17:52,766][INFO ][o.e.c.u.FeatureFlag      ] [yamlRestTest-0] The current build is a snapshot, feature flag [index_mode] is enabled
[2023-04-11T10:17:53,011][WARN ][stderr                   ] [yamlRestTest-0] Apr 11, 2023 10:17:53 AM org.apache.lucene.store.MemorySegmentIndexInputProvider <init>
[2023-04-11T10:17:53,017][INFO ][o.e.e.NodeEnvironment    ] [yamlRestTest-0] using [1] data paths, mounts [[/System/Volumes/Data (/dev/disk3s5)]], net usable_space [676.5gb], net total_space [926.3gb], types [apfs]
[2023-04-11T10:17:53,017][INFO ][o.e.e.NodeEnvironment    ] [yamlRestTest-0] heap size [512mb], compressed ordinary object pointers [true]
[2023-04-11T10:17:53,030][INFO ][o.e.n.Node               ] [yamlRestTest-0] node name [yamlRestTest-0], node ID [mYWbzdyvTlOgUXGuEC0wnA], cluster name [yamlRestTest], roles [ingest, data_frozen, ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold]
[2023-04-11T10:17:54,025][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [yamlRestTest-0] [controller/63136] [Main.cc@123] controller (64 bit): Version 8.8.0-SNAPSHOT (Build 4adddac391a6f5) Copyright (c) 2023 Elasticsearch BV
[2023-04-11T10:17:54,144][INFO ][o.e.x.s.Security         ] [yamlRestTest-0] Security is enabled
[2023-04-11T10:17:54,521][INFO ][o.e.x.s.a.s.FileRolesStore] [yamlRestTest-0] parsed [1] roles from file [/Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/config/roles.yml]
[2023-04-11T10:17:54,737][INFO ][o.e.p.Preallocate        ] [yamlRestTest-0] pre-allocating cache file [/Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/data/shared_snapshot_cache] (16777216 bytes) using native methods
[2023-04-11T10:17:54,739][ERROR][o.e.b.Elasticsearch      ] [yamlRestTest-0] fatal exception while booting Elasticsearch java.lang.Error: Exception reading field 'fst_flags' in class org.elasticsearch.preallocate.MacOsPreallocator$Natives$Fcntl$FStore
    at com.sun.jna@5.10.0/com.sun.jna.Structure.getFieldValue(Structure.java:653)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.deriveLayout(Structure.java:1318)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.calculateSize(Structure.java:1192)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.calculateSize(Structure.java:1144)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.allocateMemory(Structure.java:426)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.<init>(Structure.java:216)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.<init>(Structure.java:204)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.<init>(Structure.java:191)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.<init>(Structure.java:183)
    at org.elasticsearch.preallocate@8.8.0-SNAPSHOT/org.elasticsearch.preallocate.MacOsPreallocator$Natives$Fcntl$FStore.<init>(MacOsPreallocator.java:83)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
    at org.elasticsearch.preallocate@8.8.0-SNAPSHOT/org.elasticsearch.preallocate.MacOsPreallocator.preallocate(MacOsPreallocator.java:30)
    at org.elasticsearch.preallocate@8.8.0-SNAPSHOT/org.elasticsearch.preallocate.Preallocate.preallocate(Preallocate.java:55)
    at org.elasticsearch.preallocate@8.8.0-SNAPSHOT/org.elasticsearch.preallocate.Preallocate.preallocate(Preallocate.java:39)
    at org.elasticsearch.blobcache@8.8.0-SNAPSHOT/org.elasticsearch.blobcache.shared.SharedBytes.<init>(SharedBytes.java:75)
    at org.elasticsearch.blobcache@8.8.0-SNAPSHOT/org.elasticsearch.blobcache.shared.SharedBlobCacheService.<init>(SharedBlobCacheService.java:293)
    at org.elasticsearch.searchablesnapshots@8.8.0-SNAPSHOT/org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshots.createComponents(SearchableSnapshots.java:344)
    at org.elasticsearch.server@8.8.0-SNAPSHOT/org.elasticsearch.node.Node.lambda$new$16(Node.java:723)
    at org.elasticsearch.server@8.8.0-SNAPSHOT/org.elasticsearch.plugins.PluginsService.lambda$flatMap$2(PluginsService.java:266)
    at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
    at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
    at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
    at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
    at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
    at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
    at org.elasticsearch.server@8.8.0-SNAPSHOT/org.elasticsearch.node.Node.<init>(Node.java:738)
    at org.elasticsearch.server@8.8.0-SNAPSHOT/org.elasticsearch.node.Node.<init>(Node.java:326)
    at org.elasticsearch.server@8.8.0-SNAPSHOT/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216)
    at org.elasticsearch.server@8.8.0-SNAPSHOT/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216)
    at org.elasticsearch.server@8.8.0-SNAPSHOT/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67)
Caused by: java.lang.IllegalAccessException: class com.sun.jna.Structure (in module com.sun.jna) cannot access class org.elasticsearch.preallocate.MacOsPreallocator$Natives$Fcntl$FStore (in module org.elasticsearch.preallocate) because module org.elasticsearch.preallocate does not export org.elasticsearch.preallocate to module com.sun.jna
    at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:394)
    at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:709)
    at java.base/java.lang.reflect.Field.checkAccess(Field.java:1154)
    at java.base/java.lang.reflect.Field.get(Field.java:439)
    at com.sun.jna@5.10.0/com.sun.jna.Structure.getFieldValue(Structure.java:650)
    ... 33 more

ERROR: Elasticsearch did not exit normally - check the logs at /Users/davidkyle/source/elasticsearch/x-pack/plugin/build/testclusters/yamlRestTest-0/logs/yamlRestTest.log

ERROR: Elasticsearch exited unexpectedly
[2023-04-11T09:17:55.511678Z] [BUILD] Stopping node

And the log for a successful start using ./gradlew run

[2023-04-11T09:22:08.235585Z] [BUILD] Copying additional config files from distro [/Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/jvm.options.d, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/users_roles, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/role_mapping.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/users, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/elasticsearch.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/roles.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/log4j2.properties, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/elasticsearch-plugins.example.yml, /Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/config/jvm.options]
[2023-04-11T09:22:08.238172Z] [BUILD] Creating elasticsearch keystore with password set to []
[2023-04-11T09:22:09.050365Z] [BUILD] Adding 1 keystore settings and 0 keystore files
[2023-04-11T09:22:09.739501Z] [BUILD] Installing 0 modules
[2023-04-11T09:22:09.739788Z] [BUILD] Setting up 1 users
[2023-04-11T09:22:10.531505Z] [BUILD] Setting up roles.yml
[2023-04-11T09:22:10.532076Z] [BUILD] Starting Elasticsearch process
[2023-04-11T10:22:12,388][INFO ][o.e.c.u.FeatureFlag      ] [runTask-0] The current build is a snapshot, feature flag [untrusted_remote_cluster] is enabled
[2023-04-11T10:22:12,767][INFO ][o.e.n.Node               ] [runTask-0] version[8.8.0-SNAPSHOT], pid[63272], build[tar/67ccbd8aabd0047327322920168dde2106b1f37e/2023-04-11T08:32:27.730052Z], OS[Mac OS X/13.3/aarch64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/20/20+36-2344]
[2023-04-11T10:22:12,768][INFO ][o.e.n.Node               ] [runTask-0] JVM home [/Users/davidkyle/.gradle/caches/transforms-3/9724627d5cf99175aa30a9bab0366ce2/transformed/macos-20-aarch64.tar.gz/Contents/Home], using bundled JDK [false]
[2023-04-11T10:22:12,768][INFO ][o.e.n.Node               ] [runTask-0] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=/Users/davidkyle/source/elasticsearch/build/testclusters/runTask-0/tmp, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=/Users/davidkyle/source/elasticsearch/build/testclusters/runTask-0/logs, -XX:ErrorFile=/Users/davidkyle/source/elasticsearch/build/testclusters/runTask-0/logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/Users/davidkyle/source/elasticsearch/build/testclusters/runTask-0/logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms512m, -Xmx512m, -ea, -esa, -Dingest.geoip.downloader.enabled.default=true, -Dio.netty.leakDetection.level=paranoid, -XX:MaxDirectMemorySize=268435456, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/lib, --add-modules=jdk.net, --add-modules=org.elasticsearch.preallocate, -Djdk.module.main=org.elasticsearch.server]
[2023-04-11T10:22:12,769][WARN ][o.e.n.Node               ] [runTask-0] version [8.8.0-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production
[2023-04-11T10:22:13,840][WARN ][stderr                   ] [runTask-0] SLF4J: No SLF4J providers were found.
[2023-04-11T10:22:13,840][WARN ][stderr                   ] [runTask-0] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2023-04-11T10:22:13,841][WARN ][stderr                   ] [runTask-0] SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
[2023-04-11T10:22:13,841][WARN ][stderr                   ] [runTask-0] SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
[2023-04-11T10:22:13,841][WARN ][stderr                   ] [runTask-0] SLF4J: Ignoring binding found at [jar:file:///Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/modules/repository-azure/log4j-slf4j-impl-2.19.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[2023-04-11T10:22:13,841][WARN ][stderr                   ] [runTask-0] SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
[2023-04-11T10:22:14,582][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [repository-url]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [dlm]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [ingest-user-agent]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-core]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-async-search]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [test-error-query]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-monitoring]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [repository-s3]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-analytics]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-ent-search]
[2023-04-11T10:22:14,583][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-autoscaling]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [lang-painless]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-ml]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [test-die-with-dignity]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [legacy-geo]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [lang-mustache]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-ql]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [analysis-common]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [test-seek-tracking-directory]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [transport-netty4]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [aggregations]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [ingest-common]
[2023-04-11T10:22:14,584][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-identity-provider]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [frozen-indices]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-text-structure]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-shutdown]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [snapshot-repo-test-kit]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [test-delayed-aggs]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [kibana]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [constant-keyword]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-logstash]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-graph]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-ccr]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [parent-join]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-enrich]
[2023-04-11T10:22:14,585][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [repositories-metering-api]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [transform]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [repository-azure]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [repository-gcs]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [spatial]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [apm]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [mapper-extras]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [mapper-version]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-aggregate-metric]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-rollup]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [percolator]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [data-streams]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-stack]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [rank-eval]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [reindex]
[2023-04-11T10:22:14,586][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-security]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [blob-cache]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [searchable-snapshots]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [snapshot-based-recoveries]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-watcher]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [old-lucene-versions]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-ilm]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-voting-only-node]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-deprecation]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-fleet]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-profiling]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [ingest-geoip]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-write-load-forecaster]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [search-business-rules]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [wildcard]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [ingest-attachment]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [unsigned-long]
[2023-04-11T10:22:14,587][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-sql]
[2023-04-11T10:22:14,588][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [runtime-fields-common]
[2023-04-11T10:22:14,588][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-async]
[2023-04-11T10:22:14,588][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [vector-tile]
[2023-04-11T10:22:14,588][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [lang-expression]
[2023-04-11T10:22:14,588][INFO ][o.e.p.PluginsService     ] [runTask-0] loaded module [x-pack-eql]
[2023-04-11T10:22:14,669][INFO ][o.e.c.u.FeatureFlag      ] [runTask-0] The current build is a snapshot, feature flag [dlm] is enabled
[2023-04-11T10:22:15,766][INFO ][o.e.c.u.FeatureFlag      ] [runTask-0] The current build is a snapshot, feature flag [index_mode] is enabled
[2023-04-11T10:22:15,999][WARN ][stderr                   ] [runTask-0] Apr 11, 2023 10:22:15 AM org.apache.lucene.store.MemorySegmentIndexInputProvider <init>
[2023-04-11T10:22:16,004][INFO ][o.e.e.NodeEnvironment    ] [runTask-0] using [1] data paths, mounts [[/System/Volumes/Data (/dev/disk3s5)]], net usable_space [676.5gb], net total_space [926.3gb], types [apfs]
[2023-04-11T10:22:16,004][INFO ][o.e.e.NodeEnvironment    ] [runTask-0] heap size [512mb], compressed ordinary object pointers [true]
[2023-04-11T10:22:16,014][INFO ][o.e.n.Node               ] [runTask-0] node name [runTask-0], node ID [KYaPm967SF2w1vNlSm6vaw], cluster name [runTask], roles [ingest, data_frozen, ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold]
[2023-04-11T10:22:17,034][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [runTask-0] [controller/63273] [Main.cc@123] controller (64 bit): Version 8.8.0-SNAPSHOT (Build 4adddac391a6f5) Copyright (c) 2023 Elasticsearch BV
[2023-04-11T10:22:17,175][INFO ][o.e.x.s.Security         ] [runTask-0] Security is enabled
[2023-04-11T10:22:17,507][INFO ][o.e.x.s.a.s.FileRolesStore] [runTask-0] parsed [1] roles from file [/Users/davidkyle/source/elasticsearch/build/testclusters/runTask-0/config/roles.yml]
[2023-04-11T10:22:17,843][INFO ][o.e.x.p.ProfilingPlugin  ] [runTask-0] Profiling is enabled
[2023-04-11T10:22:18,134][WARN ][stderr                   ] [runTask-0] SLF4J: No SLF4J providers were found.
[2023-04-11T10:22:18,134][WARN ][stderr                   ] [runTask-0] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2023-04-11T10:22:18,134][WARN ][stderr                   ] [runTask-0] SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
[2023-04-11T10:22:18,135][WARN ][stderr                   ] [runTask-0] SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
[2023-04-11T10:22:18,135][WARN ][stderr                   ] [runTask-0] SLF4J: Ignoring binding found at [jar:file:///Users/davidkyle/source/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.8.0-SNAPSHOT/modules/x-pack-security/log4j-slf4j-impl-2.19.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[2023-04-11T10:22:18,135][WARN ][stderr                   ] [runTask-0] SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
[2023-04-11T10:22:18,152][INFO ][o.e.t.n.NettyAllocator   ] [runTask-0] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=4mb, heap_size=512mb}]
[2023-04-11T10:22:18,166][INFO ][o.e.i.r.RecoverySettings ] [runTask-0] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2023-04-11T10:22:18,187][INFO ][o.e.d.DiscoveryModule    ] [runTask-0] using discovery type [multi-node] and seed hosts providers [settings, file]
[2023-04-11T10:22:18,723][INFO ][o.e.n.Node               ] [runTask-0] initialized
[2023-04-11T10:22:18,724][INFO ][o.e.n.Node               ] [runTask-0] starting ...
[2023-04-11T10:22:18,745][INFO ][o.e.x.s.c.f.PersistentCache] [runTask-0] persistent cache index loaded
[2023-04-11T10:22:18,746][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [runTask-0] deprecation component started
[2023-04-11T10:22:18,799][INFO ][o.e.t.TransportService   ] [runTask-0] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2023-04-11T10:22:18,882][WARN ][o.e.b.BootstrapChecks    ] [runTask-0] Transport SSL must be enabled if security is enabled. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]
[2023-04-11T10:22:18,883][INFO ][o.e.c.c.ClusterBootstrapService] [runTask-0] this node has not joined a bootstrapped cluster yet; [cluster.initial_master_nodes] is set to [runTask-0]
[2023-04-11T10:22:18,886][WARN ][o.e.d.FileBasedSeedHostsProvider] [runTask-0] expected, but did not find, a dynamic hosts list at [/Users/davidkyle/source/elasticsearch/build/testclusters/runTask-0/config/unicast_hosts.txt]
[2023-04-11T10:22:18,886][INFO ][o.e.c.c.Coordinator      ] [runTask-0] setting initial configuration to VotingConfiguration{KYaPm967SF2w1vNlSm6vaw}
[2023-04-11T10:22:18,934][INFO ][o.e.h.AbstractHttpServerTransport] [runTask-0] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2023-04-11T10:22:18,936][INFO ][o.e.n.Node               ] [runTask-0] started {runTask-0}{KYaPm967SF2w1vNlSm6vaw}{YXf9vn9gQL6vK3j8iugdKA}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.8.0}{ml.max_jvm_size=536870912, ml.allocated_processors=10, testattr=test, ml.allocated_processors_double=10.0, ml.machine_memory=34359738368, xpack.installed=true}
[2023-04-11T10:22:19,048][INFO ][o.e.c.s.MasterService    ] [runTask-0] elected-as-master ([1] nodes joined in term 1)[_FINISH_ELECTION_, {runTask-0}{KYaPm967SF2w1vNlSm6vaw}{YXf9vn9gQL6vK3j8iugdKA}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.8.0} completing election], term: 1, version: 1, delta: master node changed {previous [], current [{runTask-0}{KYaPm967SF2w1vNlSm6vaw}{YXf9vn9gQL6vK3j8iugdKA}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.8.0}]}
[2023-04-11T10:22:19,081][INFO ][o.e.c.c.CoordinationState] [runTask-0] cluster UUID set to [S22Qy2hSSaKcv5py1KgPOw]
[2023-04-11T10:22:19,104][INFO ][o.e.c.s.ClusterApplierService] [runTask-0] master node changed {previous [], current [{runTask-0}{KYaPm967SF2w1vNlSm6vaw}{YXf9vn9gQL6vK3j8iugdKA}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.8.0}]}, term: 1, version: 1, reason: Publication{term=1, version=1}
[2023-04-11T10:22:19,134][INFO ][o.e.r.s.AbstractFileWatchingService] [runTask-0] starting file watcher ...
[2023-04-11T10:22:19,137][INFO ][o.e.r.s.AbstractFileWatchingService] [runTask-0] file settings service up and running [tid=58]
[2023-04-11T10:22:19,139][INFO ][o.e.c.c.NodeJoinExecutor ] [runTask-0] node-join: [{runTask-0}{KYaPm967SF2w1vNlSm6vaw}{YXf9vn9gQL6vK3j8iugdKA}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.8.0}] with reason [completing election]
[2023-04-11T10:22:19,180][INFO ][o.e.g.GatewayService     ] [runTask-0] recovered [0] indices into cluster_state
[2023-04-11T10:22:19,249][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [behavioral_analytics-events-mappings]
[2023-04-11T10:22:19,256][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-state] for index patterns [.ml-state*]
[2023-04-11T10:22:19,277][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-kibana-mb] for index patterns [.monitoring-kibana-8-*]
[2023-04-11T10:22:19,279][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [behavioral_analytics-events-settings]
[2023-04-11T10:22:19,284][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-notifications-000002] for index patterns [.ml-notifications-000002]
[2023-04-11T10:22:19,288][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-stats] for index patterns [.ml-stats-*]
[2023-04-11T10:22:19,302][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-logstash-mb] for index patterns [.monitoring-logstash-8-*]
[2023-04-11T10:22:19,315][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-anomalies-] for index patterns [.ml-anomalies-*]
[2023-04-11T10:22:19,319][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-7-*]
[2023-04-11T10:22:19,328][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-beats] for index patterns [.monitoring-beats-7-*]
[2023-04-11T10:22:19,338][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-es] for index patterns [.monitoring-es-7-*]
[2023-04-11T10:22:19,344][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-7-*]
[2023-04-11T10:22:19,346][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-alerts-7] for index patterns [.monitoring-alerts-7]
[2023-04-11T10:22:19,358][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-ent-search-mb] for index patterns [.monitoring-ent-search-8-*]
[2023-04-11T10:22:19,382][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-beats-mb] for index patterns [.monitoring-beats-8-*]
[2023-04-11T10:22:19,415][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-es-mb] for index patterns [.monitoring-es-8-*]
[2023-04-11T10:22:19,417][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics-settings]
[2023-04-11T10:22:19,418][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-settings]
[2023-04-11T10:22:19,422][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-mappings]
[2023-04-11T10:22:19,424][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs-mappings]
[2023-04-11T10:22:19,425][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics-mappings]
[2023-04-11T10:22:19,426][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs-settings]
[2023-04-11T10:22:19,432][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [data-streams-mappings]
[2023-04-11T10:22:19,437][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [ilm-history] for index patterns [ilm-history-5*]
[2023-04-11T10:22:19,446][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.watch-history-16] for index patterns [.watcher-history-16*]
[2023-04-11T10:22:19,448][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [.deprecation-indexing-mappings]
[2023-04-11T10:22:19,452][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.slm-history] for index patterns [.slm-history-5*]
[2023-04-11T10:22:19,454][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-file-data] for index patterns [.fleet-file-data-*-*]
[2023-04-11T10:22:19,457][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-files] for index patterns [.fleet-files-*-*]
[2023-04-11T10:22:19,458][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [.deprecation-indexing-settings]
[2023-04-11T10:22:19,524][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [behavioral_analytics-events-default] for index patterns [behavioral_analytics-events-*]
[2023-04-11T10:22:19,527][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [logs] for index patterns [logs-*-*]
[2023-04-11T10:22:19,530][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [synthetics] for index patterns [synthetics-*-*]
[2023-04-11T10:22:19,533][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.deprecation-indexing-template] for index patterns [.logs-deprecation.*]
[2023-04-11T10:22:19,537][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [metrics] for index patterns [metrics-*-*]
[2023-04-11T10:22:19,564][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.monitoring-8-ilm-policy]
[2023-04-11T10:22:19,603][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [ml-size-based-ilm-policy]
[2023-04-11T10:22:19,634][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [behavioral_analytics-events-default_policy]
[2023-04-11T10:22:19,676][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [metrics]
[2023-04-11T10:22:19,706][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [synthetics]
[2023-04-11T10:22:19,742][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [7-days-default]
[2023-04-11T10:22:19,771][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [30-days-default]
[2023-04-11T10:22:19,809][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [logs]
[2023-04-11T10:22:19,842][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [90-days-default]
[2023-04-11T10:22:19,877][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [365-days-default]
[2023-04-11T10:22:19,904][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [180-days-default]
[2023-04-11T10:22:19,938][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [watch-history-ilm-policy-16]
[2023-04-11T10:22:19,970][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [ilm-history-ilm-policy]
[2023-04-11T10:22:19,995][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.deprecation-indexing-ilm-policy]
[2023-04-11T10:22:20,021][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [slm-history-ilm-policy]
[2023-04-11T10:22:20,059][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-actions-results-ilm-policy]
[2023-04-11T10:22:20,084][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-files-ilm-policy]
[2023-04-11T10:22:20,109][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-data-ilm-policy]
[2023-04-11T10:22:20,199][INFO ][o.e.h.n.s.HealthNodeTaskExecutor] [runTask-0] Node [{runTask-0}{KYaPm967SF2w1vNlSm6vaw}] is selected as the current health node.
[2023-04-11T10:22:20,286][INFO ][o.e.l.ClusterStateLicenseService] [runTask-0] license [54c6670a-0845-4969-b0d9-00f15fdb3b47] mode [basic] - valid
[2023-04-11T10:22:20,287][INFO ][o.e.x.s.a.Realms         ] [runTask-0] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
elasticsearchmachine commented 1 year ago

Pinging @elastic/es-core-infra (Team:Core/Infra)

salvatore-campagna commented 1 year ago

I am hitting this issue locally while running the following YAML test

./gradlew ':x-pack:plugin:ilm:qa:multi-node:javaRestTest' --tests "org.elasticsearch.xpack.ilm.actions.DownsampleActionIT.testRollupIndex" -Dtests.seed=87E7B73A4FA3E50A -Dtests.locale=pl-PL -Dtests.timezone=Asia/Qatar

I am running it to try to reproduce a CI test failure before trying to mute the test.

salvatore-campagna commented 1 year ago

I fixed the issue locally applying the following change (suggested by @pgomulka)

diff --git a/libs/preallocate/src/main/java/module-info.java b/libs/preallocate/src/main/java/module-info.java
index 1eb89daa607..89c85d95ab2 100644
--- a/libs/preallocate/src/main/java/module-info.java
+++ b/libs/preallocate/src/main/java/module-info.java
@@ -11,7 +11,7 @@ module org.elasticsearch.preallocate {
     requires org.elasticsearch.logging;
     requires com.sun.jna;

-    exports org.elasticsearch.preallocate to org.elasticsearch.blobcache;
+    exports org.elasticsearch.preallocate to org.elasticsearch.blobcache, com.sun.jna;