search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
984 stars 24.82k forks source link

[CI] KeystoreManagementTests test42KeystorePasswordOnTtyRunningInForeground failing #84204

Closed tvernum closed 2 years ago

tvernum commented 2 years ago

Failed for me (twice) on a PR build, but seems to also be failing in 7.17 For example: https://gradle-enterprise.elastic.co/s/pewkycvqsaedo

Build scan: https://gradle-enterprise.elastic.co/s/r2c5syvnxcr5c/tests/:qa:os:destructiveDistroTest.default-linux-archive-no-jdk/org.elasticsearch.packaging.test.KeystoreManagementTests/test42KeystorePasswordOnTtyRunningInForeground

Reproduction line: null

Applicable branches: 7.17

Reproduces locally?: Didn't try

Failure history: https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.packaging.test.KeystoreManagementTests&tests.test=test42KeystorePasswordOnTtyRunningInForeground

Failure excerpt:

java.lang.AssertionError: Startup command should succeed. Stderr: [exitCode = [1] stdout = [spawn -ignore HUP sudo -E -u elasticsearch /tmp/elasticsearch/bin/elasticsearch -p /tmp/elasticsearch/elasticsearch.pid
Future versions of Elasticsearch will require Java 11; your Java version from [/var/lib/jenkins/.java/oracle-8u301-linux/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
warning: no-jdk distributions that do not bundle a JDK are deprecated and will be removed in a future release
Future versions of Elasticsearch will require Java 11; your Java version from [/var/lib/jenkins/.java/oracle-8u301-linux/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
warning: no-jdk distributions that do not bundle a JDK are deprecated and will be removed in a future release
Elasticsearch keystore password: [2022-02-22T01:35:27,712][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] version[7.17.1-SNAPSHOT], pid[27419], build[default/tar/c2a89bdbb0b1462f1ad8b17e4332b374873c83a8/2022-02-22T01:15:27.240639745Z], OS[Linux/5.11.0-1029-gcp/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_301/25.301-b09]
[2022-02-22T01:35:27,713][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] JVM home [/var/lib/jenkins/.java/oracle-8u301-linux/jre]
[2022-02-22T01:35:27,714][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,JRE, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-465560516198596817, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xmx1g, -Xms1g, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/tmp/elasticsearch, -Des.path.conf=/tmp/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=false]
[2022-02-22T01:35:27,714][WARN ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] version [7.17.1-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production
[2022-02-22T01:35:30,687][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [aggs-matrix-stats]
[2022-02-22T01:35:30,688][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [analysis-common]
[2022-02-22T01:35:30,688][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [constant-keyword]
[2022-02-22T01:35:30,688][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [frozen-indices]
[2022-02-22T01:35:30,688][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [ingest-common]
[2022-02-22T01:35:30,689][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [ingest-geoip]
[2022-02-22T01:35:30,689][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [ingest-user-agent]
[2022-02-22T01:35:30,689][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [kibana]
[2022-02-22T01:35:30,689][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [lang-expression]
[2022-02-22T01:35:30,690][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [lang-mustache]
[2022-02-22T01:35:30,690][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [lang-painless]
[2022-02-22T01:35:30,690][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [legacy-geo]
[2022-02-22T01:35:30,690][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [mapper-extras]
[2022-02-22T01:35:30,690][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [mapper-version]
[2022-02-22T01:35:30,691][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [parent-join]
[2022-02-22T01:35:30,691][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [percolator]
[2022-02-22T01:35:30,691][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [rank-eval]
[2022-02-22T01:35:30,691][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [reindex]
[2022-02-22T01:35:30,692][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [repositories-metering-api]
[2022-02-22T01:35:30,692][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [repository-encrypted]
[2022-02-22T01:35:30,692][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [repository-url]
[2022-02-22T01:35:30,692][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [runtime-fields-common]
[2022-02-22T01:35:30,692][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [search-business-rules]
[2022-02-22T01:35:30,692][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [searchable-snapshots]
[2022-02-22T01:35:30,693][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [snapshot-repo-test-kit]
[2022-02-22T01:35:30,693][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [spatial]
[2022-02-22T01:35:30,693][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [test-delayed-aggs]
[2022-02-22T01:35:30,693][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [test-die-with-dignity]
[2022-02-22T01:35:30,693][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [test-error-query]
[2022-02-22T01:35:30,694][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [transform]
[2022-02-22T01:35:30,694][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [transport-netty4]
[2022-02-22T01:35:30,694][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [unsigned-long]
[2022-02-22T01:35:30,694][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [vector-tile]
[2022-02-22T01:35:30,694][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [vectors]
[2022-02-22T01:35:30,695][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [wildcard]
[2022-02-22T01:35:30,695][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-aggregate-metric]
[2022-02-22T01:35:30,695][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-analytics]
[2022-02-22T01:35:30,695][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-async]
[2022-02-22T01:35:30,696][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-async-search]
[2022-02-22T01:35:30,696][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-autoscaling]
[2022-02-22T01:35:30,696][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-ccr]
[2022-02-22T01:35:30,696][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-core]
[2022-02-22T01:35:30,696][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-data-streams]
[2022-02-22T01:35:30,697][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-deprecation]
[2022-02-22T01:35:30,697][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-enrich]
[2022-02-22T01:35:30,697][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-eql]
[2022-02-22T01:35:30,697][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-fleet]
[2022-02-22T01:35:30,697][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-graph]
[2022-02-22T01:35:30,697][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-identity-provider]
[2022-02-22T01:35:30,698][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-ilm]
[2022-02-22T01:35:30,698][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-logstash]
[2022-02-22T01:35:30,698][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-ml]
[2022-02-22T01:35:30,698][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-monitoring]
[2022-02-22T01:35:30,698][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-ql]
[2022-02-22T01:35:30,699][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-rollup]
[2022-02-22T01:35:30,699][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-security]
[2022-02-22T01:35:30,699][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-shutdown]
[2022-02-22T01:35:30,699][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-sql]
[2022-02-22T01:35:30,699][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-stack]
[2022-02-22T01:35:30,699][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-text-structure]
[2022-02-22T01:35:30,700][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-voting-only-node]
[2022-02-22T01:35:30,700][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] loaded module [x-pack-watcher]
[2022-02-22T01:35:30,700][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] no plugins loaded
[2022-02-22T01:35:30,743][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] using [1] data paths, mounts [[/ (/dev/root)]], net usable_space [294.6gb], net total_space [339gb], types [ext4]
[2022-02-22T01:35:30,743][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] heap size [989.8mb], compressed ordinary object pointers [true]
[2022-02-22T01:35:30,818][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] node name [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856], node ID [yJRPenSyQ762GdZKQ03QpA], cluster name [elasticsearch], roles [transform, data_frozen, master, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold, ingest]
[2022-02-22T01:35:35,726][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] [controller/27738] [Main.cc@122] controller (64 bit): Version 7.17.1-SNAPSHOT (Build 6d8a28b39bf223) Copyright (c) 2022 Elasticsearch BV
[2022-02-22T01:35:36,245][INFO ][o.e.x.s.a.s.FileRolesStore] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] parsed [0] roles from file [/tmp/elasticsearch/config/roles.yml]
[2022-02-22T01:35:37,291][INFO ][o.e.i.g.ConfigDatabases  ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] initialized default databases [[GeoLite2-Country.mmdb, GeoLite2-City.mmdb, GeoLite2-ASN.mmdb]], config databases [[]] and watching [/tmp/elasticsearch/config/ingest-geoip] for changes
[2022-02-22T01:35:37,292][INFO ][o.e.i.g.DatabaseNodeService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] initialized database registry, using geoip-databases directory [/tmp/elasticsearch-465560516198596817/geoip-databases/yJRPenSyQ762GdZKQ03QpA]
[2022-02-22T01:35:38,176][INFO ][o.e.t.NettyAllocator     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=false, g1gc_region_size=0b, heap_size=989.8mb}]
[2022-02-22T01:35:38,219][INFO ][o.e.i.r.RecoverySettings ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2022-02-22T01:35:38,276][INFO ][o.e.d.DiscoveryModule    ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] using discovery type [zen] and seed hosts providers [settings]
[2022-02-22T01:35:39,018][INFO ][o.e.g.DanglingIndicesState] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-02-22T01:35:39,747][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] initialized
[2022-02-22T01:35:39,747][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] starting ...
[2022-02-22T01:35:39,764][INFO ][o.e.x.s.c.f.PersistentCache] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] persistent cache index loaded
[2022-02-22T01:35:39,766][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] deprecation component started
[2022-02-22T01:35:39,928][INFO ][o.e.t.TransportService   ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2022-02-22T01:35:40,205][WARN ][o.e.b.BootstrapChecks    ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2022-02-22T01:35:40,208][INFO ][o.e.c.c.Coordinator      ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] cluster UUID [NKkwAOSzQ9i-dQkFn6BHUQ]
[2022-02-22T01:35:40,223][INFO ][o.e.c.c.ClusterBootstrapService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2022-02-22T01:35:40,355][INFO ][o.e.c.s.MasterService    ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] elected-as-master ([1] nodes joined)[{elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856}{yJRPenSyQ762GdZKQ03QpA}{l9BMspCzRle9C0gVu_EQSQ}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 3, version: 48, delta: master node changed {previous [], current [{elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856}{yJRPenSyQ762GdZKQ03QpA}{l9BMspCzRle9C0gVu_EQSQ}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}
[2022-02-22T01:35:40,435][INFO ][o.e.c.s.ClusterApplierService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] master node changed {previous [], current [{elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856}{yJRPenSyQ762GdZKQ03QpA}{l9BMspCzRle9C0gVu_EQSQ}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}, term: 3, version: 48, reason: Publication{term=3, version=48}
[2022-02-22T01:35:40,509][INFO ][o.e.h.AbstractHttpServerTransport] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2022-02-22T01:35:40,510][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] started
[2022-02-22T01:35:40,890][INFO ][o.e.l.LicenseService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] license [c1845215-18ad-4da1-a827-34ed0cd5d3be] mode [basic] - valid
[2022-02-22T01:35:40,892][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] Active license is now [BASIC]; Security is disabled
[2022-02-22T01:35:40,893][WARN ][o.e.x.s.s.SecurityStatusChangeListener] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security.
[2022-02-22T01:35:40,904][INFO ][o.e.g.GatewayService     ] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] recovered [0] indices into cluster_state
[2022-02-22T01:35:42,591][INFO ][o.e.c.m.MetadataCreateIndexService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] [.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001] creating index, cause [initialize_data_stream], templates [.deprecation-indexing-template], shards [1]/[1]
[2022-02-22T01:35:42,606][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] updating number_of_replicas to [0] for indices [.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001]
[2022-02-22T01:35:42,626][INFO ][o.e.c.m.MetadataCreateDataStreamService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] adding data stream [.logs-deprecation.elasticsearch-default] with write index [.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001], backing indices [], and aliases []
[2022-02-22T01:35:42,778][INFO ][o.e.x.i.IndexLifecycleTransition] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] moving index [.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [.deprecation-indexing-ilm-policy]
[2022-02-22T01:35:42,849][INFO ][o.e.x.i.IndexLifecycleTransition] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] moving index [.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] in policy [.deprecation-indexing-ilm-policy]
[2022-02-22T01:35:42,901][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001][0]]]).
[2022-02-22T01:35:42,950][INFO ][o.e.x.i.IndexLifecycleTransition] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] moving index [.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [.deprecation-indexing-ilm-policy]
[2022-02-22T01:35:43,040][INFO ][o.e.c.m.MetadataMappingService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] [.ds-.logs-deprecation.elasticsearch-default-2022.02.22-000001/XAFoMdQbR5ymEJPumRvl3g] update_mapping [_doc]
[2022-02-22T01:35:47,285][INFO ][o.e.c.m.MetadataCreateIndexService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] [.ds-ilm-history-5-2022.02.22-000001] creating index, cause [initialize_data_stream], templates [ilm-history], shards [1]/[0]
[2022-02-22T01:35:47,287][INFO ][o.e.c.m.MetadataCreateDataStreamService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] adding data stream [ilm-history-5] with write index [.ds-ilm-history-5-2022.02.22-000001], backing indices [], and aliases []
[2022-02-22T01:35:47,352][INFO ][o.e.x.i.IndexLifecycleTransition] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] moving index [.ds-ilm-history-5-2022.02.22-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [ilm-history-ilm-policy]
[2022-02-22T01:35:47,398][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ds-ilm-history-5-2022.02.22-000001][0]]]).
[2022-02-22T01:35:47,442][INFO ][o.e.x.i.IndexLifecycleTransition] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] moving index [.ds-ilm-history-5-2022.02.22-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] in policy [ilm-history-ilm-policy]
[2022-02-22T01:35:47,490][INFO ][o.e.x.i.IndexLifecycleTransition] [elasticsearch-ci-immutable-ubuntu-2004-pkg-1645492399929529856] moving index [.ds-ilm-history-5-2022.02.22-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [ilm-history-ilm-policy]

Timed out waiting for startup to succeed] stderr = []]
Expected: <0>
     but: was <1>

  at __randomizedtesting.SeedInfo.seed([8D49449E4CE5B955:2098227AD9019E4A]:0)
  at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
  at org.junit.Assert.assertThat(Assert.java:956)
  at org.elasticsearch.packaging.test.PackagingTestCase.awaitElasticsearchStartup(PackagingTestCase.java:339)
  at org.elasticsearch.packaging.test.KeystoreManagementTests.test42KeystorePasswordOnTtyRunningInForeground(KeystoreManagementTests.java:183)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
  at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
  at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
  at java.lang.Thread.run(Thread.java:748)
elasticmachine commented 2 years ago

Pinging @elastic/es-delivery (Team:Delivery)

tvernum commented 2 years ago

This reproduction

./gradlew :qa:os:debian-8:distroTest.default-linux-archive-no-jdk --tests "org.elasticsearch.packaging.test.KeystoreManagementTests"

Seems to fail more often on the branch for my PR (https://github.com/elastic/elasticsearch/pull/84180) but I don't know whether my results are statistically significant.

My guess is that something has just pushed it to the point of a timeout, but I'm not set up to debug the expect script and work out what's going on there.

elasticmachine commented 2 years ago

Pinging @elastic/es-core-infra (Team:Core/Infra)

pugnascotia commented 2 years ago

I fiddled with this most recently, I'll take a look.

cbuescher commented 2 years ago

Another one here: https://gradle-enterprise.elastic.co/s/lwoju5ciheuew

cbuescher commented 2 years ago

Several other earlier today, only seems to be 7.17 branch as far as I can see, all "multijob+packaging-tests-unix"

pugnascotia commented 2 years ago

Muted in 7.17 in d90d673f5e5

pugnascotia commented 2 years ago

Gyah - I think I broke test44WrongKeystorePasswordOnTty.

pugnascotia commented 2 years ago

I can't repro the failure now, so I'm going to assume it was a one off.

pugnascotia commented 2 years ago

Definitely not a one-off 😞