[RELEASE] Release version 2.3.0

[github-actions[bot]]

Release OpenSearch and OpenSearch Dashboards 2.3.0

I noticed that a manifest was automatically created in manifests/2.3.0. Please follow the following checklist to make a release.

How to use this issue

## This Release Issue This issue captures the state of the OpenSearch release, its assignee is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. More details are included in the Maintainers [Release owner](https://github.com/opensearch-project/opensearch-build/blob/main/MAINTAINERS.md#release-owner) section. ## Release Steps There are several steps to the release process, these steps are completed as the whole release and components that are behind present risk to the release. The release owner completes the tasks in this ticket, whereas component owners resolve tasks on their ticket in their repositories. Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release. ### Component List To aid in understanding the state of the release there is a table with status indicating each component state. This is updated based on the status of the component issues.

Preparation

• [x] Assign this issue to a release owner.
• [x] Declare a pencils down date for new features to be merged.
• [x] September 02, 2022 is pencils down date for feature freeze.
• [x] Update the Campaigns section to include monitoring campaigns during this release.
• [x] Update this issue so all __REPLACE_RELEASE-__ placeholders have actual dates.
• [x] Document any new quality requirements or changes.
• [x] Finalize scope and feature set and update the Public Roadmap.
• [x] Create a release issue in every component repo that links back to this issue, update Components section with these links.
• [x] Ensure the label is created in each component repo for this new version, and the next minor release. Create a version label
• [x] Ensure that all release issues created above are assigned to an owner in the component team.

CI/CD (Feature Freeze) - Ends September 07, 2022

• [x] Create Jenkins workflows that run daily snapshot builds for OpenSearch and OpenSearch Dashboards.
• [x] Increment each component version to 2.3.0 and ensure working CI in component repositories.
• [x] Make pull requests to add each component to manifests/2.3.0/opensearch-2.3.0.yml and manifests/2.3.0/opensearch-dashboards-2.3.0.yml with the corresponding checks.
• [x] OpenSearch / OpenSearch-Dashboards core and components teams finalize their features
• [x] OpenSearch / OpenSearch-Dashboards core cut branch <MajorVersion>.<MinorVersion> early.

Campaigns

• [ ] https://github.com/opensearch-project/opensearch-plugins/issues/138
• [ ] https://github.com/opensearch-project/opensearch-plugins/issues/139
• [ ] https://github.com/opensearch-project/OpenSearch/issues/2589
  • [x] https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1318
  • [ ] https://github.com/opensearch-project/OpenSearch/issues/472
• [ ] https://github.com/opensearch-project/opensearch-build/issues/58
• [x] https://github.com/opensearch-project/opensearch-plugins/issues/142

Code Complete - Ends September 07, 2022

• [x] Code Complete: Make sure that the code for this specific version of the release is ready and the branch corresponding to this release has been added to this release version manifest.
• [x] Verify pull requests to add each component to manifests/2.3.0/opensearch-2.3.0.yml and manifests/2.3.0/opensearch-dashboards-2.3.0.yml have been merged.
• [x] Gather, review and combine the release notes from components repositories.

Release testing - Ends September 12, 2022

• [x] All components should have cut branch <MajorVersion>.<MinorVersion> for the release.
• [x] Declare a release candidate build, and provide the instructions with the release candidates for teams on testing (September 08, 2022).
• [x] Stop builds for this version of OpenSearch and/or OpenSearch Dashboards in order to avoid accidental commits going in unknowingly. Restart only if necessary else manually run the build workflow and declare new release candidate.
• [x] Sanity Testing (September 08, 2022 - September 12, 2022): Sanity testing and fixing of critical issues found by teams. Teams test their components within the distribution, ensuring integration, backwards compatibility, and perf tests pass.
• [ ] Publish all test results in the comments of this issue.

Performance testing validation - Ends September 12, 2022

• [x] Performance tests do not show a regression
• [x] Longevity tests do not show any issues

Release - Ends September 14 2022

• [x] Verify all issues labeled v2.3.0 in all projects have been resolved.
• [x] Complete documentation for this release.
• [x] Author blog post for this release.
• [x] September 13 2022 - Publish this release on opensearch.org.
• [x] September 14 2022 - Publish a blog post - release is launched!

Post Release

• [x] Create release tags for each component (Jenkins job name: release-tag-creation).
• [x] Replace refs in manifests/2.3.0 with tags and remove checks.
• [x] If this is a major or minor version release, stop building previous patch version.
• [x] Generate distribution release notes reviewed by PM team for opensearch-build repository.
• [x] Increment version for Helm Charts (sample PR) for the 2.3.0 release.
• [x] Increment version for Ansible Charts (sample PR) for the 2.3.0 release.
• [x] Prepare for next patch release by incrementing patch versions for each component.
• [x] Update this template with any new or missed steps.
• [x] Create an issue for a retrospective, solicit feedback, and publish a summary.

Components

Replace with links to all component tracking issues.

Component	On track	Release Notes
https://github.com/opensearch-project/OpenSearch/issues/4357
https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2230
https://github.com/opensearch-project/alerting/issues/534
https://github.com/opensearch-project/anomaly-detection/issues/648
https://github.com/opensearch-project/asynchronous-search/issues/173
https://github.com/opensearch-project/dashboards-reports/issues/440
https://github.com/opensearch-project/dashboards-visualizations/issues/110
https://github.com/opensearch-project/index-management/issues/481
https://github.com/opensearch-project/job-scheduler/issues/226
https://github.com/opensearch-project/k-NN/issues/524
https://github.com/opensearch-project/performance-analyzer/issues/260
https://github.com/opensearch-project/security-dashboards-plugin/issues/1089
https://github.com/opensearch-project/security/issues/2063
https://github.com/opensearch-project/sql/issues/795
https://github.com/opensearch-project/observability/issues/978
https://github.com/opensearch-project/alerting-dashboards-plugin/issues/324
https://github.com/opensearch-project/notifications/issues/516
https://github.com/opensearch-project/index-management-dashboards-plugin/issues/242
https://github.com/opensearch-project/anomaly-detection-dashboards-plugin/issues/316
https://github.com/opensearch-project/cross-cluster-replication/issues/535
https://github.com/opensearch-project/common-utils/issues/228
https://github.com/opensearch-project/ml-commons/issues/413
https://github.com/opensearch-project/geospatial/issues/136
https://github.com/opensearch-project/dashboards-maps/issues/39

Legend

| Symbol | Meaning | | -------- | ---------- | | :green_circle: | On track with overall release | | :yellow_circle: | Missed last milestone | | :red_circle: | Missed multiple milestones |

[gaiksaya]

Release testing

We have new Artifacts built and ready for testing.

Release Candidates:

OpenSearch - Build 6039 OpenSearch Dashboards - Build 4104

• Use Docker-Compose to setup a cluster
  • Download this gist as docker-compose.yml on your machine
  • Get latest image versions docker-compose pull
  • Start the cluster docker-compose up
• OpenSearch docker 2.3.0.6039
  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.3.0.6039 && docker run -it -p 9200:9200 -e “discovery.type=single-node” -e “DISABLE_SECURITY_PLUGIN=true” opensearchstaging/opensearch:2.3.0.6039
  • Connect command curl https://localhost:9200/
  • Start with security
  • Docker command docker pull opensearchstaging/opensearch:2.3.0.6039 && docker run -it -p 9200:9200 -e “discovery.type=single-node” opensearchstaging/opensearch:2.3.0.6039
  • Connect command curl --insecure https://admin:admin@localhost:9200/
• OpenSearch-Dashboards docker 2.3.0.4104
  • Start without security
  • Docker command docker pull opensearchstaging/opensearch-dashboards:2.3.0.4104 && docker run -it --network=“host” -e “DISABLE_SECURITY_DASHBOARDS_PLUGIN=true” opensearchstaging/opensearch-dashboards:2.3.0.4104
  • URL https://localhost:5601/
  • Start without security
  • Docker command docker pull opensearchstaging/opensearch-dashboards:2.3.0.4104 && docker run -it --network=“host” opensearchstaging/opensearch-dashboards:2.3.0.4104
  • URL https://localhost:5601/
• Use TARs to deploy OpenSearch Manually
  • OpenSearch - Build 6039
  • arm64 [manifest] [tar] [rpm]
  • x64 [manifest] [tar] [rpm]
  • OpenSearch Dashboards - Build 4104
  • arm64 [manifest] [tar][rpm]
  • x64 [manifest] [tar][rpm]

How to install via yum, tarball, etc?

[navneet1v]

While running the command:

docker pull opensearchstaging/opensearch:2.3.0.6039 && docker run -it -p 9200:9200 -e “discovery.type=single-node” -e “DISABLE_SECURITY_PLUGIN=true” opensearchstaging/opensearch:2.3.0.6039

I am getting this error:

breaking changes documentation for the next major version.
[2022-09-09T03:57:48,479][INFO ][o.o.e.NodeEnvironment    ] [60c67cb0bad0] using [1] data paths, mounts [[/ (overlay)]], net usable_space [44.6gb], net total_space [58.4gb], types [overlay]
[2022-09-09T03:57:48,480][INFO ][o.o.e.NodeEnvironment    ] [60c67cb0bad0] heap size [1gb], compressed ordinary object pointers [true]
[2022-09-09T03:57:48,537][INFO ][o.o.n.Node               ] [60c67cb0bad0] node name [60c67cb0bad0], node ID [9wNYm7cdTYapnl3UaHVThQ], cluster name [docker-cluster], roles [ingest, remote_cluster_client, data, cluster_manager]
[2022-09-09T03:57:52,038][WARN ][o.o.s.c.Salt             ] [60c67cb0bad0] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes
[2022-09-09T03:57:52,096][INFO ][o.o.s.a.i.AuditLogImpl   ] [60c67cb0bad0] Message routing enabled: true
[2022-09-09T03:57:52,151][INFO ][o.o.s.f.SecurityFilter   ] [60c67cb0bad0] <NONE> indices are made immutable.
[2022-09-09T03:57:52,517][INFO ][o.o.a.b.ADCircuitBreakerService] [60c67cb0bad0] Registered memory breaker.
[2022-09-09T03:57:52,846][INFO ][o.o.m.c.b.MLCircuitBreakerService] [60c67cb0bad0] Registered ML memory breaker.
[2022-09-09T03:57:53,576][INFO ][o.o.t.NettyAllocator     ] [60c67cb0bad0] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={opensearch.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}]
[2022-09-09T03:57:53,675][INFO ][o.o.d.DiscoveryModule    ] [60c67cb0bad0] using discovery type [zen] and seed hosts providers [settings]
[2022-09-09T03:57:54,032][WARN ][o.o.g.DanglingIndicesState] [60c67cb0bad0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-09-09T03:57:54,667][INFO ][o.o.p.h.c.PerformanceAnalyzerConfigAction] [60c67cb0bad0] PerformanceAnalyzer Enabled: false
[2022-09-09T03:57:54,715][INFO ][o.o.n.Node               ] [60c67cb0bad0] initialized
[2022-09-09T03:57:54,715][INFO ][o.o.n.Node               ] [60c67cb0bad0] starting ...
[2022-09-09T03:57:54,853][INFO ][o.o.t.TransportService   ] [60c67cb0bad0] publish_address {240.10.0.2:9300}, bound_addresses {0.0.0.0:9300}
[2022-09-09T03:57:55,005][INFO ][o.o.b.BootstrapChecks    ] [60c67cb0bad0] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_cluster_manager_nodes / cluster.initial_master_nodes] must be configured
ERROR: OpenSearch did not exit normally - check the logs at /usr/share/opensearch/logs/docker-cluster.log
[2022-09-09T03:57:55,020][INFO ][o.o.n.Node               ] [60c67cb0bad0] stopping ...
[2022-09-09T03:57:55,024][INFO ][o.o.s.a.r.AuditMessageRouter] [60c67cb0bad0] Closing AuditMessageRouter
[2022-09-09T03:57:55,025][INFO ][o.o.s.a.s.SinkProvider   ] [60c67cb0bad0] Closing InternalOpenSearchSink
[2022-09-09T03:57:55,025][INFO ][o.o.s.a.s.SinkProvider   ] [60c67cb0bad0] Closing DebugSink
[2022-09-09T03:57:55,043][INFO ][o.o.n.Node               ] [60c67cb0bad0] stopped
[2022-09-09T03:57:55,043][INFO ][o.o.n.Node               ] [60c67cb0bad0] closing ...
[2022-09-09T03:57:55,050][INFO ][o.o.s.a.i.AuditLogImpl   ] [60c67cb0bad0] Closing AuditLogImpl
[2022-09-09T03:57:55,054][INFO ][o.o.n.Node               ] [60c67cb0bad0] closed
Killing performance analyzer process 103
OpenSearch exited with code 78
Performance analyzer exited with code 143

[navneet1v]

1 more thing while installing the discovery plugin on the cluster downloaded via tar for x86 I am getting this error.

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/discovery-ec2/2.3.0/discovery-ec2-2.3.0.zip

What should be the URL to install the discovery-plugin?

[bbarani]

1 more thing while installing the discovery plugin on the cluster downloaded via tar for x86 I am getting this error.

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/discovery-ec2/2.3.0/discovery-ec2-2.3.0.zip

What should be the URL to install the discovery-plugin?

We haven't staged the native plugins yet. We will stage it after we finalize the RC builds. @gaiksaya @peterzhuamazon

[gaiksaya]

1 more thing while installing the discovery plugin on the cluster downloaded via tar for x86 I am getting this error. Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/discovery-ec2/2.3.0/discovery-ec2-2.3.0.zip What should be the URL to install the discovery-plugin?

We haven't staged the native plugins yet. We will stage it after we finalize the RC builds. @gaiksaya @peterzhuamazon

Yes until then the plugins can be accessed via url example: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.3.0/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.3.0.zip

[bbarani]

@navneet1v You can install the native plugins using the plugin install command as below (Example)

 ./opensearch-plugin install https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip 
warning: ignoring JAVA_TOOL_OPTIONS=-Dlog4j2.formatMsgNoLookups=true
-> Installing https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip
-> Downloading https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed discovery-ec2 with folder name discovery-ec2

[navneet1v]

While running the command:

docker pull opensearchstaging/opensearch:2.3.0.6039 && docker run -it -p 9200:9200 -e “discovery.type=single-node” -e “DISABLE_SECURITY_PLUGIN=true” opensearchstaging/opensearch:2.3.0.6039

I am getting this error:

Block (32 lines)

breaking changes documentation for the next major version.
[2022-09-09T03:57:48,479][INFO ][o.o.e.NodeEnvironment    ] [60c67cb0bad0] using [1] data paths, mounts [[/ (overlay)]], net usable_space [44.6gb], net total_space [58.4gb], types [overlay]
[2022-09-09T03:57:48,480][INFO ][o.o.e.NodeEnvironment    ] [60c67cb0bad0] heap size [1gb], compressed ordinary object pointers [true]
[2022-09-09T03:57:48,537][INFO ][o.o.n.Node               ] [60c67cb0bad0] node name [60c67cb0bad0], node ID [9wNYm7cdTYapnl3UaHVThQ], cluster name [docker-cluster], roles [ingest, remote_cluster_client, data, cluster_manager]
[2022-09-09T03:57:52,038][WARN ][o.o.s.c.Salt             ] [60c67cb0bad0] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes
[2022-09-09T03:57:52,096][INFO ][o.o.s.a.i.AuditLogImpl   ] [60c67cb0bad0] Message routing enabled: true
[2022-09-09T03:57:52,151][INFO ][o.o.s.f.SecurityFilter   ] [60c67cb0bad0] <NONE> indices are made immutable.
[2022-09-09T03:57:52,517][INFO ][o.o.a.b.ADCircuitBreakerService] [60c67cb0bad0] Registered memory breaker.
[2022-09-09T03:57:52,846][INFO ][o.o.m.c.b.MLCircuitBreakerService] [60c67cb0bad0] Registered ML memory breaker.
[2022-09-09T03:57:53,576][INFO ][o.o.t.NettyAllocator     ] [60c67cb0bad0] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={opensearch.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}]
[2022-09-09T03:57:53,675][INFO ][o.o.d.DiscoveryModule    ] [60c67cb0bad0] using discovery type [zen] and seed hosts providers [settings]
[2022-09-09T03:57:54,032][WARN ][o.o.g.DanglingIndicesState] [60c67cb0bad0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-09-09T03:57:54,667][INFO ][o.o.p.h.c.PerformanceAnalyzerConfigAction] [60c67cb0bad0] PerformanceAnalyzer Enabled: false
[2022-09-09T03:57:54,715][INFO ][o.o.n.Node               ] [60c67cb0bad0] initialized
[2022-09-09T03:57:54,715][INFO ][o.o.n.Node               ] [60c67cb0bad0] starting ...
[2022-09-09T03:57:54,853][INFO ][o.o.t.TransportService   ] [60c67cb0bad0] publish_address {240.10.0.2:9300}, bound_addresses {0.0.0.0:9300}
[2022-09-09T03:57:55,005][INFO ][o.o.b.BootstrapChecks    ] [60c67cb0bad0] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_cluster_manager_nodes / cluster.initial_master_nodes] must be configured
ERROR: OpenSearch did not exit normally - check the logs at /usr/share/opensearch/logs/docker-cluster.log
[2022-09-09T03:57:55,020][INFO ][o.o.n.Node               ] [60c67cb0bad0] stopping ...
[2022-09-09T03:57:55,024][INFO ][o.o.s.a.r.AuditMessageRouter] [60c67cb0bad0] Closing AuditMessageRouter
[2022-09-09T03:57:55,025][INFO ][o.o.s.a.s.SinkProvider   ] [60c67cb0bad0] Closing InternalOpenSearchSink
[2022-09-09T03:57:55,025][INFO ][o.o.s.a.s.SinkProvider   ] [60c67cb0bad0] Closing DebugSink
[2022-09-09T03:57:55,043][INFO ][o.o.n.Node               ] [60c67cb0bad0] stopped
[2022-09-09T03:57:55,043][INFO ][o.o.n.Node               ] [60c67cb0bad0] closing ...
[2022-09-09T03:57:55,050][INFO ][o.o.s.a.i.AuditLogImpl   ] [60c67cb0bad0] Closing AuditLogImpl
[2022-09-09T03:57:55,054][INFO ][o.o.n.Node               ] [60c67cb0bad0] closed
Killing performance analyzer process 103
OpenSearch exited with code 78
Performance analyzer exited with code 143

This is fixed. The resolution was mainly around updating the command. Some how the " was not getting copied properly.

[navneet1v]

@navneet1v You can install the native plugins using the plugin install command as below (Example)

Block (18 lines)

 ./opensearch-plugin install https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip 
warning: ignoring JAVA_TOOL_OPTIONS=-Dlog4j2.formatMsgNoLookups=true
-> Installing https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip
-> Downloading https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed discovery-ec2 with folder name discovery-ec2

Yeah sure I will do that. Thanks a lot.

[opensearch-ci-bot]

OpenSearch CommitID(after 2022-09-06) & Release Notes info

Repo	Branch	CommitID	Commit Date	Release Notes
OpenSearch	2.3	d4a162b	2022-09-09	True
common-utils	2.3	b1009ed	2022-09-12	True
job-scheduler	2.3	99a9567	2022-09-09	True
security	2.3	0aedd61	2022-09-07	True
performance-analyzer	2.3	0ca646f	2022-09-13	False
cross-cluster-replication	2.3	78a6fb7	2022-09-07	True
anomaly-detection	2.3	e775dd3	2022-09-07	True
ml-commons	2.3	37d2d83	2022-09-12	True
k-NN	2.3	78e9383	2022-09-09	True
geospatial	2.3	f9ec0db	2022-09-09	True
opensearch-observability	2.3	add0e26	2022-09-09	True
dashboards-reports	2.3	34f6a09	2022-09-12	True
notifications-core	2.3	74e766d	2022-09-13	True
notifications	2.3	74e766d	2022-09-13	True
asynchronous-search	2.3	0c253c3	2022-09-13	True
sql	2.3	568f9a9	2022-09-13	True
alerting	2.3	34274a0	2022-09-09	True
index-management	2.3	7217b5b	2022-09-12	True

[opensearch-ci-bot]

OpenSearch Dashboards CommitID(after 2022-09-06) & Release Notes info

Repo	Branch	CommitID	Commit Date	Release Notes
OpenSearch-Dashboards	2.3	d8e4d4d	2022-09-13	True
functionalTestDashboards	2.3			False
securityDashboards	2.3	1d4ee91	2022-09-07	True
anomalyDetectionDashboards	2.3	bc39bb2	2022-09-13	True
observabilityDashboards	2.3	add0e26	2022-09-09	True
ganttChartDashboards	2.3	6acb32e	2022-09-09	True
reportsDashboards	2.3	34f6a09	2022-09-12	True
customImportMapDashboards	2.3	32e985a	2022-09-09	True
notificationsDashboards	2.3	74e766d	2022-09-13	True
queryWorkbenchDashboards	2.3	568f9a9	2022-09-13	True
indexManagementDashboards	2.3	0da978a	2022-09-12	True
alertingDashboards	2.3	a77eabd	2022-09-09	True

[navneet1v]

Added release notes for K-NN, geospatial and customImportMapDashboards

[gaiksaya]

Performance test runs for this version: x64: https://build.ci.opensearch.org/job/perf-test/541 arm64: https://build.ci.opensearch.org/job/perf-test/540

[dreamer-89]

Performance test runs for this version: x64: https://build.ci.opensearch.org/job/perf-test/541 arm64: https://build.ci.opensearch.org/job/perf-test/540

I compared 2.3.0 with 2.2.1 (baseline). I don’t see any anomaly in results (there are minor ups/downs in latency/throughput. I observe similar numbers comparing older releases 2.2.0 (534 build no) & 2.2.1 (568)). So, I think numbers are fine.

Side notes

Build numbers used 2.2.1 -> 568, 569, 2.3.0 -> 540, 541

X64

Without Security Minor indexing/search latency improvements, but minor degradation on throughput & search latency on p99/p100 With Security Same observation as above. But minor search latency uptick across all p50-100. But throughput improvements.

Arm64

With Security Minor indexing latency (~1.50%) degrade. ~.80% throughput degrade. Query latency improve & throughput increase ~1% Without Security Latency improve ~2%, throughput increase ~1%, minor Search query throughput decrease ~1%

[gaiksaya]

Hi @saratvemulapalli @kotwanikunal @dblock Can you take a look on the above performance test findings? Thanks! cc: @CEHENKLE @bbarani

[saratvemulapalli]

@gaiksaya @dreamer-89 trying to look at the runs, is there a human readable format for the results?

[kotwanikunal]

@saratvemulapalli

Results attached -

Version	Architecture	Security Enabled	Start Time(PDT)	Duration	Instance Type	Workload Details	Index Latency(ms) p50	Index Latency(ms) p90	Index Latency(ms) p99	Index Latency(ms) p100	Index Throughput(docs/s) p0	Index Throughput(docs/s) p50	Index Throughput(docs/s) p100	Index Throughput (docs/s) Ops Count	Index Operation Counts Op Error Count	Index Operation Counts Op Error Rate	Query Latency (ms) p50	Query Latency (ms) p90	Query Latency (ms) p99	Query Latency (ms) p100	Query Throughput (docs/s) p0	Query Throughput (docs/s) p50	Query Throughput (docs/s) p100	Query Operation Counts Ops Count	Query Operation Counts Op Error Count	Query Operation Counts Op Error Rate	CPU (%) p50	CPU (%) p90	CPU (%) p99	CPU (%) p100	Memory (%) p50	Memory (%) p90	Memory (%) p99	Memory (%) p100	GC (ms) Old	GC (ms) Young
2.2.1	arm64	with_security	9/11/22, 3:01 AM	6h 35m 33s	m6g.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	1851.8598505373673	2456.7932834712446	4549.877352294918	6659.5146484375	39204.41796875	40484.896675314696	43940.48046875	46459	0	0	360.88779627482097	373.73701316833495	384.26831719970704	390.29581985473635	1.670055627822876	1.7078201830387116	1.7455639243125916	1510	0	0	96	97	97	97	49.2	64	76.96000000000004	91	0	985508
2.2.1	arm64	without_security	9/12/22, 3:01 AM	6h 10m 30s	m6g.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	1646.453753	2210.699582	4342.278781	6122.276367	44029.94922	45598.62776	49929.39063	46013	0	0	362.3607367	374.8823477	378.7677886	391.089801	1.704830134	1.731174356	1.754105937	1510	0	0	96	97	97	97	48	62	83.8	92	0	829901
2.2.1	x64	with_security	9/11/22, 2:58 AM	8h 10m 40s	m5.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	2288.2764957074537	3038.3064399577747	5879.324521689244	9268.974609375	32105.65234375	33927.90692669141	38235.58203125	46894	0	0	286.41736221313477	293.68860984802245	301.12121810913084	318.229997253418	1.7262567162513733	1.740007507801056	1.75261892080307	1510	0	0	97	97	97	97	52	66	87.76999999999992	93	0	1231253
2.2.1	x64	without_security	9/11/22, 2:58 AM	7h 49m 37s	m5.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	2192.6901234475677	2914.758178724321	5700.923922209283	11106.1728515625	33684.34375	35614.496027572684	39991.8125	46757	0	0	295.2953796386719	302.1095615386963	309.9044530029297	322.45588912963865	1.7269416570663452	1.7409352322419487	1.7553181290626525	1510	0	0	97	97	98	98	49	63	84	94	0	1054789
2.3.0	arm64	with_security	9/9/22, 10:41 AM	6h 42m 35s	m6g.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	1884.939796	2498.486831	4672.044931	8184.402832	38684.21484	40013.98915	43568.23438	46488	0	0	351.4363827	359.4697859	368.6659151	437.3339294	1.699884045	1.722742015	1.75187211	1510	0	0	96	97	97	97	52	65	88.98	94	0	979894
2.3.0	arm64	without_security	9/9/22, 10:43 AM	6h 12m 31s	m6g.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	1625.392231	2180.028306	4150.245827	6011.796387	44516.28516	46462.11049	51156.74219	45950	0	0	364.6436938	373.184549	379.6433586	386.4868347	1.685038006	1.709957743	1.753778672	1510	0	0	96	97	97	97	48	64	83.16	91	0	853041
2.3.0	x64	with_security	9/9/22, 10:41 AM	8h 5m 38s	m5.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	2274.482999	3001.790512	5625.219549	9248.959961	32337.31836	33916.93773	37967.18359	46902	0	0	290.8293233	300.3195797	309.3330738	314.5757118	1.725921583	1.739219183	1.752308607	1510	0	0	97	97	98	98	51	65	81.66	87	0	1238934
2.3.0	x64	without_security	9/9/22, 10:41 AM	7h 41m 37s	m5.xlarge	nyc_taxis / 2 warmupIterations / 3 testIterations	2174.641495	2896.447136	5494.624087	9741.860352	33844.23438	35826.79863	40342.28125	46705	0	0	287.6040042	297.3744589	310.0044941	328.9224983	1.731490076	1.743171066	1.757029057	1510	0	0	97	98	98	98	49	64	86.56	96	0	1033209

Difference (in %) -

Version	Architecture	Security Enabled	Index Latency(ms) p50	Index Latency(ms) p90	Index Latency(ms) p99	Index Latency(ms) p100	Index Throughput(docs/s) p0	Index Throughput(docs/s) p50	Index Throughput(docs/s) p100	Index Throughput (docs/s) Ops Count	Index Operation Counts Op Error Count	Index Operation Counts Op Error Rate	Query Latency (ms) p50	Query Latency (ms) p90	Query Latency (ms) p99	Query Latency (ms) p100	Query Throughput (docs/s) p0	Query Throughput (docs/s) p50	Query Throughput (docs/s) p100	Query Operation Counts Ops Count	Query Operation Counts Op Error Count	Query Operation Counts Op Error Rate	CPU (%) p50	CPU (%) p90	CPU (%) p99	CPU (%) p100	Memory (%) p50	Memory (%) p90	Memory (%) p99	Memory (%) p100	GC (ms) Old	GC (ms) Young
2.3 vs 2.2.1	arm64	with_security	+1.79%	+1.7%	+2.69%	+22.9%	-1.33%	-1.16%	-0.85%	+0.06%			-2.62%	-3.82%	-4.06%	+12.05%	+1.79%	+0.87%	+0.36%	0%			0%	0%	0%	0%	+5.69%	+1.56%	+15.62%	+3.3%		-0.57%
2.3 vs 2.2.1	arm64	without_security	-1.28%	-1.39%	-4.42%	-1.8%	+1.1%	+1.89%	+2.46%	-0.14%			+0.63%	-0.45%	+0.23%	-1.18%	-1.16%	-1.23%	-0.02%	0%			0%	0%	0%	0%	0%	+3.23%	-0.76%	-1.09%		+2.79%
2.3 vs 2.2.1	x64	with_security	-0.60%	-1.20%	-4.32%	-0.22%	0.72%	-0.03%	-0.70%	0.02%			1.54%	2.26%	2.73%	-1.15%	-0.02%	-0.05%	-0.02%	0%			0%	0%	1.03%	1.03%	-1.92%	-1.52%	-6.96%	-6.45%		0.62%
2.3 vs 2.2.1	x64	without_security	-0.82%	-0.63%	-3.62%	-12.28%	0.47%	0.60%	0.88%	-0.11%			-2.60%	-1.57%	0.03%	2.01%	0.26%	0.13%	0.10%	0%			0%	1.03%	0%	0%	0%	1.59%	3.05%	2.13%		-2.05%

Thanks to Brandon! 🙂

Edit: Results are not anomalous, LGTM!

Edit: Updating the results. Sorry for the bad pasting skills. 🤦

[saratvemulapalli]

Thanks @kotwanikunal, looks like our query latency has improved for arm64 without security by quite a bit (>20% lower latency). Do you guys know if you made any expected changes?

[kotwanikunal]

Thanks @kotwanikunal, looks like our query latency has improved for arm64 without security by quite a bit (>20% lower latency). Do you guys know if you made any expected changes?

@saratvemulapalli My bad. I swapped the comparisons while pasting. Please have a look again.

[Zhangxunmt]

Added release notes for ml-commons.

[gaiksaya]

Docker scan results: OpenSearch: https://build.ci.opensearch.org/job/docker-scan/97/artifact/scan_docker_image.txt OpenSearch-Dashboards: https://build.ci.opensearch.org/job/docker-scan/95/artifact/scan_docker_image.txt

[saratvemulapalli]

Thanks @kotwanikunal. Looks like the results are very similar. We are good to go!

[navneet1v]

1 more thing while installing the discovery plugin on the cluster downloaded via tar for x86 I am getting this error. Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/discovery-ec2/2.3.0/discovery-ec2-2.3.0.zip What should be the URL to install the discovery-plugin?

We haven't staged the native plugins yet. We will stage it after we finalize the RC builds. @gaiksaya @peterzhuamazon

Yes until then the plugins can be accessed via url example: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.3.0/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.3.0.zip

Hi, Have we staged the native plugins? I am trying to do the custom performance test for the K-NN plugin.

[navneet1v]

Getting exception while installing the OpenSearch via tar.

warning: ignoring JAVA_TOOL_OPTIONS=-Dlog4j2.formatMsgNoLookups=true
[2022-09-13T01:16:38,994][INFO ][o.o.n.Node               ] [88665a4d7d84.ant.amazon.com] version[2.3.0], pid[83775], build[tar/6f6e84ebc54af31a976f53af36a5c69d474a5140/2022-09-09T00:07:12.137133581Z], OS[Mac OS X/12.5.1/x86_64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/11.0.16/11.0.16+8]
[2022-09-13T01:16:38,997][INFO ][o.o.n.Node               ] [88665a4d7d84.ant.amazon.com] JVM home [/Users/navneev/.sdkman/candidates/java/11.0.16-tem], using bundled JDK [false]
[2022-09-13T01:16:38,998][INFO ][o.o.n.Node               ] [88665a4d7d84.ant.amazon.com] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/var/folders/rb/dv5bnd5n65vchm4c59_qbvpj1dqcnf/T/opensearch-10526221033923683325, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/Users/navneev/Downloads/opensearch-2.3.0, -Dopensearch.path.conf=/Users/navneev/Downloads/opensearch-2.3.0/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2022-09-13T01:16:40,004][WARN ][stderr                   ] [88665a4d7d84.ant.amazon.com] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[2022-09-13T01:16:40,005][WARN ][stderr                   ] [88665a4d7d84.ant.amazon.com] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2022-09-13T01:16:40,005][WARN ][stderr                   ] [88665a4d7d84.ant.amazon.com] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2022-09-13T01:16:40,017][INFO ][o.o.s.s.t.SSLConfig      ] [88665a4d7d84.ant.amazon.com] SSL dual mode is disabled
[2022-09-13T01:16:40,017][INFO ][o.o.s.OpenSearchSecurityPlugin] [88665a4d7d84.ant.amazon.com] OpenSearch Config path is /Users/navneev/Downloads/opensearch-2.3.0/config
[2022-09-13T01:16:40,273][INFO ][o.o.s.s.DefaultSecurityKeyStore] [88665a4d7d84.ant.amazon.com] JVM supports TLSv1.3
[2022-09-13T01:16:40,275][INFO ][o.o.s.s.DefaultSecurityKeyStore] [88665a4d7d84.ant.amazon.com] Config directory is /Users/navneev/Downloads/opensearch-2.3.0/config/, from there the key- and truststore files are resolved relatively
[2022-09-13T01:16:40,344][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [88665a4d7d84.ant.amazon.com] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:184) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-2.3.0.jar:2.3.0]
    at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103) ~[opensearch-2.3.0.jar:2.3.0]
Caused by: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:790) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.node.Node.<init>(Node.java:420) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.node.Node.<init>(Node.java:347) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.3.0.jar:2.3.0]
    ... 6 more
Caused by: java.lang.reflect.InvocationTargetException
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.node.Node.<init>(Node.java:420) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.node.Node.<init>(Node.java:347) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.3.0.jar:2.3.0]
    ... 6 more
Caused by: org.opensearch.OpenSearchException: plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:419) ~[?:?]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:255) ~[?:?]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:176) ~[?:?]
    at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218) ~[?:?]
    at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:262) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.node.Node.<init>(Node.java:420) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.node.Node.<init>(Node.java:347) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.3.0.jar:2.3.0]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.3.0.jar:2.3.0]
    ... 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
Likely root cause: OpenSearchException[plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:419)
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:255)
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:176)
    at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218)
    at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:262)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781)
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730)
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532)
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195)
    at org.opensearch.node.Node.<init>(Node.java:420)
    at org.opensearch.node.Node.<init>(Node.java:347)
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
    at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
    at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
    at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
    at org.opensearch.cli.Command.main(Command.java:101)
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
For complete error details, refer to the log at /Users/navneev/Downloads/opensearch-2.3.0/logs/opensearch.log

[peternied]

@navneet1v Please create an issue with repro steps for the above issue so we can get someone assigned to look into it

[navneet1v]

@navneet1v Please create an issue with repro steps for the above issue so we can get someone assigned to look into it

Sure will do that.

[gaiksaya]

1 more thing while installing the discovery plugin on the cluster downloaded via tar for x86 I am getting this error. Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/discovery-ec2/2.3.0/discovery-ec2-2.3.0.zip What should be the URL to install the discovery-plugin?

We haven't staged the native plugins yet. We will stage it after we finalize the RC builds. @gaiksaya @peterzhuamazon

Yes until then the plugins can be accessed via url example: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.3.0/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.3.0.zip

Hi, Have we staged the native plugins? I am trying to do the custom performance test for the K-NN plugin.

No we have not. We will to do so today. Artifacts are staged after the teams finish the sanity/smoke testing which was yesterday EOD. Thanks!

[gaiksaya]

$ sudo ./opensearch-plugin install discovery-ec2
-> Installing discovery-ec2
-> Downloading discovery-ec2 from opensearch
[=================================================] 100%
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed discovery-ec2 with folder name discovery-ec2

[gaiksaya]

Maven is out https://aws.oss.sonatype.org/content/repositories/releases/org/opensearch/opensearch-core/2.3.0/

[gaiksaya]

Retrospective issue to track any missed steps, improvements: https://github.com/opensearch-project/opensearch-build/issues/2620

[gaiksaya]

OpenSearch 2.3.0 has been released. :confetti_ball:

[gaiksaya]

PRs to increment 2.3.0 to 2.3.1 are up for all components. Please review and merge. Thanks!