opensearch-project / opensearch-build

🧰 OpenSearch / OpenSearch-Dashboards Build Systems
Apache License 2.0
140 stars 273 forks source link

[RELEASE] Release version 2.11.0 #3998

Closed github-actions[bot] closed 1 year ago

github-actions[bot] commented 1 year ago

Release OpenSearch and OpenSearch Dashboards 2.11.0

I noticed that a manifest was automatically created in manifests/2.11.0. Please follow the following checklist to make a release.

How to use this issue

## This Release Issue This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the [Release Process OpenSearch Guide](https://github.com/opensearch-project/opensearch-build/blob/main/RELEASE_PROCESS_OPENSEARCH.md).

Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.

OpenSearch 2.11.0 exit criteria status:

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

OpenSearch | Status | Description | Comments -- | -- | -- | -- Performance tests are run, results are posted to the release ticket and there no unexpected regressions | :green_circle: | The perf test for 2.11.0 RC candidate would be automatically kicked off by this [workflow](https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/benchmark-test.jenkinsfile) | Performance benchmark runs for this RC will be automatically initiated soon. Documentation has been fully reviewed and signed off by the documentation community. | :green_circle: | All integration tests are passing | :green_circle:| Release blog is ready | :green_circle: |Release blog has been finalized |

OpenSearch dashboards 2.11.0 exit criteria status:

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

OpenSearch Dashboards | Status | Description | Comments -- | -- | -- | -- ~~Performance tests are run, results are posted to the release ticket and there no unexpected regressions~~ | :green_circle: | ~~Performance tests are not executed for OpenSearch Dashboards~~  |  ~~Performance tests are not executed for OpenSearch Dashboards~~ Documentation has been fully reviewed and signed off by the documentation community. | :green_circle: | All integration tests are passing | :green_circle:| Release blog is ready | :green_circle: |Release blog has been finalized |

Preparation

Campaigns

Release Branch and Version Increment - Ends Oct 04, 2023 - Owner: [Repository Owner, component release manager]

Code Complete - Ends Oct 04, 2023 5PM PST- Owner: [Repository Owner, component release manager, build release manager]

Release Candidate Creation and Testing - Starts Oct 05, 2023 - Owner: [Build release manager]

Performance testing validation - Starts Oct 05, 2023 - Owner: [Repository Owner, component release manager]

Pre Release - TBD - Owner: [Build release manager]

Release - TBD - Owner: [Build release manager]

Release Checklist.


Release Checklist

### Pre-Release activities - [x] Promote Repos. - - [x] OS - - [x] OSD - [x] Promote Artifacts. - - [x] Windows - - [x] Linux Debian - - [x] Linux RPM - - [x] Linux TAR - [x] Consolidated Release Notes. ### Release activities - [ ] Docker Promotion. - [ ] Release Validation part 1. - - [ ] OpenSearch and OpenSearch Dashboard Validation. - - [ ] Validate the native plugin installation. - [ ] Merge consolidated release notes PR. - [ ] Website and Documentation Changes. - - [ ] Merge staging website PR. - - [ ] Promote the website changes to prod. - - [ ] Add website alert. - [ ] Release Validation part 2. - - [ ] Validate the artifact download URL's and signatures. - [ ] Release Validation part 3. - - [ ] Trigger the validation build (Search for `Completed validation for <>` in the logs). - [ ] Maven Promotion. - [ ] Publish blog posts. - [ ] Advertise on Social Media. - [ ] Post on public slack and Github Release issue. ### Post-Release activities - [ ] Release Tags. - [ ] Input Manifest Update. - [ ] Decrease the Build Frequency. - [ ] OpenSearch Build Release notes. - [ ] Retrospective Issue. - [ ] Helm and Ansible Playbook release. - [ ] Upcoming Release Preparation.


Post Release

Components

OpenSearch

Component Issues tagged with 2.11.0
https://github.com/opensearch-project/OpenSearch/issues/8754
https://github.com/opensearch-project/common-utils/issues/496
https://github.com/opensearch-project/k-NN/issues/995
https://github.com/opensearch-project/geospatial/issues/357
https://github.com/opensearch-project/security/issues/3028
https://github.com/opensearch-project/cross-cluster-replication/issues/1080
https://github.com/opensearch-project/ml-commons/issues/1141
https://github.com/opensearch-project/neural-search/issues/224
https://github.com/opensearch-project/notifications/issues/711
https://github.com/opensearch-project/observability/issues/1576
https://github.com/opensearch-project/reporting/issues/729
https://github.com/opensearch-project/sql/issues/1882
https://github.com/opensearch-project/asynchronous-search/issues/316
https://github.com/opensearch-project/anomaly-detection/issues/964
https://github.com/opensearch-project/alerting/issues/1047
https://github.com/opensearch-project/security-analytics/issues/493
https://github.com/opensearch-project/index-management/issues/860
https://github.com/opensearch-project/performance-analyzer/issues/508

OpenSearch Dashboards

Component Issues tagged with 2.11.0
https://github.com/opensearch-project/OpenSearch-Dashboards/issues/4584
https://github.com/opensearch-project/opensearch-dashboards-functional-test/issues/766
https://github.com/opensearch-project/dashboards-observability/issues/703
https://github.com/opensearch-project/dashboards-reporting/issues/149
https://github.com/opensearch-project/dashboards-visualizations/issues/212
https://github.com/opensearch-project/dashboards-query-workbench/issues/96
https://github.com/opensearch-project/dashboards-maps/issues/443
https://github.com/opensearch-project/anomaly-detection-dashboards-plugin/issues/558
https://github.com/opensearch-project/ml-commons-dashboards/issues/235
https://github.com/opensearch-project/index-management-dashboards-plugin/issues/819
https://github.com/opensearch-project/dashboards-notifications/issues/86
https://github.com/opensearch-project/alerting-dashboards-plugin/issues/655
https://github.com/opensearch-project/security-analytics-dashboards-plugin/issues/661
https://github.com/opensearch-project/security-dashboards-plugin/issues/1519
https://github.com/opensearch-project/dashboards-search-relevance/issues/253
Legend

| Symbol | Meaning | | -------- | ---------- | | :green_circle: | On track with overall release | | :yellow_circle: | Missed last milestone | | :red_circle: | Missed multiple milestones |

peterzhuamazon commented 1 year ago

Pending Issues:

I can discuss both of these issues on the standup tomorrow.

Thanks.

PRs:

Placeholder.

rishabh6788 commented 1 year ago

OpenSearch 8603 / OpenSearch Dashboards 6646 is ready for your test.

OpenSearch - Build 8603 OpenSearch Dashboards - Build 6646

Check how to install opensearch and dashboards on different platforms

bbarani commented 1 year ago

Pending PR's scheduled for 2.11.0 release

dhrubo-os commented 1 year ago

For opensearch dashboard, both are marked as Start without security. Could you please check?

rishabh6788 commented 1 year ago

For opensearch dashboard, both are marked as Start without security. Could you please check?

@dhrubo-os Thanks for catching that, updated.

opensearch-ci-bot commented 1 year ago

OpenSearch CommitID(after 2023-09-25) & Release Notes info

Repo Branch CommitID Commit Date Release Notes
OpenSearch [38df519c9072fd1ffb2352a09b626a066df0a1dd] 38df519 2023-10-09 False
common-utils [aaa4c2a061601105c62cbaf9838a0a4bd9986aa3] False
job-scheduler [0a2fef228bb687995d2e3ddefa88ecf1e191ffd7] 0a2fef2 2023-10-02 True
k-NN [f807aa2997f02a7b92421be9266c0dabb41ea963] f807aa2 2023-10-06 True
geospatial [2a7f1b01589d893c0b1f153c4edc68295f85d0e5] 2a7f1b0 2023-10-06 True
security [b3f1a2925cd685a5be81344969a811b5d5a881e0] b3f1a29 2023-10-09 False
cross-cluster-replication [5c3a3c2fab950abc368a2ab69f483f825c2284d2] 5c3a3c2 2023-10-05 False
ml-commons [da5d82942385c34544016cf361b517c2bb3d36c4] da5d829 2023-10-10 False
neural-search [73cab3b8b76704bf49e43cad83b514a6b1e3be64] 73cab3b 2023-10-06 True
notifications-core [968c3d54603ac99d77eb07d8254df4edc572bf1b] 968c3d5 2023-10-07 True
notifications [968c3d54603ac99d77eb07d8254df4edc572bf1b] 968c3d5 2023-10-07 True
opensearch-observability [bd11e813ebd5ac002c3d34ed2c965325d5874b77] False
opensearch-reports [78c8c89f318ebee78a5f9ea398cafd9aea33e3fb] False
sql [bb5a025732226c044ecd5a163abf75ed2b841fcf] bb5a025 2023-10-09 False
asynchronous-search [e291c1cc8e84d1db727ecf3faf04ef30ebcf0d17] e291c1c 2023-10-04 True
anomaly-detection [35d476461feba246a2eea705e75d99553b495d0c] 35d4764 2023-10-05 True
alerting [e9c5da00fd63fa4b38207909ca6d2a97284a7a26] e9c5da0 2023-10-10 False
security-analytics [848262e64ed8f7ea64dd4ad628284d88d17f0e77] 848262e 2023-10-05 False
index-management [905868eac14ae47cd8a0c27a9bbb6d0b59ef1c9b] 905868e 2023-10-04 False
performance-analyzer [d907f1986f76a390dbe6a856f41d61203cadb6e1] d907f19 2023-10-05 True
custom-codecs [486ed652b7de41c34d81d8d663ca02302fed78ae] 486ed65 2023-10-04 False
opensearch-ci-bot commented 1 year ago

OpenSearch Dashboards CommitID(after 2023-09-25) & Release Notes info

Repo Branch CommitID Commit Date Release Notes
OpenSearch-Dashboards [3c838160fe6b453881ba326c6c8f57e11cf10fdd] 3c83816 2023-10-10 True
functionalTestDashboards [b0877cfc1a55a5b024493732b2892e22a221ff94] b0877cf 2023-10-10 False
observabilityDashboards [3f169c42d71d2dcf5775d7629c9cbbc94d36ed9f] 3f169c4 2023-10-09 False
reportsDashboards [854efb3fe8c1dcae2648c03451d199fb6275ba2d] 854efb3 2023-10-10 False
ganttChartDashboards [93092dbf97820c257d9c51a2856321e83a0a4eee] False
queryWorkbenchDashboards [aeb4974e45acf4b00ab6ffa66f28e5ff1c3d51b0] aeb4974 2023-10-05 False
customImportMapDashboards [ff5fb8027f0263ea04cf991f75febcdce88ce80b] ff5fb80 2023-10-06 True
anomalyDetectionDashboards [5f32b78294adb2a5704dc33d9267d580430a1952] 5f32b78 2023-09-26 False
mlCommonsDashboards [fdc9f446301987eda68cb821b099268566ab6a5a] fdc9f44 2023-10-05 False
indexManagementDashboards [58d9170136ab59708e2d72b089080d24cd856171] False
notificationsDashboards [9f2f96b5755b92037842d208681cab09900a7c2a] 9f2f96b 2023-10-07 True
alertingDashboards [7c573cba222c48711cd0fd170ef5e08486d4a131] 7c573cb 2023-10-02 False
securityAnalyticsDashboards [4b03bee10ee9fc88c2228a4d75e573b643c7c58a] 4b03bee 2023-10-05 False
securityDashboards [3bb8afea634a364c78ed3431cad20e5263688fdb] 3bb8afe 2023-10-06 False
searchRelevanceDashboards [a869ba08361b0ce089e7bff3cd6ea09b2c558787] a869ba0 2023-10-04 False
dhrubo-os commented 1 year ago

I'm trying to spin up the cluster, but I can't login to the dashboard:

Screenshot 2023-10-06 at 12 08 11 PM

I tried to clear my session, but still same.

bbarani commented 1 year ago

I am seeing the same error as @dhrubo-os . This seems to be related to observability dashboards plugin. @anirudha Can you look in to this issue?

hdhalter commented 1 year ago

Documentation status: RED. Out of 16 issues we are tracking, 5 are Done, 6 are in some state of progress, and 5 are in our backlog. Most issues in our backlog (meaning we have not started them) came in on or after 10/2. We have moved 7 issues to 2.12, and for another one, we are waiting for confirmation as to whether it's in 2.11 or 2.12. For a list of issues we are tracking, see https://github.com/opensearch-project/documentation-website/issues?q=is%3Aissue+label%3Av2.11.0+is%3Aopen.

ps48 commented 1 year ago

@bbarani @dhrubo-os @rishabh6788, Observability has hard dependency on a dashboards feature. Dashboards-Core backport to 2.11 for this commit went in after the initial branch cut. Should be fixed in next RC.

rishabh6788 commented 1 year ago

@ps48 Thank you for the update. We will generating a new RC candidate at 4pm PST and share the details.

rishabh6788 commented 1 year ago

OpenSearch 8607 / OpenSearch Dashboards 6649 is ready for your test.

OpenSearch - Build 8607 OpenSearch Dashboards - Build 6649

Check how to install opensearch and dashboards on different platforms

rishabh6788 commented 1 year ago

OpenSearch 8618 / OpenSearch Dashboards 6651 is ready for your test.

OpenSearch - Build 8618 OpenSearch Dashboards - Build 6651

Check how to install opensearch and dashboards on different platforms

rishabh6788 commented 1 year ago

Integration Test status with 3rd RC.

All components passing for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6317/pipeline/104

All Components failing for OpenSearch-Dashboards with Dashboards facing Authentication Exception. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4253/pipeline/115.

Attached sample dashboards-log file. opensearch_dashboards.log @bbarani @CEHENKLE @seanneumann

rishabh6788 commented 1 year ago

RC#4 OpenSearch 8624 / OpenSearch Dashboards 6654 is ready for your test.

OpenSearch - Build 8624 OpenSearch Dashboards - Build 6654

Check how to install opensearch and dashboards on different platforms

rishabh6788 commented 1 year ago

Integration Test status with 4th RC.

All components passing for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6325/pipeline/

Failing Components for OpenSearch-Dashboards:

rishabh6788 commented 1 year ago

Release Notes Readiness

Below components don't have Release Notes created yet for 2.11.0

OpenSearch:

OpenSearch Dashboards

rishabh6788 commented 1 year ago

RPM Integration Tests for OpenSearch

Failing components:

DEB Integration Tests for OpenSearch

hdhalter commented 1 year ago

Documentation status: 6 Done, 8 in various stages of doc/tech review, 3 in backlog or in progress.

rishabh6788 commented 1 year ago

RC #5 OpenSearch 8634 / OpenSearch Dashboards 6658 is ready for your test.

OpenSearch - Build 8634 OpenSearch Dashboards - Build 6658

Check how to install opensearch and dashboards on different platforms

rishabh6788 commented 1 year ago

2.11.0 Performance comparison with 2.10.0

There is no performance change between 2.11.0 and 2.10.0 on overall indexing and query metrics.

http_logs (x64)

http_logs_x64_compar_2023-10-11T20_02_23

http_logs (arm64)

http_logs_arm64_comp_2023-10-11T20_02_29

nyc_taxis (x64)

nyc_taxis_x64_compar_2023-10-11T20_02_03

nyc_taxis (arm64)

nyc_taxis_arm64_comp_2023-10-11T20_02_07

rishabh6788 commented 1 year ago

RC #6 OpenSearch 8641 / OpenSearch Dashboards 6660 is ready for your test.

OpenSearch - Build 8641 OpenSearch Dashboards - Build 6660

Check how to install opensearch and dashboards on different platforms

rishabh6788 commented 1 year ago

Integration Test Update

All components integration tests are passing for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6345/pipeline

alerting failed due to jenkins error and opensearch-reports failed due to flaky test. Re-ran tests for them and it passed. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6346/pipeline/67/.

For OpenSearch Dashboards below components are failing.

Attached are the log files for teams to take a look and take corrective action. observabilityDashboards-without-security.txt observabilityDashboards-with-security.txt reportsDashboards-without-security.txt

rishabh6788 commented 1 year ago

Final RC OpenSearch 8649 / OpenSearch Dashboards 6665 is ready for your test.

OpenSearch - Build 8649 OpenSearch Dashboards - Build 6665

Check how to install opensearch and dashboards on different platforms

rishabh6788 commented 1 year ago

Integration Test Results

All components PASSING for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6364/pipeline/104

All components PASSING for OpenSearch Dashboards. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4283/pipeline/111.

OpenSearch Dashboards had a flaky test, passed in second run for OpenSearch Dashboards. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4284/pipeline/99/

bbarani commented 1 year ago

We are :green_circle: for 2.11.0 release and it is scheduled to be released on Monday (Oct 16 2023).

You can participate in release meeting using this link - https://opensearch.org/events/2023-1018-2-11-release-meetings/

rishabh6788 commented 1 year ago

Release checklist

Pre-Release activities

rishabh6788 commented 1 year ago

Created tags for OS and OSD.

Updated 2.11.0 OS and OSD manifests https://github.com/opensearch-project/opensearch-build/pull/4133

rishabh6788 commented 1 year ago

Maven Release done https://build.ci.opensearch.org/blue/organizations/jenkins/publish-to-maven/detail/publish-to-maven/25/pipeline.

rishabh6788 commented 1 year ago

We have successfully released 2.11.0.
Please see https://github.com/opensearch-project/opensearch-build/issues/4061 retrospective issue to add your thoughts and feedback/improvements on this release.