Closed github-actions[bot] closed 7 months ago
DarshitChanpura
commented
7 months ago @prudhvigodithi For docker, without security, you can modify the template to skip -e DISABLE_INSTALL_DEMO_CONFIG=true as it is disabled by default with this new rc when security is disabled.
docker pull opensearchstaging/opensearch:2.12.0.9408 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.12.0.9408
peterzhuamazon
commented
7 months ago Awaiting merge @prudhvigodithi :
ylwu-amzn
commented
7 months ago we will merge this PR https://github.com/opensearch-project/ml-commons/pull/2094 to 2.12
rishabh6788
commented
7 months ago While there is no performance impact on indexing throughput, we are seeing massive gains in overall and certain aggregate query metrics.
hourly_agg query latency.autohisto_agg and date_histogram_agg queries.
prudhvigodithi
commented
7 months ago Most recent OpenSearch build: 🟢 Most recent OpenSearch Dashboards build: 🔴
Plugin teams please go through the issues and try to close them as soon as possible https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.12.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle+
https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/7869/pipeline/ anomaly-detection, CCR failed.
https://github.com/issues?q=is%3Aopen+is%3Apr+user%3Aopensearch-project+label%3Av2.12.0+
https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.12.0+
r1walz
commented
7 months ago @prudhvigodithi few critical changes from ISM and ISM-DP were recently merged to v2.12:
ISM:
ISM-DP:
I've updated release notes as well:
prudhvigodithi
commented
7 months ago OpenSearch - Build 9419 OpenSearch Dashboards - Build 7295
Use the following Docker-Compose to setup a cluster
version: '3'
services:
opensearch-node1:
image: opensearchstaging/opensearch:2.12.0.9419
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchstaging/opensearch:2.12.0.9419
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchstaging/opensearch-dashboards:2.12.0.7295
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
docker-compose pull.docker-compose up.docker pull opensearchstaging/opensearch:2.12.0.9419 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e DISABLE_INSTALL_DEMO_CONFIG=true -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.12.0.9419curl https://localhost:9200/docker pull opensearchstaging/opensearch:2.12.0.9419 && docker run -it -p 9200:9200 -e 'OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!' -e "discovery.type=single-node" opensearchstaging/opensearch:2.12.0.9419curl --insecure https://localhost:9200/ -u 'admin:myStrongPassword123!'OpenSearch-Dashboards docker 2.12.0.7295
docker pull opensearchstaging/opensearch-dashboards:2.12.0.7295 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.12.0.7295https://localhost:5601/docker pull opensearchstaging/opensearch-dashboards:2.12.0.7295 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.12.0.7295https://localhost:5601/Use TARs to deploy OpenSearch Manually
reta
commented
7 months ago While there is no performance impact on indexing throughput, we are seeing massive gains in overall and certain aggregate query metrics
@rishabh6788 just to satisfy personal curiosity, how many runs we do (to make sure numbers are stable across them)? Thank you.
ryanbogan
commented
7 months ago @prudhvigodithi We are going to manually sign off on this release for neural-search after discussions in the release channel and approval from @bbarani Issue with full context is here to be fixed before next release: https://github.com/opensearch-project/neural-search/issues/596
jackiehanyang
commented
7 months ago AD-Dashboard merged a PR into the 2.12 branch to fix the tests that failed after addressing this CVE - https://nvd.nist.gov/vuln/detail/CVE-2023-46308
rishabh6788
commented
7 months ago @reta The nightly runs for 2.11.0 and 2.12.0 have been running since 2.11.0 was released. We do one run a day and this has been running for past 3 months. Here's the performance graph of 2.11.0 vs 2.12.0 for past 4 months https://s12d.com/T6yK2_gh. Post 2.12.0 release we will work on improving our public dashboards and make it more meaningful and easy to understand.
peterzhuamazon
commented
7 months ago Recent discovery on Windows failing certain test on Cypress with @junqiu-lei :
cy.visit(`${BASE_PATH}/app/maps-dashboards/create`); could not be accessed correctly by cypress. This only happens on Windows with electron 15 (chromium 94) bundled with cypress 9.5.4, only on Windows.
cypress 9.5.4 version with bundled electron 15 (chromium 94) runs fine without any issues.cypress 9.5.4.
windows-chromium114-cypress954-maps-allpass-videos.logwindows image instead of the current windows/servercore.Solutions:
BROWSER_PATH=electron on jenkinsfile on windows section specifically to unblock windows integTest runs. Since the windows image already pre-load with chromium 114 binary.
windows/servercore:ltsc2019 to windows:ltsc2019 to unblock video recording.lock chromium to version 114 on all LINUX images to replace bundled electron. For example, if there is any chromium cli/binary installation available on npm or other package manager.
https://www.npmjs.com/package/download-chromium
cc: @bbarani @prudhvigodithi
Thanks.
ylwu-amzn
commented
7 months ago skills repo merged some PR
prudhvigodithi
commented
7 months ago OpenSearch - Build 9424 OpenSearch Dashboards - Build 7301
Use the following Docker-Compose to setup a cluster
version: '3'
services:
opensearch-node1:
image: opensearchstaging/opensearch:2.12.0.9424
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchstaging/opensearch:2.12.0.9424
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchstaging/opensearch-dashboards:2.12.0.7301
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
docker-compose pull.docker-compose up.docker pull opensearchstaging/opensearch:2.12.0.9424 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e DISABLE_INSTALL_DEMO_CONFIG=true -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.12.0.9424curl https://localhost:9200/docker pull opensearchstaging/opensearch:2.12.0.9424 && docker run -it -p 9200:9200 -e 'OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!' -e "discovery.type=single-node" opensearchstaging/opensearch:2.12.0.9424curl --insecure https://localhost:9200/ -u 'admin:myStrongPassword123!'OpenSearch-Dashboards docker 2.12.0.7301
docker pull opensearchstaging/opensearch-dashboards:2.12.0.7301 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.12.0.7301https://localhost:5601/docker pull opensearchstaging/opensearch-dashboards:2.12.0.7301 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.12.0.7301https://localhost:5601/Use TARs to deploy OpenSearch Manually
bbarani
commented
7 months ago prudhvigodithi
commented
7 months ago Coming from the latest RC build, following is the current status of 2.12.0 release.
Plugin teams please go through the issues and try to close them as soon as possible https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.12.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle+
The distribution build https://build.ci.opensearch.org//blue/organizations/jenkins/distribution-build-opensearch/detail/distribution-build-opensearch/9424/pipeline is SUCCESS
The docker scan URL is https://build.ci.opensearch.org/job/docker-scan/2930/artifact/scan_docker_image.txt
The distribution build https://build.ci.opensearch.org//blue/organizations/jenkins/distribution-build-opensearch-dashboards/detail/distribution-build-opensearch-dashboards/7301/pipeline is SUCCESS
The docker scan URL is https://build.ci.opensearch.org/job/docker-scan/2929/artifact/scan_docker_image.txt
https://github.com/issues?q=is%3Aopen+is%3Apr+user%3Aopensearch-project+label%3Av2.12.0+
https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.12.0+
r1walz
commented
7 months ago @prudhvigodithi fix for indexManagementDashboards
cc: @vikasvb90
rupal-bq
commented
7 months ago We merged https://github.com/opensearch-project/dashboards-reporting/pull/296 to fix Reporting button missing issue. Can we please take this change in today's RC generation? It's required to fix integration tests https://github.com/opensearch-project/opensearch-dashboards-functional-test/pull/1089.
prudhvigodithi
commented
7 months ago Fore more details please check: https://github.com/opensearch-project/opensearch-build/issues/4115#issuecomment-1948871340
bbarani
commented
7 months ago Anomaly detection with concurrent search failure is a known issue and its tracked in OpenSearch repo.. For 2.12, the workaround / shortcomings will be addressed by documentating the gaps.
ps48
commented
7 months ago @prudhvigodithi We have 3 PRs on functional test repo for cypress updates on:
ganttChartDashboards: https://github.com/opensearch-project/opensearch-dashboards-functional-test/pull/1090
reportsDashboards: https://github.com/opensearch-project/opensearch-dashboards-functional-test/pull/1089
observabilityDashboards: https://github.com/opensearch-project/opensearch-dashboards-functional-test/pull/1088
cc: @anirudha @ashwin-pc @kavilla
paulstn
commented
7 months ago @prudhvigodithi Piggybacking off of @ps48's comment, have one more PR on the functional test repository.
queryWorkbenchDashboards: https://github.com/opensearch-project/opensearch-dashboards-functional-test/pull/1092
cc: @anirudha @ashwin-pc @kavilla
Zhangxunmt
commented
7 months ago ml-commons IT tests are due to flaky tests issue tracked here https://github.com/opensearch-project/ml-commons/issues/2124. This is a known issue and often times not reproducable from local environment. In the worst case, only some IT tests needs tuning for time_out parameters, etc, which is not a blocker for any features in 2.12.
bbarani
commented
7 months ago ml-commons IT tests are due to flaky tests issue tracked here opensearch-project/ml-commons#2124. This is a known issue and often times not reproducable from local environment. In the worst case, only some IT tests needs tuning for time_out parameters, etc, which is not a blocker for any features in 2.12.
Update the ml-commons to green.
vamsi-amazon
commented
7 months ago sql plugin issue is a known issue https://github.com/opensearch-project/sql/issues/1275 which happens only when integ tests are run using few timezones. The timezones are randomly selected and so the results of that particular test are flaky. The issue shouldn't be a blocker. I will spend some time today on the fix. cc: @bbarani @ps48 @prudhvigodithi @anirudha
bbarani
commented
7 months ago @vamsi-amazon
sql plugin issue is a known issue opensearch-project/sql#1275 which happens only when integ tests are run using few timezones. The timezones are randomly selected and so the results of that particular test are flaky. The issue shouldn't be a blocker. I will spend some time today on the fix. cc: @bbarani @ps48 @prudhvigodithi @anirudha
Are you signing-off on the test failures related to sql plugin? Can I update the table?
vamsi-amazon
commented
7 months ago @bbarani Yes. Also, I just raised the PR for fix here: https://github.com/opensearch-project/sql/pull/2522. Its not a code fix rather fix in the integ test itself. Should I backport to 2.12 branch?
bbarani
commented
7 months ago @bbarani Yes. Also, I just raised the PR for fix here: opensearch-project/sql#2522. Its not a code fix rather fix in the integ test itself. Should I backport to 2.12 branch?
Yes please do. We will generate a new RC at 4pm PT today.
vamsi-amazon
commented
7 months ago Merged the PR. thanks @bbarani cc: @anirudha @ps48
prudhvigodithi
commented
7 months ago OpenSearch - Build 9430 OpenSearch Dashboards - Build 7307
Use the following Docker-Compose to setup a cluster
version: '3'
services:
opensearch-node1:
image: opensearchstaging/opensearch:2.12.0.9430
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchstaging/opensearch:2.12.0.9430
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchstaging/opensearch-dashboards:2.12.0.7307
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
docker-compose pull.docker-compose up.docker pull opensearchstaging/opensearch:2.12.0.9430 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e DISABLE_INSTALL_DEMO_CONFIG=true -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.12.0.9430curl https://localhost:9200/docker pull opensearchstaging/opensearch:2.12.0.9430 && docker run -it -p 9200:9200 -e 'OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!' -e "discovery.type=single-node" opensearchstaging/opensearch:2.12.0.9430curl --insecure https://localhost:9200/ -u 'admin:myStrongPassword123!'OpenSearch-Dashboards docker 2.12.0.7307
docker pull opensearchstaging/opensearch-dashboards:2.12.0.7307 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.12.0.7307https://localhost:5601/docker pull opensearchstaging/opensearch-dashboards:2.12.0.7307 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.12.0.7307https://localhost:5601/Use TARs to deploy OpenSearch Manually
xinyual
commented
7 months ago Hi team, I just merge a PR for skills plugin here https://github.com/opensearch-project/skills/pull/232/files. Will it be in the 2.12 release?
prudhvigodithi
commented
7 months ago Coming from the latest 8th RC build, following is the current status of 2.12.0 release.
Plugin teams please go through the issues and try to close them as soon as possible https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.12.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle+
The distribution build https://build.ci.opensearch.org//blue/organizations/jenkins/distribution-build-opensearch/detail/distribution-build-opensearch/9430/pipeline is SUCCESS
The docker scan URL is https://build.ci.opensearch.org/job/docker-scan/2934/artifact/scan_docker_image.txt
The distribution build https://build.ci.opensearch.org//blue/organizations/jenkins/distribution-build-opensearch-dashboards/detail/distribution-build-opensearch-dashboards/7307/pipeline is SUCCESS
The docker scan URL is https://build.ci.opensearch.org/job/docker-scan/2935/artifact/scan_docker_image.txt
https://github.com/issues?q=is%3Aopen+is%3Apr+user%3Aopensearch-project+label%3Av2.12.0+
https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.12.0+
prudhvigodithi
commented
7 months ago Hi team, I just merge a PR for skills plugin here https://github.com/opensearch-project/skills/pull/232/files. Will it be in the 2.12 release?
Hey @xinyual the RC is already successfully built. We have the Go/No-Go call for 2.12.0 release scheduled on Feb 19th at 9am PT. We will go over the exit criteria for the release and get the votes to finalize the next steps.
Thank you
prudhvigodithi
commented
7 months ago BWC test job triggered for 8th RC https://build.ci.opensearch.org/blue/organizations/jenkins/bwc-test/detail/bwc-test/4845/pipeline
2024-02-17 01:30:33 ERROR | alerting | with-security | FAIL |
2024-02-17 01:30:33 ERROR | anomaly-detection | with-security | FAIL |
2024-02-17 01:30:33 ERROR | index-management | with-security | FAIL |
2024-02-17 01:30:33 INFO | notifications | with-security | PASS |
2024-02-17 01:30:33 ERROR | opensearch-observability | with-security | FAIL |
2024-02-17 01:30:33 ERROR | sql | with-security | FAIL |There is an open META issue for plugins to onboard to the BWC test framework https://github.com/opensearch-project/opensearch-build/issues/3671, so at this point the assumption is the BWC tests are executed at the plugin CI level. @bbarani
hdhalter
commented
7 months ago Documentation is GREEN for the 2.12 release.
CaptainDredge
commented
7 months ago Tracking CCR Plugin integ test failure issue in #1334 Issue. Tests are passing locally but there are some admin credential issues in release CI with latest changes. By this we're manually signing-off on CCR plugin
CaptainDredge
commented
7 months ago ISM dashboard plugin has a flaky test failure for which the fix has been raised but will take some time to get to approval stage https://github.com/opensearch-project/index-management-dashboards-plugin/issues/997 Giving manual signoff on index management dashboard
ps48
commented
7 months ago Tracking flaky integ tests on dashboardsReports, queryWorkbench and ganttCharts here: https://github.com/opensearch-project/opensearch-dashboards-functional-test/issues/1113. We manually tested these failures manually. We're giving manual sign-off.
prudhvigodithi
commented
7 months ago Tracking flaky integ tests on dashboardsReports, queryWorkbench and ganttCharts here: opensearch-project/opensearch-dashboards-functional-test#1113. We manually tested these failures manually. We're giving manual sign-off.
Updated ganttChartDashboards, queryWorkbenchDashboards and reportsDashboards to green.
https://github.com/opensearch-project/opensearch-build/issues/4115#issuecomment-1948972821
bbarani
commented
7 months ago The vote was No-go for 2.12.0 release as we couldn't clear the exit criteria for integration tests.
Next Go-No go call is scheduled for tomorrow (Feb 20 2024) morning at 9am PT.
kavilla
commented
7 months ago For OpenSearch Dashboards:
We are seeing test failures on VisBuilder tests. When looking at the results we see a component overlaying it with the classname: vbConfig.
Which aligns to: https://github.com/opensearch-project/dashboards-observability/pull/1135.
After uninstalling the observability plugin:
bbarani
commented
7 months ago For OpenSearch Dashboards:
We are seeing test failures on VisBuilder tests. When looking at the results we see a component overlaying it with the classname:
vbConfig.
Which aligns to: opensearch-project/dashboards-observability#1135.
After uninstalling the observability plugin:
@vamsi-amazon @anirudha @joshuali925 Can you please comment on this issue?
joshuali925
commented
7 months ago @mengweieric i think you can use a different classname on https://github.com/opensearch-project/dashboards-observability/blob/2409cf7ddfaa255653a5156d0fc9f3a7d6a9457d/public/components/event_analytics/explorer/visualizations/config_panel/config_panes/config_controls/data_configurations_panel.scss#L44
sohami
commented
7 months ago As discussed while debugging AD integ test failure, the issue is found to be generic with composite aggregation when concurrent segment search is enabled. So we are disabling concurrent search for composite aggregation as part of 2.12. Tracking PR for main branch: https://github.com/opensearch-project/OpenSearch/pull/12375
PR for 2.12 branch: https://github.com/opensearch-project/OpenSearch/pull/12380
prudhvigodithi
commented
7 months ago OpenSearch - Build 9445 OpenSearch Dashboards - Build 7326
Use the following Docker-Compose to setup a cluster
version: '3'
services:
opensearch-node1:
image: opensearchstaging/opensearch:2.12.0.9445
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchstaging/opensearch:2.12.0.9445
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123@456"
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchstaging/opensearch-dashboards:2.12.0.7326
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
docker-compose pull.docker-compose up.docker pull opensearchstaging/opensearch:2.12.0.9445 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e DISABLE_INSTALL_DEMO_CONFIG=true -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.12.0.9445curl https://localhost:9200/docker pull opensearchstaging/opensearch:2.12.0.9445 && docker run -it -p 9200:9200 -e 'OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!' -e "discovery.type=single-node" opensearchstaging/opensearch:2.12.0.9445curl --insecure https://localhost:9200/ -u 'admin:myStrongPassword123!'OpenSearch-Dashboards docker 2.12.0.7326
docker pull opensearchstaging/opensearch-dashboards:2.12.0.7326 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.12.0.7326https://localhost:5601/docker pull opensearchstaging/opensearch-dashboards:2.12.0.7326 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.12.0.7326https://localhost:5601/Use TARs to deploy OpenSearch Manually
prudhvigodithi
commented
7 months ago | Repo | Integ-Tests | AUTOCUT Issue | Comments |
|---|---|---|---|
| security-analytics | :green_circle: | https://github.com/opensearch-project/security-analytics/issues/792 | looks flaky as passed for x64 tar, Manual sign off in the Go/No-Go call by @praveensameneni. |
| cross-cluster-replication | :green_circle: | https://github.com/opensearch-project/cross-cluster-replication/issues/1317 | Manually signed-off as per this comment |
| neural-search | :green_circle: | https://github.com/opensearch-project/neural-search/issues/547 | Fail with and without concurrent search enabled, Manually signed-off as per this comment, recent integ build passed. |
| alerting | :green_circle: | https://github.com/opensearch-project/alerting/issues/1381 | Manual sign off in the Go/No-Go call by @praveensameneni. |
| ml-commons | :green_circle: | https://github.com/opensearch-project/ml-commons/issues/1789 | Fails with concurrent search enabled, looks flaky as passed for x64 tar and failed for tar arm64. Passed on regular integration test without concurrent search enabled. Confirmed by the team as flaky tests and its tracked here. Manually signed off per this comment, manual sign off from comment. |
| Repo | Integ-Tests | AUTOCUT Issue | Comments |
|---|---|---|---|
| OpenSearch-Dashboards | :green_circle: | https://github.com/opensearch-project/OpenSearch-Dashboards/issues/5633 | Manually signed-off as per comment |
| reportsDashboards | :green_circle: | https://github.com/opensearch-project/dashboards-reporting/issues/306 | Manually signed-off as per comment |
| ganttChartDashboards | :green_circle: | https://github.com/opensearch-project/dashboards-visualizations/issues/345 | Manually signed-off as per comment |
| alertingDashboards | :green_circle: | https://github.com/opensearch-project/alerting-dashboards-plugin/issues/867 | Manual sign off from comment. |
| indexManagementDashboards | :green_circle: | https://github.com/opensearch-project/index-management-dashboards-plugin/issues/968 | Manually signed-off as per comment and comment |
AWSHurneyt
commented
7 months ago Providing manual sign off for alertingDashboards.
The tests pass with security in this run from earlier today - https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/5633/pipeline/117#step-638-log-678
And the tests pass without security in this run from 3 days ago - https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/5621/pipeline/#step-1046-log-162
Since the last new changes merged to the 2.12 branch 5 days ago, this suggests the tests are flaky. PR https://github.com/opensearch-project/opensearch-dashboards-functional-test/pull/1115 updated our functional test repo to increase wait times as we noticed it was taking a long time for pages to load, which was causing tests to fail. The tests became much more reliable when executed locally after these changes.
kavilla
commented
7 months ago For OpenSearch Dashboards,
When looking without-security results. I see the following:
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ core-opensearch-dashboards/opensear 94ms 1 1 - - - │
│ ch-dashboards/apps/data_explorer/aa │
│ a_before.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:10 4 4 - - - │
│ ch-dashboards/apps/data_explorer/da │
│ ta_source_selector.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:12 1 1 - - - │
│ ch-dashboards/apps/data_explorer/da │
│ te_nanos_mixed.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:11 1 1 - - - │
│ ch-dashboards/apps/data_explorer/da │
│ te_nanos.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 01:36 3 3 - - - │
│ ch-dashboards/apps/data_explorer/di │
│ scover_histogram.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 01:42 17 14 - 3 - │
│ ch-dashboards/apps/data_explorer/di │
│ scover.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:52 2 2 - - - │
│ ch-dashboards/apps/data_explorer/do │
│ c_navigation.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:21 2 2 - - - │
│ ch-dashboards/apps/data_explorer/do │
│ c_table.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:08 1 1 - - - │
│ ch-dashboards/apps/data_explorer/er │
│ rors.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:47 6 6 - - - │
│ ch-dashboards/apps/data_explorer/fi │
│ eld_data.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:47 5 5 - - - │
│ ch-dashboards/apps/data_explorer/fi │
│ eld_visualize.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:34 3 3 - - - │
│ ch-dashboards/apps/data_explorer/fi │
│ lter_editor.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:11 2 2 - - - │
│ ch-dashboards/apps/data_explorer/in │
│ dex_pattern_with_encoded_id.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:13 2 2 - - - │
│ ch-dashboards/apps/data_explorer/in │
│ dex_pattern_without_field.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:16 2 2 - - - │
│ ch-dashboards/apps/data_explorer/in │
│ spector.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:14 4 3 - 1 - │
│ ch-dashboards/apps/data_explorer/la │
│ rge_string.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 34ms 11 - - 11 - │
│ ch-dashboards/apps/data_explorer/sa │
│ ved_queries.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:37 6 6 - - - │
│ ch-dashboards/apps/data_explorer/sh │
│ ared_links.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:06 1 1 - - - │
│ ch-dashboards/apps/data_explorer/si │
│ debar.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:07 1 1 - - - │
│ ch-dashboards/apps/data_explorer/so │
│ urce_filter.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:05 1 1 - - - │
│ ch-dashboards/apps/data_explorer/zz │
│ z_after.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:35 5 5 - - - │
│ ch-dashboards/apps/vis_builder/basi │
│ c.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ core-opensearch-dashboards/opensear 04:19 4 2 2 - - │
│ ch-dashboards/apps/vis_builder/dash │
│ board.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:21 4 4 - - - │
│ ch-dashboards/apps/vis_builder/expe │
│ rimental.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:13 1 1 - - - │
│ ch-dashboards/apps/vis_builder/vis_ │
│ types/area.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:14 1 1 - - - │
│ ch-dashboards/apps/vis_builder/vis_ │
│ types/bar.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:13 1 1 - - - │
│ ch-dashboards/apps/vis_builder/vis_ │
│ types/line.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:12 1 1 - - - │
│ ch-dashboards/apps/vis_builder/vis_ │
│ types/metric.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:16 1 1 - - - │
│ ch-dashboards/apps/vis_builder/vis_ │
│ types/table.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:22 4 4 - - - │
│ ch-dashboards/apps/vis_type_table/b │
│ asic.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 01:02 9 9 - - - │
│ ch-dashboards/apps/vis_type_table/d │
│ ata.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ core-opensearch-dashboards/opensear 01:43 4 3 1 - - │
│ ch-dashboards/apps/vis_type_table/e │
│ mbed.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:41 11 11 - - - │
│ ch-dashboards/apps/vis_type_table/o │
│ ptions.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 16ms 7 - - 7 - │
│ ch-dashboards/apps/vis_type_table/s │
│ plit.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ core-opensearch-dashboards/opensear 03:28 2 - 2 - - │
│ ch-dashboards/apps/vis-augmenter/da │
│ shboard_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 2ms - - - - - │
│ ch-dashboards/dashboard_sample_data │
│ _with_datasource_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 00:38 34 34 - - - │
│ ch-dashboards/dashboard_sanity_test │
│ _spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 1ms - - - - - │
│ ch-dashboards/dashboard_share_copy_ │
│ link_test.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 1ms - - - - - │
│ ch-dashboards/datasource-management │
│ -plugin/1_create_datasource.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 1ms - - - - - │
│ ch-dashboards/datasource-management │
│ -plugin/2_datasource_table.spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ core-opensearch-dashboards/opensear 1ms - - - - - │
│ ch-dashboards/datasource-management │
│ -plugin/3_update_datasource.spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✖ 3 of 41 failed (7%) 23:41 165 138 5 22 - Running locally, the first two spec files succeed every time. With and without security:
.
The last test failure is due to it trying to open the doc site from the branch 2.12 in the test: https://opensearch.org/docs/2.12/ which doesn't exist yet.
The last one I will associate a PR to ensure that it doesn't kill the test when the doc site tries to open but spins. And I will fix the flakiness of the first two but not seeing anything that worries me there. Manual sign off for me.
ylwu-amzn
commented
7 months ago For ml-commons, the test looks flaky as they only failed on ARM64. From the log , seems all tests failed, that's not normal. I guess the integ test cluster has some issue. All failed test throw same error Connection refused, that seems test cluster not ready or has some issue. Possible reason is the cluster heap size is too small and deploying ML model crashed the cluster.
rishabh6788
commented
7 months ago For ml-commons, the test looks flaky as they only failed on ARM64. From the log , seems all tests failed, that's not normal. I guess the integ test cluster has some issue. All failed test throw same error
Connection refused, that seems test cluster not ready or has some issue. Possible reason is the cluster heap size is too small and deploying ML model crashed the cluster.
The stderr log has the following:
/opt/java/openjdk-17/bin/java: relocation error: /tmp/tmp3mw_9esm/1/local-test-cluster/opensearch-2.12.0/data/ml_cache/pytorch/1.13.1-cpu-precxx11-linux-aarch64/libstdc++.so.6: symbol __cxa_thread_atexit_impl, version GLIBC_2.18 not defined in file libc.so.6 with link time reference@ylwu-amzn
Release OpenSearch and OpenSearch Dashboards 2.12.0
I noticed that a manifest was automatically created in manifests/2.12.0. Please follow the following checklist to make a release.
How to use this issue
## This Release Issue This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the [Release Process OpenSearch Guide](https://github.com/opensearch-project/opensearch-build/blob/main/RELEASE_PROCESS_OPENSEARCH.md).
Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.
Entrance Criteria
OSD Components
OpenSearch 2.12.0 exit criteria status:
OpenSearch-Dashboards 2.12.0 exit criteria status:
Preparation
Campaigns
dashboards-assistantOnboard: https://github.com/opensearch-project/opensearch-build/issues/4322opensearch-skillsonboard: https://github.com/opensearch-project/opensearch-build/issues/4378opensearch-flow-frameworkOnboard: https://github.com/opensearch-project/opensearch-build/issues/4209Release Branch and Version Increment - End of February 4th, 2024
Feature Freeze - End of February 5th, 2024
Code Complete - End of February 5th, 2024
Release Candidate Creation and Testing - First RC Generated (release window opens) on February 6th, 2024
Performance testing validation - End of February 17th, 2024
Pre Release - End of February 20th, 2024
Release - February 20th, 2024
Release Checklist.
Release Checklist
### Pre-Release activities - [x] Promote Repos. - - [x] OS https://build.ci.opensearch.org/job/distribution-promote-repos/95/console https://build.ci.opensearch.org/job/distribution-promote-repos/96/console - - [x] OSD https://build.ci.opensearch.org/job/distribution-promote-repos/97/console https://build.ci.opensearch.org/job/distribution-promote-repos/98/console - [x] Promote Artifacts. - - [x] Windows OS: https://build.ci.opensearch.org/job/distribution-promote-artifacts/396/console OSD: https://build.ci.opensearch.org/job/distribution-promote-artifacts/397/console - - [x] Linux Debian - - [x] OS: https://build.ci.opensearch.org/job/distribution-promote-artifacts/398/console https://build.ci.opensearch.org/job/distribution-promote-artifacts/399/console - - [x] OSD: https://build.ci.opensearch.org/job/distribution-promote-artifacts/400/console https://build.ci.opensearch.org/job/distribution-promote-artifacts/401/console - - [x] Linux RPM - - [x] OS: https://build.ci.opensearch.org/job/distribution-promote-artifacts/402/console https://build.ci.opensearch.org/job/distribution-promote-artifacts/403/console - - [x] OSD: https://build.ci.opensearch.org/job/distribution-promote-artifacts/404/console https://build.ci.opensearch.org/job/distribution-promote-artifacts/405/console - - [x] Linux TAR - - [x] OS: https://build.ci.opensearch.org/job/distribution-promote-artifacts/408/console https://build.ci.opensearch.org/job/distribution-promote-artifacts/409/console - - [x] OSD: https://build.ci.opensearch.org/job/distribution-promote-artifacts/406/console https://build.ci.opensearch.org/job/distribution-promote-artifacts/407/console - [x] Consolidated Release Notes. https://github.com/opensearch-project/opensearch-build/pull/3803 ### Release activities - [x] Docker Promotion: https://build.ci.opensearch.org/job/docker-promotion/62/console - [x] Release Validation part 1. - - [x] OpenSearch and OpenSearch Dashboard Validation. - - [x] Validate the native plugin installation. - [x] Merge consolidated release notes PR. - [x] Website and Documentation Changes. - - [x] Merge staging website PR. - - [x] Promote the website changes to prod. - - [x] Add website alert. - [x] Release Validation part 2. - - [x] Validate the artifact download URL's and signatures. - [x] Release Validation part 3. - - [x] Trigger the validation build (Search for `Completed validation for <>` in the logs). - [x] Maven Promotion: https://build.ci.opensearch.org/job/publish-to-maven/28/ - [x] Publish blog posts. - [x] Advertise on Social Media. - [x] Post on public slack and Github Release issue. ### Post-Release activities - [x] Release Tags. https://build.ci.opensearch.org/job/distribution-release-tag-creation/98/console https://build.ci.opensearch.org/job/distribution-release-tag-creation/99/console https://build.ci.opensearch.org/job/distribution-release-tag-creation/100/ - [x] Input Manifest Update: https://github.com/opensearch-project/opensearch-build/pull/4467/files - [x] Decrease the Build Frequency. - [x] OpenSearch Build Release notes: https://github.com/opensearch-project/opensearch-build/releases/tag/2.12.0 - [x] Retrospective Issue: https://github.com/opensearch-project/opensearch-build/issues/4454 - [x] Helm and Ansible Playbook release. https://github.com/opensearch-project/helm-charts/pull/518 https://github.com/opensearch-project/ansible-playbook/pull/153 - [x] Upcoming Release Preparation: This process is automated as the version increment PR's, release issues are created by automated workflows.
Post Release
Components
OpenSearch
OpenSearch Dashboards
Documentation
Legend
| Symbol | Meaning | | -------- | ---------- | | :green_circle: | On track with overall release | | :yellow_circle: | Missed last milestone | | :red_circle: | Missed multiple milestones |