search

openshift / origin-aggregated-logging

139 stars 230 forks source link

Bug 1895581: Replace DLS rule script with more performant terms filter #2032

Closed lukas-vlcek closed 3 years ago

lukas-vlcek commented 3 years ago

Description

Use terms query instead of painless scripting. This is much more performant and uses a lot less resources.

Relevant ES docs:

This PR assume that es-proxy provide list of projects in comma-delimited form where every item is quoted.

Watch for associated PR in https://github.com/openshift/elasticsearch-proxy before merging.

/cc @vimalk78 /assign @jcantrill

Links

lukas-vlcek commented 3 years ago

BTW, it should be pointed out that project names are used for filtering as is without any modifications (precisely, they are indexed as keywords which means no text analysis is run on them before they are put into index). I.e. "Dummy" and "dummy" are treated as two different projects. The original script was lowercasing the values.

lukas-vlcek commented 3 years ago

/test smoke

openshift-ci-robot commented 3 years ago

@lukas-vlcek: This pull request references Bugzilla bug 1895581, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug * bug is open, matching expected state (open) * bug target release (4.7.0) matches configured target release for branch (4.7.0) * bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
In response to [this](https://github.com/openshift/origin-aggregated-logging/pull/2032): >Bug 1895581: Replace DLS rule script with more performant terms filter Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
lukas-vlcek commented 3 years ago

/hold

openshift-merge-robot commented 3 years ago

@lukas-vlcek: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/clo-functional 4d959457f0e5127a32c3b5e9915300347b1208c5 link /test clo-functional
ci/prow/smoke 4d959457f0e5127a32c3b5e9915300347b1208c5 link /test smoke
ci/prow/elastic-operator-e2e 4d959457f0e5127a32c3b5e9915300347b1208c5 link /test elastic-operator-e2e

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
lukas-vlcek commented 3 years ago

/hold cancel

jcantrill commented 3 years ago

/retest

ewolinetz commented 3 years ago

this is going to be dependent on https://github.com/openshift/elasticsearch-proxy/pull/68 being merged first

ewolinetz commented 3 years ago

/approve

jcantrill commented 3 years ago

/retest

ewolinetz commented 3 years ago

/test smoke

lukas-vlcek commented 3 years ago

/test smoke

ewolinetz commented 3 years ago

/test smoke

ewolinetz commented 3 years ago

/lgtm

openshift-ci-robot commented 3 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ewolinetz, lukas-vlcek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openshift/origin-aggregated-logging/blob/master/OWNERS)~~ [ewolinetz] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
openshift-bot commented 3 years ago

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented 3 years ago

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented 3 years ago

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented 3 years ago

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented 3 years ago

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented 3 years ago

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot commented 3 years ago

@lukas-vlcek: All pull requests linked via external trackers have merged:

Bugzilla bug 1895581 has been moved to the MODIFIED state.

In response to [this](https://github.com/openshift/origin-aggregated-logging/pull/2032): >Bug 1895581: Replace DLS rule script with more performant terms filter Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
openshift-ci-robot commented 3 years ago

@lukas-vlcek: Bugzilla bug 1895581 is in an unrecognized state (ON_QA) and will not be moved to the MODIFIED state.

In response to [this](https://github.com/openshift/origin-aggregated-logging/pull/2032): >Bug 1895581: Replace DLS rule script with more performant terms filter Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
openshift-ci-robot commented 3 years ago

@lukas-vlcek: Bugzilla bug 1895581 is in an unrecognized state (ON_QA) and will not be moved to the MODIFIED state.

In response to [this](https://github.com/openshift/origin-aggregated-logging/pull/2032): >Bug 1895581: Replace DLS rule script with more performant terms filter Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
lukas-vlcek commented 3 years ago

/cherry-pick release-4.6

lukas-vlcek commented 3 years ago

/cherry-pick release-4.5

openshift-cherrypick-robot commented 3 years ago

@lukas-vlcek: new pull request created: #2040

In response to [this](https://github.com/openshift/origin-aggregated-logging/pull/2032#issuecomment-756216584): >/cherry-pick release-4.6 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
openshift-cherrypick-robot commented 3 years ago

@lukas-vlcek: new pull request created: #2041

In response to [this](https://github.com/openshift/origin-aggregated-logging/pull/2032#issuecomment-756216660): >/cherry-pick release-4.5 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.