search

opensearch-project / k-NN

🆕 Find the k-nearest neighbors (k-NN) for your vector data
https://opensearch.org/docs/latest/search-plugins/knn/index/
Apache License 2.0
156 stars 123 forks source link

Should we setup e2e tests separately from integ tests? #2208

Open jmazanec15 opened 1 month ago

jmazanec15 commented 1 month ago

Description

For this comment from @heemin32 (https://github.com/opensearch-project/k-NN/pull/2200#pullrequestreview-2361589027), I was thinking I could add recall tests for this as well, but this can come with cost of adding minutes to the CI.

That being said, should we have e2e tests that we know take longer to run and could be triggered selectively by a maintainer and/or on a nightly or weekly basis? Thus, for most changes we wouldnt need to run these. What does everyone think?

I think AD has setup their repo to do this. See https://github.com/opensearch-project/anomaly-detection/blob/main/build.gradle#L321-L369.

jmazanec15 commented 1 month ago

Discussed step: As first step, we should split up e2e tests and integ tests so that they can run in parallel for CI. In future, we can revisit selectively running.