We currently have a few things we'd like to clean up in our integration test suite:
There's a lot of flakiness in our current test setup as we're not checking for index readiness when deleting, working with collections, and configuring indexes. We end up seeing a lot of transient errors along the lines of "Index was not ready for that operation yet."
We have a number of unit tests which are included in the IntegrationTests struct.
We have different integration tests for Client with and without sourceTag, which is kind of overkill. We unit test applying the user-agent details, and we don't need to rerun the same tests with and without sourceTag.
Solution
Refactor GetIndexStatus into WaitUntilIndexReady - just felt a bit easier to reason about, and also polls at a more frequent cadence.
Make sure we're calling WaitUntilIndexReady in places where we've called ConfigureIndex or creating collections from indexes. We also want to make sure indexes are ready before deleting.
Move unit tests out of the IntegrationTests struct and reorganize things a bit.
Create the collection for pod indexes in SetupSuite and reuse this for collection tests rather than creating numerous collections.
Type of Change
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[X] Infrastructure change (CI configs, etc)
[ ] Non-code change (docs, etc)
[ ] None of the above: (explain here)
Test Plan
Make sure integration tests pass in CI. To run locally use just test after setting up your PINECONE_API_KEY.
To see the specific tasks where the Asana app for GitHub is being used, see below:
Problem
We currently have a few things we'd like to clean up in our integration test suite:
IntegrationTests
struct.Client
with and withoutsourceTag
, which is kind of overkill. We unit test applying the user-agent details, and we don't need to rerun the same tests with and withoutsourceTag
.Solution
GetIndexStatus
intoWaitUntilIndexReady
- just felt a bit easier to reason about, and also polls at a more frequent cadence.WaitUntilIndexReady
in places where we've calledConfigureIndex
or creating collections from indexes. We also want to make sure indexes are ready before deleting.IntegrationTests
struct and reorganize things a bit.SetupSuite
and reuse this for collection tests rather than creating numerous collections.Type of Change
Test Plan
Make sure integration tests pass in CI. To run locally use
just test
after setting up yourPINECONE_API_KEY
.