search

spiceai / quickstarts

Quickstart projects to get up and running with Spice.ai quickly 🚀
Apache License 2.0
20 stars 9 forks source link

Serverops quickstart e2e test is flaky #55

Closed phillipleblanc closed 2 years ago

phillipleblanc commented 2 years ago

The serverops quickstart e2e test is flaky.

In this run, attempt #2 was successful and attempt #1 failed - despite no differences between them:

Attempt 2 (succeeded): https://github.com/spiceai/quickstarts/actions/runs/1625609207 Attempt 1 (failed): https://github.com/spiceai/quickstarts/actions/runs/1625609207/attempts/1

On cursory investigation, it looks like the training doesn't complete in time before the test gets the recommendation, so the recommendation has a confidence of 0. Re-running the test can allow the training to finish in time and make it pass.

lukekim commented 2 years ago

Resolved. Tests have been stable for months now.