Garden-AI / garden

https://garden-ai.readthedocs.io
MIT License
21 stars 4 forks source link

Handle modal invocations asynchronously #553

Closed hholb closed 5 days ago

hholb commented 6 days ago

Resolves: #552

Overview

This PR updates the backend client to hit the new async modal invocation routes. The invocations are still synchronous from the user's perspective, but this allows for long-running invocations without hitting the gateway timeout for a single request to the backend.

There are no major changes in logic other than a polling loop to check the status of the invocation. There is a new environment variable "BACKEND_POLL_INTERVAL_SECONDS" to configure how often we poll for the invocation status. The default is set for 0.1 seconds, or 100ms.

Discussion

We may want some sort of back-off logic in the poll interval so really long running jobs don't ping the server so much, but this probably won't be an issue until we have hundreds of invocations in-progress simultaneously. I'll leave that for an issue in the future.

Testing

Updated existing tests. Manual testing.

Documentation

Nothing changes from the users perspective.


πŸ“š Documentation preview πŸ“š: https://garden-ai--553.org.readthedocs.build/en/553/

codecov[bot] commented 6 days ago

Codecov Report

Attention: Patch coverage is 37.50000% with 10 lines in your changes missing coverage. Please review.

Project coverage is 74.66%. Comparing base (3b7e1bc) to head (65ec0af). Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
garden_ai/backend_client.py 16.66% 10 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #553 +/- ## ========================================== - Coverage 74.91% 74.66% -0.25% ========================================== Files 38 38 Lines 2408 2424 +16 ========================================== + Hits 1804 1810 +6 - Misses 604 614 +10 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features: