yomorun / yomo

🦖 Stateful Serverless Framework for Geo-distributed Edge AI Infra

https://yomo.run

Apache License 2.0

1.64k stars 128 forks source link

refactor: use goroutinue instead of mutex to sync messages #842

Closed woorui closed 1 week ago

woorui commented 2 weeks ago

Description

The HTTP server startup creates a CallerProvider struct, which provides a Provide(credential string) method that returns a Caller.

The Caller and provider work together to provide invoke and chat completion functionality.

The Caller has a CallerSyncer attribute, which is responsible for synchronizing pub/sub information to the req/res.

Impact

The register no longer requires the Factor method, which ensures the correct execution of yomo test-prompt.

On the server-side, it is guaranteed that llm-sfn can only have one return, and if there are multiple returns, the first one will be used. If there is no return, the client will be notified of a timeout.

Test

Provide a mock provider which can record the requests to OpenAI and mock the responses from OpenAI.

Provide a mockDataFlow which is used to mock the data flow in Yomo, from the source -> handler -> reducer.

vercel[bot] commented 2 weeks ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 73.90300% with 113 lines in your changes missing coverage. Please review.

Project coverage is 78.73%. Comparing base (cc2e6f2) to head (68a4399).

Files	Patch %	Lines
pkg/bridge/ai/caller.go	68.18%	38 Missing and 4 partials :warning:
pkg/bridge/ai/api_server.go	57.57%	24 Missing and 4 partials :warning:
pkg/bridge/ai/call_syncer.go	81.60%	16 Missing and 7 partials :warning:
pkg/bridge/ai/provider/mock.go	85.07%	6 Missing and 4 partials :warning:
pkg/bridge/ai/ai.go	72.00%	5 Missing and 2 partials :warning:
pkg/bridge/ai/provider/openai/provider.go	60.00%	1 Missing and 1 partial :warning:
serverless/mock/mock_context.go	0.00%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #842 +/- ## =========================================== + Coverage 64.93% 78.73% +13.79% =========================================== Files 45 47 +2 Lines 2658 2718 +60 =========================================== + Hits 1726 2140 +414 + Misses 820 427 -393 - Partials 112 151 +39 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.