search

substratusai / lingo

Lightweight ML model proxy and autoscaler for kubernetes
https://www.substratus.ai
Apache License 2.0
102 stars 6 forks source link

Refactor towards sync.Map #36

Closed alpe closed 7 months ago

alpe commented 7 months ago

There was a problem with the mutex protecting the endpoints map. The getEndpoints method is used by the internal reconcile as well as external calls via GetHost/GetAllHosts from the http handler. This can be abused and may affect the service quality and block the reconcile process

The sync.Map though is optimized for this use case:

(1) when the entry for a given key is only ever written once but read many times, as in caches that only grow

https://pkg.go.dev/sync#Map

I noticed this pattern in other areas as well. I am very new to this project and will do smaller PRs to address them.

alpe commented 7 months ago

The preference is on type safety. See: https://github.com/substratusai/lingo/pull/39#discussion_r1432360834