Fix llm memory leak and replace mutex_lock with unique_lock

v6d-io / v6d

vineyard (v6d): an in-memory immutable data manager. (Project under CNCF, TAG-Storage)

https://v6d.io

Apache License 2.0

819 stars 117 forks source link

Fix llm memory leak and replace mutex_lock with unique_lock #1815

Closed vegetableysm closed 4 months ago

vegetableysm commented 4 months ago

What do these changes do?

Fix memory leak of llm module.
Use unique_lock to manager mutex locks in llm module.

Related issue number

Fixes #1813 Fixes #1814

vegetableysm commented 4 months ago

The recycle process cannot be placed in the cache manager destructor. Because the client stored in cache manager is provided by the user. We cannot assume the order of destruction of cache manager and client. Additionally, if the client calls disconnect function before the manager destroys, the recycle process will also fail. So I add a Close function to recycle memory. Even if user forgets to call the Close function, it will not affect the subsequent use of the cache, only cause a memory leak.