Closed markvai closed 1 year ago
@matteo-grella
Does implementing the io.Closer
interface for all models sounds like a proper solution?
Hi @markvai thanks for this. In two to three days we have a better memory handling incl. embeddings by updating spago to the latest main branch. That will solve your problem. I’ll back to you here shortly asking for help on testing!
The ability to free up resources was added to all tasks in https://github.com/nlpodyssey/cybertron/pull/18 Closing
Summary: Each time a model is loaded, there is no way to free up the used resources for the local DB (
embeddingsRepo
)Example:
I am using code that loads a model on each classification task like so:
And I noticed that on each execution the memory increases.
This is due to the fact that nowhere in code it's possible to call
embeddingsRepo.Close()
as the docs state. The field is private, so even if trying to cast explicitly, the field can't be accessed.This is enforcing me to create a single object of the classifier while running and hence to increase the memory footprint.