Memory leak when loading a model

[markvai]

Summary: Each time a model is loaded, there is no way to free up the used resources for the local DB (embeddingsRepo)

Example:

I am using code that loads a model on each classification task like so:

classifier, err := tasks.LoadModelForTokenClassification(&tasks.Config{...})
...

And I noticed that on each execution the memory increases.

This is due to the fact that nowhere in code it's possible to call embeddingsRepo.Close() as the docs state. The field is private, so even if trying to cast explicitly, the field can't be accessed.

This is enforcing me to create a single object of the classifier while running and hence to increase the memory footprint.

[markvai]

@matteo-grella Does implementing the io.Closer interface for all models sounds like a proper solution?

[matteo-grella]

Hi @markvai thanks for this. In two to three days we have a better memory handling incl. embeddings by updating spago to the latest main branch. That will solve your problem. I’ll back to you here shortly asking for help on testing!

[markvai]

The ability to free up resources was added to all tasks in https://github.com/nlpodyssey/cybertron/pull/18 Closing