benfred / implicit

Fast Python Collaborative Filtering for Implicit Feedback Datasets
https://benfred.github.io/implicit/
MIT License
3.54k stars 611 forks source link

Freeze pretrained weights #467

Closed phiweger closed 2 years ago

phiweger commented 3 years ago

Following up on #341 : Is it possible to freeze the pretrained weights, so that they are not changed during training?

benfred commented 3 years ago

Is the use case here incremental retraining ? I'd like to add support for that at some point - though I was thinking of a different API than freezing items you don't want to update

phiweger commented 3 years ago

@benfred yes, I'd like to train say item embeddings using some other method (like a variational autoencoder) and then use these latent features in the recommender, but not modify them during training.

slowwavesleep commented 2 years ago

It seems that I came up with same idea as @phiweger.

I think adding something like if not self.freeze_item_factors before this line would do the trick. Which would make Alternating Least Squares not alternating any more, I guess? But still it would be nice to be able to use an existing library for this. For example, to be able to easily evaluate the effect of using existing item factors.

I do realize though, that it would take quite a bit more more work than that to implement this feature.

benfred commented 2 years ago

There is a PR here which should make this possible #527 (as well as let you incrementally retrain models on subsets of users/items).

As an example given a user_items sparse matrix containing interactions, and a existing_item_factors dense matrix containing precalculated item factors:

# Train an ALS model with pre-existing item factors, but calculating user factors
model = AlternatingLeastSquares()
model.item_factors = existing_item_factors
userids = np.arange(user_items.shape[0])
model.partial_fit_users(userids, user_items)