lyst / lightfm

A Python implementation of LightFM, a hybrid recommendation algorithm.
Apache License 2.0
4.66k stars 679 forks source link

Feature Request: Support for Freezing Item Embeddings while Updating User Embeddings During Training #694

Closed iDataist closed 6 days ago

iDataist commented 12 months ago

I am currently working on a project using LightFM for building a hybrid recommendation system. During the training process, I am trying to fix the item embeddings and only update the user embeddings.

I found a discussion in issue #300 where a workaround was mentioned to effectively freeze the item embeddings by setting the accumulated squared gradients to their maximum value using the adagrad optimizer:

import numpy as np

model.item_embedding_gradients = np.finfo(np.float32).max
model.item_bias_gradients = np.finfo(np.float32).max

While this approach mostly accomplishes what I am looking for, it's more of a workaround and still leaves room for very tiny updates in item embeddings due to the nature of the adagrad optimizer.

It would be extremely beneficial for many use cases, including mine, if LightFM could natively support an option to freeze item embeddings during training. This feature could be integrated as a parameter in the fit method, for example:

model.fit(interactions, user_features, item_features, epochs=10, freeze_item_embeddings=True)

This would give developers more flexibility in fine-tuning their models, especially in cases where it's essential to retain item representations while updating the user representations.

Please let me know if there's any additional information needed, or if there's a way I can contribute to making this feature available.

Thank you for your time and consideration.