Incorporating item and user biases in LightFM embeddings for accurate recommendations in Vespa

[Homa-Radaei]

I am working on deploying a recommender system using LightFM and Vespa. I have trained a LightFM model and saved the item embeddings in Vespa for deployment. However, I am facing an issue with prediction accuracy as I can only use the dot product of user and item embeddings for prediction in Vespa. The LightFM model's prediction formula includes item_biases and user_biases, which are crucial for accurate recommendations (user_e . item_e + user_b + item_b). Unfortunately, Vespa does not support entering bias information for user and item embeddings, resulting in reduced prediction accuracy.

I would appreciate any guidance or suggestions on how to train the model without biases or how to collect the information of embeddings and biases together, so that I can make accurate predictions in Vespa. Unfortunately, ignoring biases or concatenating them to embeddings led to worsened prediction accuracy. Thank you for your help.

[Natsuhadder]

Why do you only can use the dot product and not use biases too in Vespa ? As i don't know much about Vespa i don't know how it works. Because you can access the item embeddings and biases by writing the following code : item_biases,item_embeddings=model.get_item_representations(features=item_feature_matrix)