I'm currently using the Transformers4Rec for a Next Item Prediction task. I've successfully pre-trained my model, and now I wish to extract user representation vectors from this model on a user-level basis for further analysis.
Despite my efforts, I've been unable to figure out how to correctly extract the hidden vectors that represent users from the pre-trained model. Could you kindly provide some guidance or instructions on how to accomplish this?
More specifically, I've found that when I inspect the tensor in the body of the 'head' before it enters the 'NextItemPredictionTask' module in the model, it outputs the vector for all elements in the 100-length sequence after padding. My goal, however, is to extract the vector located at the variable-length final position within this sequence.
Any assistance on this matter would be greatly appreciated. Thank you.
❓ Questions & Help
Details
I'm currently using the Transformers4Rec for a Next Item Prediction task. I've successfully pre-trained my model, and now I wish to extract user representation vectors from this model on a user-level basis for further analysis.
Despite my efforts, I've been unable to figure out how to correctly extract the hidden vectors that represent users from the pre-trained model. Could you kindly provide some guidance or instructions on how to accomplish this?
More specifically, I've found that when I inspect the tensor in the body of the 'head' before it enters the 'NextItemPredictionTask' module in the model, it outputs the vector for all elements in the 100-length sequence after padding. My goal, however, is to extract the vector located at the variable-length final position within this sequence.
Any assistance on this matter would be greatly appreciated. Thank you.