[QST] Target feature for causal masking

[AresDan]

Hi. I have a question regarding target feature for causal masking

Details

I am working on a recommendation system for users who clicks on items. The user has a click history and I want to detect the next item that will be clicked. I figure out that using ">> TagAsItemID()" is responsible for specifying that the following feature is a target feature. So i am using it for user click history feature. However, if I want to detect only the last item that is clicked (list[-1]), how can I specify it? And using causal masking, how exactly transformer identifies the target and which elements it masks in user click history?

[rnyak]

@AresDan we have an example nb for that in Merlin Models if you want to use our Tensorflow API. In this example we separate the last item as a target column named as purchase_id_first and then If you go to Training a Transformer-based Model you will see that we use CategoricalOutput class and we set target column so model knows that it will use the input sequence user click history ( without last item) , and predict the target column.

for your And using causal masking, how exactly transformer identifies the target and which elements it masks in user click history? question, this merged PR explains how CLM works in TF4Rec lib.

[AresDan]

@AresDan we have an example nb for that in Merlin Models if you want to use our Tensorflow API. In this example we separate the last item as a target column named as purchase_id_first and then If you go to Training a Transformer-based Model you will see that we use CategoricalOutput class and we set target column so model knows that it will use the input sequence user click history ( without last item) , and predict the target column.

for your And using causal masking, how exactly transformer identifies the target and which elements it masks in user click history? question, this merged PR explains how CLM works in TF4Rec lib.

Thank you for your answer. You have mentioned that I can use Categorical Output, however, for T4R torch library is used with NextItemPredictionTask. Does this function masks only last element and tries to predict it? But how exactly this function determines the target feature? Because there is not explanation in the documentation.

In addition to that, I work on a problem where I try to recommend items based on items features, click history and user data. Do you recommend some tutorial about T4R which covers all these aspects?

[rnyak]

@AresDan this tutorial is using item features, user-item interaction history (click history in your case) and context features (can be user-data in your case). If you have user data say (location, age, gender, etc.) they are broadcasted using BroadcastToSequence class. Please note that this tutorial is using Merlin Models TF API.

You need to to start from data preprocessing notebooks before you execute the transformer model.

Does this function masks only last element and tries to predict it? But how exactly this function determines the target feature?

we set the target name in the output block:

prediction_task= mm.CategoricalOutput(
    to_call=seq_inputs["categorical"][item_id_name],
    logits_temperature=TEMPERATURE_SCALING,
    target='purchase_id_first',
)