search

NVIDIA-Merlin / Transformers4Rec

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation and works with PyTorch.
https://nvidia-merlin.github.io/Transformers4Rec/main
Apache License 2.0
1.07k stars 142 forks source link

[QST] Prediction Output Length Not Matching Input Length #780

Closed dpendleton22 closed 2 months ago

dpendleton22 commented 2 months ago

❓ Questions & Help

Details

This question references following the demo which uses synthetic data: synthetic data demo

I'm trying to understand what is going on with this model and how it works. I got it installed and I'm following the demo where the synthetic data is generated. The demo uses a NextItemPrediction head which I follow and train.

I'm able to serve the model on Triton but when I generate more data to now predict on I can quite understand what's going on. I generated 1k new rows with 966 unique session_ids but the prediction have be back an array of size (34, 500). Why am I only getting back 34 rows of predictions??

I followed the demo exactly so feel free to reference that if you'd like. The demo itself shows in it's example that passing in 1k examples returned 28 predictions:

NUM_ROWS =1000
long_tailed_item_distribution = np.clip(np.random.lognormal(3., 1., int(NUM_ROWS)).astype(np.int32), 1, 50000)
# generate random item interaction features 
df = pd.DataFrame(np.random.randint(70000, 90000, int(NUM_ROWS)), columns=['session_id'])
df['item_id'] = long_tailed_item_distribution

# generate category mapping for each item-id
df['category'] = pd.cut(df['item_id'], bins=334, labels=np.arange(1, 335)).astype(np.int32)
df['age_days'] = np.random.uniform(0, 1, int(NUM_ROWS)).astype(np.float32)
df['weekday_sin']= np.random.uniform(0, 1, int(NUM_ROWS)).astype(np.float32)

# generate day mapping for each session 
map_day = dict(zip(df.session_id.unique(), np.random.randint(1, 10, size=(df.session_id.nunique()))))
df['day'] =  df.session_id.map(map_day)

print(df.head(2))
   session_id  item_id  category  age_days  weekday_sin  day
0       79856        3         2  0.327276     0.080060    2
1       74117        6         4  0.012172     0.147716    1
from merlin.systems.triton.utils import send_triton_request
response = send_triton_request(workflow.input_schema, df, output_schema.column_names, endpoint="localhost:8001")
response

{'next-item': array([[-3.9399953, -2.632081 , -4.2211075, ..., -3.6699016, -3.673493 , -3.1244578], [-3.940445 , -2.6335964, -4.2203593, ..., -3.671566 , -3.6745713, -3.1240335], [-3.9393594, -2.6300201, -4.222065 , ..., -3.6674871, -3.672068 , -3.1251097], ..., [-3.9396427, -2.6304667, -4.2218847, ..., -3.6677885, -3.6724825, -3.1250875], [-3.939829 , -2.6316376, -4.221267 , ..., -3.6693997, -3.6732295, -3.1245873], [-3.9399223, -2.631995 , -4.2210817, ..., -3.669589 , -3.6734715, -3.1244512]], dtype=float32)} response['next-item'].shape (28, 495)

dpendleton22 commented 2 months ago

Answered my own question. There is a minimum sequence length of 2 set when creating the schema so any sessions that do now have more then 1 event will be discarded