Prediction Scores not varying its same for all test users

[Harika-3196]

Hi All,

I'm using lightfm recommendation for usecase where i have to predict user interested app screens for an android app.For this i'm using implicit data - how many times user visited particular screen (count based).There are total 10 screens/items which we need recommend.

I have created train data as first appeared records in intial months and test data has only last 2 weeks records .

My problem is for all users in the test data its predicting same screens/items and even same scores for all the items.

Assuming that model is under fitting; I tried few things

1. Normalizing the visited count of screens -sample weights(tied various ways like subtracting the values from mean,l1,l2)
2. Increased user features

dataset1 = Dataset()
dataset1.fit((interactions['user_id'].values),
            (interactions['firebase_screen'].values),
            user_features=uf)

# plugging in the interactions and their weights
(interactions_csr, weights) = dataset1.build_interactions([(x[0], x[1], x[2]) for x in interactions.values ])
user_id_map, user_feature_map, item_id_map, item_feature_map = dataset1.mapping()

model = LightFM(loss='warp',
                random_state=2016,
                learning_rate=0.08,
                no_components=200,
                user_alpha=0.0000000001)
model.fit(interactions_csr, # spase matrix representing whether user u and item i interacted
      user_features= user_features_csr, # we have built the sparse matrix above
      sample_weight= weights, # spase matrix representing how much value to give to user u and item i inetraction: i.e ratings
      epochs=20,num_threads=2)

precision = precision_at_k(model,interactions_csr,user_features=user_features_csr,k=5).mean()
recall = recall_at_k(model,interactions_csr,user_features=user_features_csr,k=5).mean()

print('Precision at k:',precision)
print('recall at k:',recall)

Precision at k: 0.6130435 recall at k: 0.9521348394430235

_

My total users - 1262 train data users - 976 test data users - 558 no of users who are only in test but not train - 203 (cold start) but we have user features

_

def format_newuser_input_test(user_feature_map, user_feature_list):
  #user_feature_map = user_feature_map  
  num_features = len(user_feature_list)
  normalised_val = 1.0 
  target_indices = []
  for feature in user_feature_list:
    try:
        target_indices.append(user_feature_map[feature])
    except KeyError:
        # print("new user feature encountered '{}'".format(feature))
        pass
  #print("target indices: {}".format(target_indices))
  new_user_features = np.zeros(len(user_feature_map.keys()))
  for i in target_indices:
    new_user_features[i] = normalised_val
  new_user_features = sparse.csr_matrix(new_user_features)
  return(new_user_features)

new_user_features=format_newuser_input_test(user_feature_map,[f1.value,f2.value,f3.value,f4.value,f5.value,f6.value,f7.value,f8.value])
test_res=model.predict(0, np.arange(n_items),user_features=new_user_features)
print(test_res)
[ 1.4126569   0.74451625 -0.68796086 -0.88912046 -0.20068406 -0.37704834
  0.81743217 -1.0723714  -0.80331606 -0.8504304 ]

test_res are same even if i give different features for other users

What are the reasons for same scores what can i do to overcome it.