questions regarding the implementation of mean shift

[cmxcn]

Hi Pan, regarding the code below: def mean_shift(self, input_X, window_radius=0.2):

input_X: batchsizehist_longembdim

    X1 = tf.expand_dims(tf.transpose(input_X, perm=[0,2,1]), 1)
    X2 = tf.expand_dims(input_X, 1)
    C = input_X
    def _mean_shift_step(C):
        C = tf.expand_dims(C, 3)
        Y = tf.reduce_sum(tf.pow((C - X1) / window_radius, 2), axis=2)
        gY = tf.exp(-Y)
        num = tf.reduce_sum(tf.expand_dims(gY, 3) * X2, axis=2)
        denom = tf.reduce_sum(gY, axis=2, keep_dims=True)
        C = num / denom
        return C
    def _mean_shift(i, C, max_diff):
        new_C = _mean_shift_step(C)
        max_diff = tf.reshape(tf.reduce_max(tf.sqrt(tf.reduce_sum(tf.pow(new_C - C, 2), axis=1))), [])
        return i + 1, new_C, max_diff 
    def _cond(i, C, max_diff):
        return max_diff > 1e-5
    n_updates, C , max_diff = tf.while_loop(cond=_cond, body=_mean_shift, loop_vars=(tf.constant(0), C, tf.constant(1e10)))
    return C

I don't quite get the idea how this implement mean shift, especially " C = num/denom" part. Is there any reference materials for it? thanks in advance/

[lpworld]

Hello, thanks for your interest. Please refer to the implementation of mean-shift in this github repository: https://github.com/lephong/tf-meanshift

[cmxcn]

Thanks for the quick reply. I have another problem regarding the calculation of the unexpected factor here:

def unexp_attention(self, querys, keys, keys_id):
    """
    Same Attention as in the DIN model
    queries:     [Batchsize, 1, embedding_size]
    keys:        [Batchsize, max_seq_len, embedding_size]  max_seq_len is the number of keys(e.g. number of clicked creativeid for each sample)
    keys_id:     [Batchsize, max_seq_len]
    """
    querys = tf.expand_dims(querys, 1)
    keys_length = tf.shape(keys)[1] # padded_dim
    embedding_size = querys.get_shape().as_list()[-1]
    keys = tf.reshape(keys, shape=[-1, keys_length, embedding_size])
    querys = tf.reshape(tf.tile(querys, [1, keys_length, 1]), shape=[-1, keys_length, embedding_size])

    net = tf.concat([keys, keys - querys, querys, keys*querys], axis=-1)
    for units in [32,16]:
        net = tf.layers.dense(net, units=units, activation=tf.nn.relu)
    att_wgt = tf.layers.dense(net, units=1, activation=tf.sigmoid)        # shape(batch_size, max_seq_len, 1)
    outputs = tf.reshape(att_wgt, shape=[-1, 1, keys_length], name="weight")  #shape(batch_size, 1, max_seq_len)
    scores = outputs
    scores = scores / (embedding_size ** 0.5)       # scale
    scores = tf.nn.softmax(scores)
    outputs = tf.matmul(scores, keys)    #(batch_size, 1, embedding_size)
    outputs = tf.reduce_sum(outputs, 1, name="unexp_embedding")   #(batch_size, embedding_size)
    return outputs

    #Personalized & Contextualized Unexpected Factor
    unexp_factor = self.unexp_attention(item_emb, unexp_emb, [long_memory_window]*batch_size)
    unexp_factor = tf.layers.batch_normalization(inputs = unexp_factor)
    unexp_factor = tf.reshape(unexp_factor, [-1, hidden_size])
    unexp_factor = tf.layers.dense(unexp_factor, hidden_size)
    unexp_factor = tf.layers.dense(unexp_factor, 1, activation=None)
    #If we choose to use binary values
    #unexp_gate = tf.to_float(tf.reshape(unexp_gate, [-1]) > 0.5)
    unexp_factor = tf.reshape(unexp_factor, [-1])

My question is:

1. why is the item itself included in the unexpected factor calcuation? Could u please provide some scenairos? From this previous paragraph

"For example, some people tend to be ‘’variety-seekers” [31] and are more willing to click on novel item recommendations, while others prefer to stay in their own comfort zones and are in favor of familiar item recommendations. In addition, even the same person might have different perceptions of unexpectedness in different contexts [5], which also motivates us to include session-based information into the design of an unexpected recommender system. For example, it is more reasonable to recommend the next episode of a TV series to the user who has just finished the first episode, instead of recommending new types of videos to that person. On the other hand, if the user has been binge-watching the same TV series in one night, it is better to recommend something different to him or her."

basically this looks as if this factor itself depends only on the user and user's previous behaviors?

2. the second question is how does the unexpected attention reflect the correalation of the item and the user's unexpected willingness?

[lpworld]

Hi,

You second question basically answer your first question; we need this item information to identify the unexpected attention which reflects the correlation of the item and the user's unexpected willingness. It is done through the activation unit mechanism (please refer to the DIN paper: https://arxiv.org/pdf/1706.06978.pdf).

[cmxcn]

Hi Pan, can u please give a more detailed scenario about how the item itself affects the user's expected willingness? thanks

[lpworld]

Hello, you can consider the following recommendation scenario:

View History: "NBA Funny Moments", "Lakers vs Clippers Highlights", "Inside the NBA", "Dear Basketball Kobe Bryant", "The Legend of Lebron James"

Candidate Videos for Recommendation: "10 Secrets about Basketball You Must Know" & "Tips for Street Basketball"

Those two videos have similar unexpectedness value towards the user's view history, but the first video would stimulate more willingness for the user to click on and explore the "unexpectedness".

[cmxcn]

Got it. Thanks for your patient explanation!

---

From: Pan Li notifications@github.com Sent: Friday, December 25, 2020 8:19:18 PM To: lpworld/PURS PURS@noreply.github.com Cc: cmxcn cmxcn@hotmail.com; Author author@noreply.github.com Subject: Re: [lpworld/PURS] questions regarding the implementation of mean shift (#5)

Hello, you can consider the following recommendation scenario:

View History: "NBA Funny Moments", "Lakers vs Clippers Highlights", "Inside the NBA", "Dear Basketball Kobe Bryant", "The Legend of Lebron James"

Candidate Videos for Recommendation: "10 Secrets about Basketball You Must Know" & "Tips for Street Basketball"

Those two videos have similar unexpectedness value towards the user's view history, but the first video would stimulate more willingness for the user to click on and explore the "unexpectedness".

― You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flpworld%2FPURS%2Fissues%2F5%23issuecomment-751241739&data=04%7C01%7C%7C1244834d8ce24e5ebab908d8a8cf4db6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637444955599456302%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zStQZ4GWPoql5YoVZu9UyxDrZNbGlQhq%2BSWZxvc5BCM%3D&reserved=0, or unsubscribehttps://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABGGK4NHN35A6FP6TOAP4Y3SWR7MLANCNFSM4VH5M3NQ&data=04%7C01%7C%7C1244834d8ce24e5ebab908d8a8cf4db6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637444955599456302%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oYWM8qGF2zhShGswf9iNM8BvXAUTjfmkiFfueZXUHD4%3D&reserved=0.