bojone / SimCSE

SimCSE在中文任务上的简单实验
591 stars 83 forks source link

在计算output pooling时不用mask吗? #6

Open shaomai00 opened 3 years ago

shaomai00 commented 3 years ago

if pooling == 'first-last-avg': outputs = [ keras.layers.GlobalAveragePooling1D()(outputs[0]), keras.layers.GlobalAveragePooling1D()(outputs[-1]) ] output = keras.layers.Average()(outputs) elif pooling == 'last-avg': output = keras.layers.GlobalAveragePooling1D()(outputs[-1]) elif pooling == 'cls': output = keras.layers.Lambda(lambda x: x[:, 0])(outputs[-1]) elif pooling == 'pooler': output = bert.output

平时不用keras, 但是查了一下GlobalAveragePooling1D是有mask入参的,这个不加attention_mask进来不是会有问题吗? https://keras.io/api/layers/pooling_layers/global_average_pooling1d/

bojone commented 3 years ago

自带有mask