Open shaomai00 opened 3 years ago
if pooling == 'first-last-avg': outputs = [ keras.layers.GlobalAveragePooling1D()(outputs[0]), keras.layers.GlobalAveragePooling1D()(outputs[-1]) ] output = keras.layers.Average()(outputs) elif pooling == 'last-avg': output = keras.layers.GlobalAveragePooling1D()(outputs[-1]) elif pooling == 'cls': output = keras.layers.Lambda(lambda x: x[:, 0])(outputs[-1]) elif pooling == 'pooler': output = bert.output
平时不用keras, 但是查了一下GlobalAveragePooling1D是有mask入参的,这个不加attention_mask进来不是会有问题吗? https://keras.io/api/layers/pooling_layers/global_average_pooling1d/
自带有mask
if pooling == 'first-last-avg': outputs = [ keras.layers.GlobalAveragePooling1D()(outputs[0]), keras.layers.GlobalAveragePooling1D()(outputs[-1]) ] output = keras.layers.Average()(outputs) elif pooling == 'last-avg': output = keras.layers.GlobalAveragePooling1D()(outputs[-1]) elif pooling == 'cls': output = keras.layers.Lambda(lambda x: x[:, 0])(outputs[-1]) elif pooling == 'pooler': output = bert.output
平时不用keras, 但是查了一下GlobalAveragePooling1D是有mask入参的,这个不加attention_mask进来不是会有问题吗? https://keras.io/api/layers/pooling_layers/global_average_pooling1d/